MI50 Vulkan (gfx906, ROCM 6.3.3) support patch for ggml

[whoreson]

I hate PRs so here it is. ACEstep compiles and works fine after applying this!

--- a/src/ggml-vulkan/ggml-vulkan.cpp
+++ b/src/ggml-vulkan/ggml-vulkan.cpp
@@ -5501,6 +5501,7 @@ static void ggml_vk_instance_init() {
         extensions.push_back("VK_EXT_debug_utils");
     }
     VkBool32 enable_best_practice = layer_settings;
+#if defined(VK_EXT_layer_settings) && VK_EXT_layer_settings
     std::vector<vk::LayerSettingEXT> settings = {
         {
             "VK_LAYER_KHRONOS_validation",
@@ -5512,6 +5513,11 @@ static void ggml_vk_instance_init() {
     };
     vk::LayerSettingsCreateInfoEXT layer_setting_info(settings);
     vk::InstanceCreateInfo instance_create_info(vk::InstanceCreateFlags{}, &app_info, layers, extensions, &lay
+#else
+    // VK_EXT_layer_settings not available in this SDK version
+    (void)enable_best_practice;
+    vk::InstanceCreateInfo instance_create_info(vk::InstanceCreateFlags{}, &app_info, layers, extensions);
+#endif
 #ifdef __APPLE__
     if (portability_enumeration_ext) {
         instance_create_info.flags |= vk::InstanceCreateFlagBits::eEnumeratePortabilityKHR;

[ServeurpersoCom]

OK, you're the first Vulkan tester, since I don't have the hardware available. I'll integrate your fix and cite your contribution. Then, if you can git pull and test again, that would be perfect :)

[ServeurpersoCom]

Could you share your Vulkan SDK version? (vulkaninfo | grep apiVersion or check VK_HEADER_VERSION in your headers)

[whoreson]

Sure:

GPU1:
VkPhysicalDeviceProperties:
---------------------------
        apiVersion        = 1.3.230 (4206822)
        driverVersion     = 22.3.6 (92286982)
        vendorID          = 0x1002
        deviceID          = 0x66a1
        deviceType        = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
        deviceName        = AMD Radeon Graphics (RADV VEGA20)

[ServeurpersoCom]

1.3.230 ships with the legacy AMD driver for gfx906 and won't be updated. Fix justified

[whoreson]

That being said... Using the SFT model on Vulkan sounds really garbled, vs RTX 4090.

http://www.gabucino.hu/tmp/01.json
http://www.gabucino.hu/tmp/01cuda.mp3
http://www.gabucino.hu/tmp/01vulkan.mp3

Is that the distinction here? (I'm completely new to AceStep btw)

dit-vae --request request0.json --text-encoder Qwen3-Embedding-0.6B-Q8_0.gguf --dit acestep-v15-sft-Q8_0.gguf --vae vae-BF16.gguf

[ServeurpersoCom]

You did what was necessary: ​​I have the reference JSON with the seed and token/codes. There's a loss of precision; I'll see where it's coming from! I need a --dump /path/to/directory/ of the intermediate values, and I'll see at which layer the discrepancy occurs.

[ServeurpersoCom]

I think I've found it, I'll have you test a patch :)

[whoreson]

Sweet. Although I can give you SSH access as well, if you send me a mail at github@gabucino.hu

[ServeurpersoCom]

Done, give a try on the last GGML submodule fork. Two custom ops (col2im_1d and snake) were missing Vulkan backends entirely, so the scheduler was bouncing tensors back and forth between GPU and CPU 41 times per tile. Now everything stays on-device.

Also, since you're on the Turbo model, you don't need 50 DiT inference steps. Turbo is distilled for denoising, after ~8 steps it's hard to hear any difference, though some people push to 12-16. Saves a lot of compute on that MI50.

Here are a few renders with different configs for reference:

Your JSON on my Linux/Disabled-CUDA, full CPU Q8_0 :
Linux-CPU-Q8_0-Reproduction.mp3

Your JSON on my Windows/CUDA with Q4_K_M :
Windows-CUDA-Q4_K_M-Reproduction.mp3

Your JSON also in Q4_K_M but down to 8 inference step :
Windows-CUDA-Q4_K_M-Reproduction-Only-8-Steps.mp3

Hope your Vulkan sings in tune this time!

[ServeurpersoCom]

Sweet. Although I can give you SSH access as well, if you send me a mail at github@gabucino.hu

That could be useful if the DiT diverges, to avoid what I did for Metal by renting an Apple Silicon M2 from Scaleway lol

Please try

[whoreson]

/data4/0/acestep.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function 'void ggml_vk_snake(ggml_backend_vk_context*, vk_context&, const ggml_tensor*, const ggml_tensor*, const ggml_tensor*, ggml_tensor*)':
/data4/0/acestep.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:11521:108: error: cannot bind rvalue reference of type 'vk_op_snake_push_constants&&' to lvalue of type 'vk_op_snake_push_constants'
11521 |     ggml_vk_op_f32<vk_op_snake_push_constants>(ctx, subctx, src0, src1, src2, nullptr, dst, GGML_OP_SNAKE, p);
      |                                                                                                            ^
/data4/0/acestep.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:9628:219: note:   initializing argument 9 of 'void ggml_vk_op_f32(ggml_backend_vk_context*, vk_context&, const ggml_tensor*, const ggml_tensor*, const ggml_tensor*, const ggml_tensor*, ggml_tensor*, ggml_op, PC&&) [with PC = vk_op_snake_push_constants; vk_context = std::shared_ptr<vk_context_struct>]'
 9628 | sor * src0, const ggml_tensor * src1, const ggml_tensor * src2, const ggml_tensor * src3, ggml_tensor * dst, ggml_op op, PC&& pc) {
      |                                                                                                                          ~~~~~^~

/data4/0/acestep.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function 'void ggml_vk_col2im_1d(ggml_backend_vk_context*, vk_context&, const ggml_tensor*, ggml_tensor*)':
/data4/0/acestep.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:11548:122: error: cannot bind rvalue reference of type 'vk_op_col2im_1d_push_constants&&' to lvalue of type 'vk_op_col2im_1d_push_constants'
11548 |     ggml_vk_op_f32<vk_op_col2im_1d_push_constants>(ctx, subctx, src0, nullptr, nullptr, nullptr, dst, GGML_OP_COL2IM_1D, p);
      |                                                                                                                          ^
/data4/0/acestep.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:9628:219: note:   initializing argument 9 of 'void ggml_vk_op_f32(ggml_backend_vk_context*, vk_context&, const ggml_tensor*, const ggml_tensor*, const ggml_tensor*, const ggml_tensor*, ggml_tensor*, ggml_op, PC&&) [with PC = vk_op_col2im_1d_push_constants; vk_context = std::shared_ptr<vk_context_struct>]'
 9628 | sor * src0, const ggml_tensor * src1, const ggml_tensor * src2, const ggml_tensor * src3, ggml_tensor * dst, ggml_op op, PC&& pc) {
      |                                                                                                                          ~~~~~^~

Too many of dem' damn sand sneks!

[ServeurpersoCom]

Too many of dem' damn sand sneks!

ggml: fix Vulkan rvalue ref for col2im_1d and snake push constants

Please git pull --rebase ggml and git pull the parent, and retry:)

Interesting. I installed the Vulkan SDK from https://vulkan.lunarg.com/sdk/home on Windows to try the build and make a universal all-in-one CPU/CUDA/Vulkan binary. here: https://www.serveurperso.com/temp/acestep.cpp-win64/
should let you test the build on your end.

[whoreson]

Ayy I'd never trust git with such complex operations, and it did indeed error out
A clean checkout is more healthy.

http://www.gabucino.hu/tmp/suxx.mp3

I may be a bit drunk, but this doesn't sound right..? And it's suspiciously fast now:

[DiT] Total generation: 18411.9 ms (18411.9 ms/sample)
[VAE] Tiled decode: 44 tiles (chunk=256, overlap=64, stride=128)
[VAE] Graph: 340 nodes, T_latent=192
[VAE] Upsample factor: 1920.00 (expected ~1920)
[VAE] Graph: 340 nodes, T_latent=256
[VAE] Graph: 340 nodes, T_latent=170
[VAE] Tiled decode done: 44 tiles -> T_audio=10771200 (224.40s @ 48kHz)
[VAE Batch0] Decode: 49494.4 ms
[VAE Batch0] Wrote request00.wav: 10771200 samples (224.40s @ 48kHz stereo)
[Request 1/1] Done
[Pipeline] All done

Should be like this:

[DiT] Total generation: 50410.6 ms (50410.6 ms/sample)
[VAE] Tiled decode: 44 tiles (chunk=256, overlap=64, stride=128)
[VAE] Graph: 474 nodes, T_latent=192
[VAE] Upsample factor: 1920.00 (expected ~1920)
[VAE] Graph: 474 nodes, T_latent=256
[VAE] Graph: 474 nodes, T_latent=170
[VAE] Tiled decode done: 44 tiles -> T_audio=10771200 (224.40s @ 48kHz)
[VAE Batch0] Decode: 125576.5 ms
[VAE Batch0] Wrote request00.wav: 10771200 samples (224.40s @ 48kHz stereo)
[Request 1/1] Done
[Pipeline] All done

[VAE Batch0] Decode: 125576.5 ms

Additionally, I think I might have ran a 8-step "base" infer earlier, which is why it sounded bad.

[ServeurpersoCom]

Structured noise !!! I put Vulkan on my local dev container with RTX PRO 6000 :) If I reproduce the same noise, it will be debugged in less than an hour. Else I email you for SSH

[ServeurpersoCom]

sound right..? And it's suspiciously fast now:

No, that's the actual speed gain of fused operators and no more useless CPU<->GPU copy!