Not really a bug, more of a discussion - NVIDIA jetpack 5?

[Nurb4000]

I don't think its doable, but just wondering if anyone finds a way to get this to work on an older Jetson running jetpack 5, please post how you pulled it off. Regular llama.cpp will compile and run just fine, so im hopeful...

[ServeurpersoCom]

It has to work, I'm counting on you to try it and keep me informed!

[Nurb4000]

it didn't work on the initial build step. i will grab the actual error later today and post it here. It was related to NVCC not liking the compute level.

I have a couple of 32G NX's and would be nice to be able to use them for this, as they use SOOO Much less electricity than my big zeon + 2 GPUS..

[ServeurpersoCom]

Can you try again with the latest version? We recently broadened the CUDA architecture targets (75-121a), so your Xavier NX (sm_72) is close but might still need sm_72 added explicitly.

Pull the latest and rebuild. If it still fails, post the exact nvcc error and I'll adjust the arch list.

[Nurb4000]

Can you try again with the latest version? We recently broadened the CUDA architecture targets (75-121a), so your Xavier NX (sm_72) is close but might still need sm_72 added explicitly.

Pull the latest and rebuild. If it still fails, post the exact nvcc error and I'll adjust the arch list.

It will be tonight, but i will try again and let you know. Thank you so much.

[Nurb4000]

Updating to most recent and passing the architecture in the command line let it compile. Thank you very much, and my light bill thanks you too. lol :)

( i passed in -> -DCMAKE_CUDA_ARCHITECTURES=72 )

[ServeurpersoCom]

Updating to most recent and passing the architecture in the command line let it compile. Thank you very much, and my light bill thanks you too. lol :)

( i passed in -> -DCMAKE_CUDA_ARCHITECTURES=72 )

Great! Feel free to share some music; it makes for some exotic platform demos :)

[Nurb4000]

Updating to most recent and passing the architecture in the command line let it compile. Thank you very much, and my light bill thanks you too. lol :)
( i passed in -> -DCMAKE_CUDA_ARCHITECTURES=72 )

Great! Feel free to share some music; it makes for some exotic platform demos :)

Just now started testing with the sample batches, its producing the wav file as expected but no audio in them. And size of file does change if i hard code the time ... Im sure its me. Will try a few more in the morning.

[ServeurpersoCom]

Updating to most recent and passing the architecture in the command line let it compile. Thank you very much, and my light bill thanks you too. lol :)
( i passed in -> -DCMAKE_CUDA_ARCHITECTURES=72 )

Great! Feel free to share some music; it makes for some exotic platform demos :)

Just now started testing with the sample batches, its producing the wav file as expected but no audio in them. And size of file does change if i hard code the time ... Im sure its me. Will try a few more in the morning.

Are you using LLM batch or DiT batch processing? Can you share the JSONs from same batch here? We'll see right away if LLM works well. It's DiT; I only do batching on CUDA, so it might be an architecture version support issue.

[Nurb4000]

i had not ventured out on my own yet so i was was using your samples, so the jsons are in your example folder using the shell files. I used both "full" and 'simple', adding duration to expand length on simple ( it was really short without ). Only other change i made to your samples was change to English. ( i noticed it was french when i was reading what the LLM step created for lyrics )

it did say cuda was found, IDed my device ok, and the LLM created lyrics and passed them in. No error messages while the DIT was doing its job , and all seemed like it should be. And VAE decode looked like normal messages too. I can post the entire console if you want on a run, but nothing i read looked like anything went wrong.

EDIT: i went ahead and built it on my desktop ( cpu only ) and ran the same samples, and they worked ok, so something does not like the Jetson it seems )

and ya, ace can do some pretty neat stuff, i have been posting 'albums' out on YouTube last few months. Both 1.0 and 1.5 versions. My external local LLM creates song structure, lyrics, and i feed that into the engine.. it also came up with the song titles, the prompt for stable diffusion for the cover art.. all i do is weed out the garbage, and once in a while add some fade for when it starts off good but sort of goes sideways towards the end.

[Nurb4000]

Using the dit-only sample did produce sounds. So seems to be related to the LLM.

[ServeurpersoCom]

Using the dit-only sample did produce sounds. So seems to be related to the LLM.

Please give me the full log with the problem

[Nurb4000]

Using the dit-only sample did produce sounds. So seems to be related to the LLM.

Please give me the full log with the problem

Console output below ( unless you wanted something different? Too bad we cant attach a file here ) I did try 2 other LLM models, same results. It generated the lyrics, fed it into the DiT, then into VAE, and out came a file of expected size, but zero sound. Reading thru, every step seemed to have been successful.

Tried the 'stock' Q8, the Q6 and then the BF16.

EDIT it cut off my comment, this was using the BF16 LLM. Results same with the other 2.

[Request] parsed simple.json (5 fields)
[Request] seed=-1
  caption:    Upbeat pop rock anthem with driving electric guitars, punchy...
  lyrics:     0 bytes
  bpm=0 dur=220 key= ts= lang=en
  lm: temp=0.85 cfg=2.0 top_p=0.90 top_k=0
  dit: steps=8 guidance=1.0 shift=3.0
  audio_codes: (none)
[BPE] Loaded from GGUF: 151643 vocab, 151387 merges
ggml_cuda_init: found 1 CUDA devices:
  Device 0: Xavier, compute capability 7.2, VMM: yes
[Load] LM backend: CUDA0 (CPU threads: 4)
[GGUF] ../models/acestep-5Hz-lm-0.6B-BF16.gguf: 310 tensors, data at offset 5339424
[LM-Config] 28L, H=1024, V=217204, Nh=16, Nkv=8, D=128, tied=1
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[WeightCtx] Loaded 310 tensors, 1264.5 MB into backend
[LM-Load] CPU embed lookup: type=bf16, row=2048 bytes
[LM-KV] Allocated 2 sets x 28 layers (4D batched), 1792.0 MB
[FSM] Prefix trees: bpm=301, dur=601, key=654, lang=18, tsig=5 nodes
[Fill] lyrics=generate metas=fill gaps | 59 tokens, CFG: 1.00, N=1
[Phase1] Prefill 2509ms, 59 tokens, N=1, CFG=1.00
[Phase1] step 100, 1 active, 17.1 tok/s
[Phase1] step 200, 1 active, 17.3 tok/s
[Phase1] step 300, 1 active, 18.0 tok/s
[Phase1] step 400, 1 active, 18.8 tok/s
[Phase1] Decode 21679ms
[Phase1 Batch0] seed=1720907861961213028, 408 tokens
[Fill Batch0] seed=1720907861961213028:
bpm:115
caption: An upbeat and anthemic pop-rock track driven by a bright, strummed acoustic
duration:160
keyscale:G major
language:en
timesignature:2
</think>

# Lyric
[Intro - Acoustic Guitar Riff]

[Verse 1]
Siiht Vaffeli
Fak nu ma za u
Wen' is we're a bad and you go we will be
Fak ku kehu zeng
Gu a ko bu nia u
Wen' is ke u wa
Lu lu wa, lu lu wa

[Chorus]
Ko ma ja guza
A fai ja tu fa
U wa gu gu la
No no no, no no no
Wen' is ke u wa, lu lu wa
La la la la la
Ko ma ja guza
A fai ja tu fa
U wa gu gu la
No no no no no no no
Wen' is ke u wa, lu lu wa

[Instrumental Break - Acoustic Guitar Riff]

[Verse 2]
Siiht Vaffeli
Fak nu ma za u
Wen' is we're a bad and you go we will be
Fak ku kehu zeng
Gu a ko bu nia u
Wen' is ke u wa
Lu lu wa, lu lu wa

[Chorus]
Ko ma ja guza
A fai ja tu fa
U wa gu gu la
No no no, no no no
Wen' is ke u wa, lu lu wa
La la la la la
Ko ma ja guza
A fai ja tu fa
U wa gu gu la
No no no no no no no
Wen' is ke u wa, lu lu wa

[Outro - Instrumental]
[Acoustic guitar riff and drums continue]
[Song fades out]

[Phase2] N=1, CoT[0]:
bpm: 115
caption: Upbeat pop rock anthem with driving electric guitars, punchy drums, catchy
  vocal hooks, and a singalong chorus
duration: 220
keyscale: G major
language: en
timesignature: 2
[Phase2] max_tokens: 1200, CFG: 2.00, seeds: 1720907861961213028..1720907861961213028
[Phase2] Prefill 1389ms (shared, 1 cond + 1 uncond)
[Decode] step 50, 1 active, 51 total codes, 25.8 tok/s
[Decode] step 100, 1 active, 101 total codes, 26.0 tok/s
[Decode] step 150, 1 active, 151 total codes, 26.0 tok/s
[Decode] step 200, 1 active, 201 total codes, 26.0 tok/s
[Decode] step 250, 1 active, 251 total codes, 26.1 tok/s
[Decode] step 300, 1 active, 301 total codes, 26.1 tok/s
[Decode] step 350, 1 active, 351 total codes, 26.0 tok/s
[Decode] step 400, 1 active, 401 total codes, 26.0 tok/s
[Decode] step 450, 1 active, 451 total codes, 26.0 tok/s
[Decode] step 500, 1 active, 501 total codes, 25.9 tok/s
[Decode] step 550, 1 active, 551 total codes, 25.9 tok/s
[Decode] step 600, 1 active, 601 total codes, 25.8 tok/s
[Decode] step 650, 1 active, 651 total codes, 25.8 tok/s
[Decode] step 700, 1 active, 701 total codes, 25.7 tok/s
[Decode] step 750, 1 active, 751 total codes, 25.7 tok/s
[Decode] step 800, 1 active, 801 total codes, 25.6 tok/s
[Decode] step 850, 1 active, 851 total codes, 25.6 tok/s
[Decode] step 900, 1 active, 901 total codes, 25.5 tok/s
[Decode] step 950, 1 active, 951 total codes, 25.5 tok/s
[Decode] step 1000, 1 active, 1001 total codes, 25.5 tok/s
[Decode] step 1050, 1 active, 1051 total codes, 25.4 tok/s
[Phase2] Decode 43142ms
[Batch 0] seed=1720907861961213028, 1095 codes
[Request] wrote simple0.json
[Output] Wrote simple0.json
[Ace-Qwen3] Load 2105 | Total 71183ms | seed=1720907861961213028
ggml_cuda_init: found 1 CUDA devices:
  Device 0: Xavier, compute capability 7.2, VMM: yes
[Load] DiT backend: CUDA0 (CPU threads: 4)
[Load] Backend init: 30.7 ms
[GGUF] ../models/acestep-v15-turbo-Q8_0.gguf: 678 tensors, data at offset 56864
[DiT] Self-attn: Q+K+V fused
[DiT] Cross-attn: Q+K+V fused
[DiT] MLP: gate+up fused
[Load] null_condition_emb found (CFG available)
[WeightCtx] Loaded 478 tensors, 1600.7 MB into backend
[Load] DiT: 24 layers, H=2048, Nh=16/8, D=128
[Load] DiT weight load: 2057.7 ms
[GGUF] ../models/acestep-v15-turbo-Q8_0.gguf: 678 tensors, data at offset 56864
[Load] silence_latent: [15000, 64] from GGUF
[GGUF] ../models/vae-BF16.gguf: 365 tensors, data at offset 30048
[Load] VAE backend: CUDA0 (CPU threads: 4)
[VAE] Backend: CUDA0, Weight buffer: 161.1 MB
[VAE] Loaded: 5 blocks, upsample=1920x
[Load] VAE weights: 1709.3 ms
[Request 1/1] simple0.json (batch=1)
[Request] parsed simple0.json (17 fields)
[Pipeline] seed=1720907861961213028, steps=8, guidance=1.0, shift=3.0, duration=220.0s
[Pipeline] 1095 audio codes (219.0s @ 5Hz)
[Pipeline] T=5476, S=2738
[BPE] Loaded from GGUF: 151643 vocab, 151387 merges
[Load] BPE tokenizer: 191.0 ms
[Pipeline] caption: 79 tokens, lyrics: 368 tokens
[Load] TextEncoder backend: CUDA0 (CPU threads: 4)
[GGUF] ../models/Qwen3-Embedding-0.6B-Q8_0.gguf: 310 tensors, data at offset 5337664
[Load] TextEncoder: 28L, H=1024, Nh=16/8
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[WeightCtx] Loaded 310 tensors, 742.7 MB into backend
[Load] TextEncoder: 556.8 ms
[Encode] TextEncoder (79 tokens): 1929.3 ms
[GGUF] ../models/Qwen3-Embedding-0.6B-Q8_0.gguf: 310 tensors, data at offset 5337664
[Encode] Lyric vocab lookup (368 tokens): 84.2 ms
[Load] CondEncoder backend: CUDA0 (CPU threads: 4)
[GGUF] ../models/acestep-v15-turbo-Q8_0.gguf: 678 tensors, data at offset 56864
[Load] LyricEncoder: 8L
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[Load] TimbreEncoder: 4L
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[WeightCtx] Loaded 140 tensors, 616.6 MB into backend
[Load] CondEncoder: lyric(8L), timbre(4L), text_proj, null_cond
[Load] ConditionEncoder: 449.5 ms
[CondEnc] Lyric sliding mask: 368x368, window=128
[CondEnc] Timbre sliding mask: 750x750, window=128
[Encode] Packed: lyric=368 + timbre=1 + text=79 = 448 tokens
[Encode] ConditionEncoder: 238.3 ms, enc_S=448
[GGUF] ../models/acestep-v15-turbo-Q8_0.gguf: 678 tensors, data at offset 56864
[WeightCtx] Loaded 30 tensors, 106.5 MB into backend
[Load] Detokenizer: FSQ(6->2048) + 2L encoder(S=5, 2048->64)
[Load] Detokenizer: 230.8 ms
[Context] Decoded: 1095 codes -> 5475 frames (219.0s @ 25Hz)
[Context] Detokenizer: 2870.4 ms
[Context Batch0] Philox noise seed=1720907861961213028, [5476, 64]
[DiT] Starting: T=5476, S=2738, enc_S=448, steps=8, batch=1
[DiT] Batch N=1, T=5476, S=2738, enc_S=448
[DiT] Graph: 1841 nodes
[DiT] step 1/8 t=1.000
[DiT] step 2/8 t=0.955
[DiT] step 3/8 t=0.900
[DiT] step 4/8 t=0.833
[DiT] step 5/8 t=0.750
[DiT] step 6/8 t=0.643
[DiT] step 7/8 t=0.500
[DiT] step 8/8 t=0.300
[DiT] Total generation: 29070.5 ms (29070.5 ms/sample)
[VAE] Tiled decode: 43 tiles (chunk=256, overlap=64, stride=128)
[VAE] Graph: 335 nodes, T_latent=192
[VAE] Upsample factor: 1920.00 (expected ~1920)
[VAE] Graph: 335 nodes, T_latent=256
[VAE] Graph: 335 nodes, T_latent=164
[VAE] Tiled decode done: 43 tiles -> T_audio=10513920 (219.04s @ 48kHz)
[VAE Batch0] Decode: 52528.0 ms
[VAE Batch0] Wrote simple00.wav: 10513920 samples (219.04s @ 48kHz stereo)
[Request 1/1] Done
[Pipeline] All done

[Nurb4000]

Just to add, sorry for being a problem child. I think i could do ok with DiT only since that is now working, but i'm sure this will help out others out there.

[ServeurpersoCom]

Can you send me some JSON with your LLM audiocode + WAV produced with ./dit-only.sh (structured noise vs proper sound) ?
And after :
Can you try the LLM on CPU only? cmake .. -DGGML_CUDA=OFF (explicit)

[Nurb4000]

Can you send me some JSON with your LLM audiocode + WAV produced with ./dit-only.sh (structured noise vs proper sound) ? And after : Can you try the LLM on CPU only? cmake .. -DGGML_CUDA=OFF (explicit)

Yes i can do that this afternoon.. EDIT: Since we cant attach files, ill stick them all out on google drive and drop a link here this evening.