[Question] 现在支持rollout和actor的共卡场景了吗？代码里有共卡，但是似乎不是这个场景

[PrometheusComing]

Question

想确定下现在支持rollout和actor的共卡场景了吗？代码里有共卡，但是似乎不是这个场景.如果没支持，未来会有支持共卡on policy的打算吗？求告知

Question Scope

• Algorithm
• System
• Documentation
• Code Design
• Others

Additional Context (Optional)

[Add any additional context, code examples, or screenshots that might help]

[garrett4wade]

目前不支持，目前实现共卡的优先级比较低，因为训练会block住推理，在online serving这种场景不太适用。

共卡的主要问题是要支持权重同步。现在训练/推理的offload都是写好的（但是在trainer没有调用），主要是要把现在纯nccl的解决方案替换成nccl+cuda ipc/update_from_tensor的混合方案，因为同一张卡的两个进程没有办法进行nccl通信。这些解决方案在verl/slime/awex中都有实现，现在我们核心团队没有人去做这件事情。

如果有兴趣，欢迎给我们contribute这个feature～

[NJX-njx]

Hi @garrett4wade! I'd like to work on this colocation feature.

Plan (following your guidance):

1. Implement a \ColocationWeightSync\ module that uses CUDA IPC / \update_from_tensor\ for same-GPU weight transfer between training and inference processes
2. Replace the pure NCCL weight sync path with NCCL + CUDA IPC hybrid when colocation is detected
3. Wire up the existing \offload()/\unload()\ infrastructure in the trainer to support colocation mode scheduling
4. Add configuration support via CLI args for enabling colocation mode

I'll reference verl/slime/awex for the CUDA IPC weight sync pattern. Submitting an initial PR with the core infrastructure!

Note: This is a framework-level change that requires multi-GPU testing for full validation I'll include the architecture and tests I can run locally, and note what needs GPU cluster validation.