Skip to content

WIP: LoRA loading feature#15

Closed
lmangani wants to merge 3 commits intoServeurpersoCom:masterfrom
audiohacking:copilot/port-lora-loading-feature
Closed

WIP: LoRA loading feature#15
lmangani wants to merge 3 commits intoServeurpersoCom:masterfrom
audiohacking:copilot/port-lora-loading-feature

Conversation

@lmangani
Copy link
Contributor

@lmangani lmangani commented Mar 7, 2026

This pull request adds LoRA (Low-Rank Adaptation) support to the dit-vae pipeline, enabling users to apply PEFT LoRA adapters in safetensors format for more flexible and efficient model fine-tuning. The changes include new code for loading LoRA adapters, modifications to the DiT model to support LoRA at key projection points, and updated documentation and examples to demonstrate usage.

LoRA integration and model changes:

  • Added a new source file, src/dit-lora.cpp, which implements loading LoRA adapters from PEFT-compatible safetensors files and integrates them into the DiT model by mapping adapter weights to the appropriate layers and projections.
  • Modified the DiT model graph in src/dit-graph.h to support LoRA at all major linear projections (self-attention, cross-attention, and MLP layers) by introducing the dit_ggml_linear_lora helper and updating all relevant computation paths to optionally add LoRA contributions. [1] [2] [3] [4] [5] [6] [7]

Build system and CLI updates:

  • Updated CMakeLists.txt to include src/dit-lora.cpp when building the dit-vae executable, enabling LoRA support in the main pipeline.

Documentation and usage examples:

  • Expanded README.md with a new section describing LoRA support, usage instructions for the new --lora and --lora-scale options, and example commands and JSON inputs.
  • Added examples/lora.sh and examples/lora.json to provide a fully working example of using a LoRA adapter in the pipeline. [1] [2]

@lmangani lmangani marked this pull request as ready for review March 7, 2026 11:55
@ServeurpersoCom ServeurpersoCom force-pushed the master branch 3 times, most recently from 3f0515d to b99bb72 Compare March 9, 2026 08:16
@lmangani
Copy link
Contributor Author

closing as already elegantly implemented by @ServeurpersoCom

@lmangani lmangani closed this Mar 10, 2026
@lmangani lmangani deleted the copilot/port-lora-loading-feature branch March 13, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants