LoRA - AshokBhat/ml GitHub Wiki
About
- LoRA (Low-Rank Adaptation)
- A technique in natural language processing that reduces the number of trainable parameters and GPU memory requirements
- By introducing trainable rank decomposition matrices into each layer of the Transformer architecture
- Allowing for efficient adaptation of pre-trained language models to specific tasks or domains.