What is LoRA (Low-Rank Adaptation)?
An efficient method to fine-tune AI models using much less compute and memory.
Definition
LoRA (Low-Rank Adaptation) is a technique for efficiently fine-tuning large AI models by only modifying a small number of additional parameters rather than updating the entire model. This dramatically reduces the computational cost, memory requirements, and training time needed for customization while achieving results comparable to full fine-tuning.
💡 Example
Instead of fine-tuning all 70 billion parameters of Llama 3, LoRA might only train 10 million additional parameters (0.01%) to adapt the model for medical diagnosis. This can be done on a single GPU instead of requiring a cluster.
Related concepts
Explore AI tools
Find tools that use lora (low-rank adaptation) in practice.
What is LoRA (Low-Rank Adaptation)?
LoRA (Low-Rank Adaptation) is a technique for efficiently fine-tuning large AI models by only modifying a small number of additional parameters rather than updating the entire model. This dramatically reduces the computational cost, memory requirements, and training time needed for customization while achieving results comparable to full fine-tuning.
How does LoRA (Low-Rank Adaptation) work in practice?
Instead of fine-tuning all 70 billion parameters of Llama 3, LoRA might only train 10 million additional parameters (0.01%) to adapt the model for medical diagnosis. This can be done on a single GPU instead of requiring a cluster.