What Low-Rank Adaptation Really is and Its Applications

Ching (Chingis)
5 min readFeb 10, 2024

This blog piece was inspired by the recent video uploaded by the author of Low-Rank Adaptation, Edward Hu. Low-rank adaptation (LoRA) is one of the most popular methods for fine-tuning LLMs and more!

Edward uploaded a video where he covers

  • What is LoRA?
  • How to choose the rank r?
  • Does LoRA work for my model architecture?
  • Benefits of using LoRA
  • Engineering ideas enabled by LoRA



Ching (Chingis)

I am a passionate student. I enjoy studying and sharing my knowledge. Follow me/Connect with me and join my journey.