methods

Fine-Tuning Methods

QLoRA, LoRA, PEFT, full fine-tuning and preference tuning, explained with the memory maths that decides which one you can run.

LoRA Fine-Tuning: Rank, Alpha and Which Modules to Target How LoRA works, how to pick rank and alpha without guesswork, which projection layers to target, and when a higher rank is a waste of memory. TOFU LoRA vs QLoRA: The Memory Gap, the Quality Gap, and How to Choose LoRA vs QLoRA compared with real numbers: 6.2GB vs 18.4GB for an 8B model, 20-40% slower steps, and a decision table by GPU size. TOFU QLoRA: How 4-Bit Fine-Tuning Fits a 65B Model on One GPU QLoRA explained with the actual memory maths: NF4 quantisation, double quantisation, paged optimisers, what it costs in quality, and when to use LoRA instead. TOFU