Training & Alignment

Parameter-Efficient Fine-Tuning (PEFT)

By Arpit Tripathi, Founder

Parameter-Efficient Fine-Tuning (PEFT) is a family of methods that adapt a large pretrained model to a downstream task by training only a small set of new or selected parameters while keeping most pretrained weights frozen, cutting memory and storage costs versus full fine-tuning.

What PEFT Is and Why It Exists

Parameter-Efficient Fine-Tuning (PEFT) refers to a family of techniques that adapt a large pretrained model to a specific downstream task by updating only a small number of parameters, while the bulk of the original pretrained weights stay frozen. Full fine-tuning, by contrast, updates every weight in the model and produces a complete new copy of the model for each task. As models have grown to billions of parameters, full fine-tuning has become costly in both compute and storage, because each task requires storing a full-size checkpoint and optimizer states for every trainable weight.

PEFT methods break this scaling problem by introducing a small set of trainable parameters, often a fraction of a percent of the base model, and freezing the rest. The frozen base model is shared across every task, and only the small task-specific delta is trained and stored separately. This makes it practical to maintain dozens of task adaptations on top of one base model, and to train large models on a single consumer-grade or modest cloud GPU rather than a large multi-GPU cluster.

The trade-off PEFT accepts is that, in principle, restricting which parameters can change limits the space of functions the adaptation can express. In practice, multiple methods match or come close to full fine-tuning quality on many tasks while training orders of magnitude fewer parameters. The original adapter work reported reaching within 0.4 percent of full fine-tuning on the GLUE benchmark while adding only 3.6 percent parameters per task.

  • Full fine-tuning updates all weights and stores a full model copy per task; PEFT updates a small subset and stores only the delta.
  • The pretrained base model is frozen and shared across every task adaptation.
  • Trainable parameters are often well under 1 percent of the base model.
  • Lower trainable count reduces GPU memory, optimizer state, and per-task storage.

How PEFT Saves Memory and Storage

The savings come from two sources. First, freezing the base weights means the optimizer only tracks state for the small trainable subset. Optimizers such as Adam keep two additional tensors (first and second moment estimates) per trainable parameter, so cutting the trainable count sharply reduces optimizer memory, which is often a dominant cost during training. Second, because the frozen base is identical across tasks, only the small task-specific weights need to be saved per task rather than a full checkpoint.

The scale of these savings is large. The LoRA paper reports reducing the number of trainable parameters by roughly 10,000 times compared with fine-tuning GPT-3 175B using Adam, and cutting GPU memory requirements by about 3 times. In the Hugging Face PEFT documentation, a LoRA adapter trained on a 350M-parameter OPT model produces an adapter file of only a few megabytes, compared with the hundreds of megabytes of the full model.

This separation also helps at serving time. Many task adapters can be stored cheaply and swapped on top of one resident base model. Methods differ in inference behavior: LoRA can merge its learned update back into the base weights so there is no extra inference latency, while soft-prompt methods prepend learned vectors to the input and adapter modules insert extra layers, which add a small amount of computation.

  • Freezing the base removes optimizer state (for example Adam moments) for most parameters.
  • Only the small per-task delta is stored, often a few megabytes versus a full checkpoint.
  • LoRA reported about 10,000 times fewer trainable parameters and about 3 times less GPU memory versus full GPT-3 175B fine-tuning.
  • LoRA can merge into the base for zero added inference latency; soft prompts and adapters add a little compute.

The Main Method Families

Adapters, introduced by Houlsby and colleagues in 2019, insert small trainable bottleneck modules between the existing frozen layers of a Transformer. Each adapter projects the hidden state down to a small dimension, applies a nonlinearity, and projects back up, so only the adapter weights are trained. Low-Rank Adaptation (LoRA), introduced by Hu and colleagues in 2021, instead freezes the original weight matrices and learns a low-rank update added to them, which can be merged at inference.

QLoRA, introduced by Dettmers and colleagues in 2023, extends LoRA by quantizing the frozen base model to 4-bit precision using a data type called 4-bit NormalFloat (NF4), together with double quantization and paged optimizers. The paper reports finetuning a 65B-parameter model on a single 48GB GPU while preserving 16-bit finetuning task performance. Prefix-tuning (Li and Liang, 2021) and prompt-tuning (Lester and colleagues, 2021) are soft-prompt methods that keep the model frozen and instead learn continuous vectors prepended to the input or to each layer.

Two lightweight methods round out the common set. (IA)^3, introduced by Liu and colleagues in 2022, learns to rescale internal activations with a small number of learned vectors and was paired with a recipe called T-Few. BitFit, introduced by Ben-Zaken, Ravfogel, and Goldberg in 2021, trains only the bias terms of the model, an extremely small subset, and was shown to be competitive on smaller datasets.

  • Adapters (Houlsby et al., 2019) insert trainable bottleneck modules between frozen layers.
  • LoRA (Hu et al., 2021) and QLoRA (Dettmers et al., 2023) learn low-rank weight updates, with QLoRA on a 4-bit frozen base.
  • Prefix-tuning (Li and Liang, 2021) and prompt-tuning (Lester et al., 2021) learn soft continuous prompts.
  • (IA)^3 (Liu et al., 2022) rescales activations; BitFit (Ben-Zaken et al., 2021) trains only bias terms.

The LoRA Update in Detail

LoRA is the most widely used PEFT method, so its math is worth stating precisely. For a frozen pretrained weight matrix W_0 of shape d by k, LoRA represents the task-specific change as a low-rank product BA, where B has shape d by r and A has shape r by k, with rank r much smaller than both d and k. During training W_0 is frozen and only A and B are updated; A is typically initialized with small random values and B with zeros, so the adaptation starts as a no-op.

The effective weight used in the forward pass is W_0 plus the scaled low-rank update. Because the update is additive, after training the product BA can be folded directly into W_0, so a merged LoRA model has exactly the same architecture and inference cost as the original. This is why LoRA adds no inference latency once merged, a property that distinguishes it from adapter and soft-prompt methods.

The parameter count of a single LoRA matrix pair is r times (d plus k), compared with d times k for the full matrix. With r in the single or low double digits and d and k in the thousands, this is a reduction of orders of magnitude. The rank r is the central knob: smaller r means fewer trainable parameters and less capacity, larger r approaches full-rank fine-tuning.

W = W_0 + (alpha/r) * B A, B in R^(d x r), A in R^(r x k), r << min(d, k)
The LoRA update adds a scaled low-rank product to the frozen weight W_0; alpha is a fixed scaling factor and r is the rank.
trainable params per matrix = r * (d + k) vs. full = d * k
Each adapted matrix trains r(d+k) parameters instead of d*k, a large reduction when r is much smaller than d and k.
  • W_0 is frozen; only the low-rank factors A and B are trained.
  • B is initialized to zero so the adaptation begins as an identity (no change).
  • The merged weight W_0 + (alpha/r)BA has the same shape and inference cost as the original.
  • Rank r trades capacity against parameter count: smaller r is cheaper but more constrained.

Using PEFT in Practice with the Hugging Face Library

The de facto standard implementation is the open-source Hugging Face peft library, which integrates with the Transformers, Diffusers, and Accelerate libraries. It exposes a configuration class for each method, such as LoraConfig, and a get_peft_model function that wraps a frozen base model and inserts the trainable parameters. The print_trainable_parameters helper reports how small the trainable subset is.

A typical LoRA setup specifies the task type, the rank r, the scaling factor lora_alpha, and a dropout rate, then wraps the loaded base model. The wrapped model is trained with an ordinary training loop or the Transformers Trainer. When saved, the library writes only the extra PEFT weights, so the resulting artifact is a small adapter rather than a full model checkpoint.

At inference, a saved adapter can be reloaded onto the base model, for example through the AutoPeftModelForCausalLM class, which loads both the base model and the trained adapter. Because the base model is shared, multiple adapters can be stored cheaply and applied as needed. The code below shows the minimal LoRA configuration and wrapping pattern from the current library documentation.

python
from peft import LoraConfig, TaskType, get_peft_model
from transformers import AutoModelForSeq2SeqLM

# 1. Load the frozen base model
model = AutoModelForSeq2SeqLM.from_pretrained("bigscience/mt0-large")

# 2. Configure LoRA: rank r, scaling lora_alpha, dropout
peft_config = LoraConfig(
    task_type=TaskType.SEQ_2_SEQ_LM,
    inference_mode=False,
    r=8,
    lora_alpha=32,
    lora_dropout=0.1,
)

# 3. Wrap the base model; only the LoRA weights are trainable
model = get_peft_model(model, peft_config)
model.print_trainable_parameters()
# trainable params: 2359296 || all params: 1231940608 || trainable%: 0.19151...
Minimal Hugging Face peft pattern: build a LoraConfig and wrap a frozen base with get_peft_model; here only about 0.19 percent of parameters are trainable.
  • The Hugging Face peft library is the standard implementation and integrates with Transformers.
  • LoraConfig plus get_peft_model wraps a frozen base and inserts trainable low-rank weights.
  • Saving writes only the small adapter weights, not a full checkpoint.
  • AutoPeftModel classes reload a trained adapter onto the shared base for inference.

Choosing a Method and Its Limits

Method choice depends on constraints. LoRA and QLoRA are the common defaults for large language models because they reach quality close to full fine-tuning, merge cleanly for inference, and QLoRA further reduces memory by training over a 4-bit base. Soft-prompt methods such as prompt-tuning become more competitive as model scale grows, and the prompt-tuning paper found that at billions of parameters it closes the gap with full model tuning. BitFit and (IA)^3 train the fewest parameters and suit tight budgets or smaller datasets.

PEFT does not change what the base model knows; it adapts behavior on top of frozen knowledge. For tasks that require absorbing large amounts of genuinely new information, a low-capacity adapter can underperform full fine-tuning, and increasing the LoRA rank or adapter size narrows the gap at the cost of more parameters. Hyperparameters such as rank, alpha, and which modules are targeted materially affect results and usually need tuning per task.

Because PEFT only adapts and does not erase pretrained behavior, it pairs naturally with later alignment stages. A base model is often first fine-tuned with a PEFT method on instruction data, then aligned with a preference method such as RLHF or DPO. The small, swappable nature of adapters also makes A/B comparison and rollback cheaper than maintaining many full model copies.

  • LoRA and QLoRA are common defaults; QLoRA adds a 4-bit frozen base for extra memory savings.
  • Soft-prompt methods grow more competitive with model scale.
  • Very low-capacity methods can underperform full fine-tuning when much new knowledge is needed.
  • Rank, alpha, and targeted modules are key hyperparameters that need per-task tuning.

Key takeaways

  • PEFT adapts a large model by training a small set of new or selected parameters while freezing most pretrained weights.
  • Freezing the base removes most optimizer state and lets one base model be shared across many task adapters.
  • LoRA learns a low-rank update W = W_0 + (alpha/r)BA with rank r much smaller than the matrix dimensions, and merges into the base for zero added inference latency.
  • A single LoRA matrix trains r(d+k) parameters instead of d*k, often under 1 percent of the model.
  • Method families include adapters, LoRA and QLoRA, prefix-tuning, prompt-tuning, (IA)^3, and BitFit, each with verified origin papers.
  • The Hugging Face peft library, using LoraConfig and get_peft_model, is the standard implementation.

Frequently asked questions

Full fine-tuning updates every weight in the model and saves a complete new checkpoint for each task. PEFT freezes most of the pretrained weights and trains only a small subset of new or selected parameters, so the optimizer tracks far less state and only a small per-task delta is stored. On many tasks PEFT reaches quality close to full fine-tuning at a fraction of the memory and storage cost.
LoRA freezes the original weight matrices and learns a low-rank additive update, keeping the base model in its normal precision. QLoRA, introduced by Dettmers and colleagues in 2023, first quantizes the frozen base model to 4-bit precision using a data type called 4-bit NormalFloat, then trains LoRA adapters on top. The QLoRA paper reports finetuning a 65B-parameter model on a single 48GB GPU while preserving 16-bit finetuning task performance.
No, not after merging. Because the LoRA update is added to the frozen weight, the low-rank product can be folded directly into the original weight matrix once training is finished. The merged model has exactly the same architecture and inference cost as the base model. This distinguishes LoRA from adapter modules and soft-prompt methods, which insert extra layers or vectors that add a small amount of computation.
It varies by method and configuration, but it is typically a small fraction of the base model. The Hugging Face quicktour example trains about 0.19 percent of a 1.2B-parameter model with LoRA. The original adapter paper added 3.6 percent of parameters per task, and BitFit trains only bias terms, an even smaller subset. Lower trainable counts reduce memory but can limit how much new behavior the adaptation can express.
The rank r sets the inner dimension of the low-rank update BA and directly controls how many parameters are trained, which is r times (d plus k) per adapted matrix. A smaller r means fewer trainable parameters and less capacity, while a larger r approaches full-rank fine-tuning at higher cost. Choosing r, along with the scaling factor alpha and which modules to target, is one of the main tuning decisions in LoRA.
The open-source Hugging Face peft library is the de facto standard. It provides a configuration class for each method, such as LoraConfig, and a get_peft_model function that wraps a frozen base model and inserts the trainable parameters. It integrates with the Transformers, Diffusers, and Accelerate libraries and saves only the small adapter weights rather than a full checkpoint.

Put the idea into practice

MemX is an AI memory agent built on these ideas: store anything, skip the folders, and find it again by asking in plain English.

Try MemX Free