Enterprise
AWS SageMaker now supports full fine-tuning for LLMs
New serverless capability lets enterprises achieve superior model quality on complex tasks without managing GPU infrastructure.
Key takeaways
- AWS SageMaker AI Serverless Model Customization now supports Full Fine-Tuning, updating all model parameters for superior quality on complex tasks.
- FFT achieved 90.0 percent semantic correctness on code generation with 100K+ examples, outperforming LoRA across BLEU, ROUGE, and other metrics.
- Aggressive learning rates (≥5×10⁻⁵) trigger catastrophic forgetting; safe zone is 2×10⁻⁵ with three epochs maximum and batch size of 8.
- Switching between LoRA and FFT requires changing one parameter; the service handles all infrastructure, compute allocation, and distributed training automatically.
- FFT is not universally better—LoRA can outperform on subjective quality metrics, and choice depends on dataset scale, task complexity, and hyperparameter tolerance.
Amazon Web Services has added Full Fine-Tuning (FFT) support to SageMaker AI Serverless Model Customization, giving enterprises a new path to higher-quality domain-specific models without the operational burden of managing training infrastructure.
The capability addresses a fundamental limitation in parameter-efficient methods like Low-Rank Adaptation (LoRA). While LoRA reduces computational requirements by updating only a small subset of model parameters, its low-rank structure inherently constrains how much complex knowledge a model can absorb. FFT updates all model parameters simultaneously, allowing models to fully capture intricate domain patterns that LoRA cannot represent.
Why it matters
For businesses with large, high-quality datasets and complex domain requirements—such as specialized code generation, technical reasoning, or industry-specific language understanding—the quality ceiling imposed by parameter-efficient methods can directly limit production value. FFT removes that ceiling, and doing so in a serverless environment eliminates the traditional trade-off between model quality and operational complexity.
Performance gains on code generation
AWS tested FFT against LoRA using a 110,000-example coding instruction dataset and the Qwen3-4B base model, according to details first reported on the AWS Machine Learning Blog. At scale, FFT achieved 90.0 percent semantic correctness on an LLM-as-Judge evaluation for code generation tasks. The method showed consistent advantages across multiple metrics including BLEU, ROUGE-1, ROUGE-2, and ROUGE-L scores.
The quality gap widened as dataset size increased. With more than 100,000 training examples, FFT's ability to modify weight matrices uniformly—rather than restricting updates to a constrained subspace—produced measurably stronger convergence and better activation alignment across model layers.
Hyperparameter sensitivity and forgetting risk
FFT's power comes with trade-offs. Because the method modifies every parameter in the model, aggressive hyperparameter choices can trigger catastrophic forgetting, where the model loses general capabilities acquired during pre-training. AWS experiments showed that learning rates at or above 5×10⁻⁵ caused MMLU general knowledge scores to collapse to 3.6 percent and mathematical reasoning accuracy to drop to 2–3 percent.
The research identified a safe configuration zone: a learning rate of 2×10⁻⁵, three training epochs maximum, and a global batch size of 8 preserved general knowledge while maximizing task-specific gains. Extending beyond three epochs degraded performance across all tested variants.
Serverless implementation
Switching between LoRA and FFT in SageMaker requires changing a single parameter in the training configuration. The service handles compute allocation, distributed training setup, and checkpoint management automatically. Models are registered in SageMaker Model Registry as versioned packages immediately after training completes.
AWS ran dozens of ablation experiments across multiple base models—including Qwen3-4B, DeepSeek-R1-1.5B, Nemotron-30B, and Gemma-4-26B—systematically testing learning rate, batch size, warmup ratio, epoch count, and dataset scale. Each experiment completed in hours without infrastructure reconfiguration.
When to choose FFT over LoRA
FFT is not universally superior. In AWS testing, LoRA slightly outperformed FFT on subjective quality assessments (81.5 percent versus 79.5 percent on LLM-as-Judge metrics), and both methods underperformed the base model on exact-match mathematical reasoning tasks. The choice depends on dataset characteristics, task complexity, and tolerance for hyperparameter tuning.
For enterprises with sufficient high-quality training data and requirements for deep domain adaptation, FFT now offers a path to frontier-quality specialized models without the traditional infrastructure overhead.
These details were first reported by AWS on the Machine Learning Blog.