---
title: "Fine-tuning and post-training"
description: "Compare LLM fine-tuning and post-training tools, from Unsloth, Axolotl, and TRL to managed services such as Tinker and Amazon Bedrock, by method and license."
type: "platform-category"
canonical_url: "https://theaiengineering.com/platforms/fine-tuning"
markdown_url: "https://theaiengineering.com/platforms/fine-tuning.md"
topic: "LLM models and benchmarks"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
citation: "The AI Engineering Editors. \"Fine-tuning and post-training.\" The AI Engineering. https://theaiengineering.com/platforms/fine-tuning"
---

# Fine-tuning and post-training

Fine-tuning tools adapt a pretrained language model to a task, domain, or style by continuing training on new examples, through supervised fine-tuning, preference optimization such as DPO, or reinforcement learning with methods such as GRPO. Post-training is the broader term for these steps after pretraining.

Open-source libraries such as Unsloth, Axolotl, TRL, LlamaFactory, and ms-swift run on your own GPUs, often with LoRA or QLoRA adapters that reduce memory needs, while RL frameworks such as verl, OpenRLHF, NeMo RL, and ART target reasoning and agent training. Managed services, including Tinker, Together AI, Fireworks, Amazon Bedrock, and Google Cloud, run jobs on provider infrastructure and usually serve the tuned model too.

The field changed quickly in 2025 and 2026: OpenAI is winding down self-serve fine-tuning, Mistral AI deprecated its fine-tuning API, Databricks ended Foundation Model Fine-tuning, and Predibase and OpenPipe were acquired.

## Fine-tuning and post-training compared

| Platform | Open source | Deployment | Best for |
| --- | --- | --- | --- |
| [Amazon Bedrock model customization](https://aws.amazon.com/bedrock/customize/) | No | Managed cloud, API | Customizing models within AWS governance controls |
| [Amazon SageMaker AI](https://aws.amazon.com/sagemaker/ai/) | No | Managed cloud | Full control over training infrastructure on AWS |
| [ART (Agent Reinforcement Trainer)](https://art.openpipe.ai) | Yes (Apache-2.0) | Library | Reinforcement learning for tool-using agents |
| [Axolotl](https://axolotl.ai) | Yes (Apache-2.0) | Library, CLI | Config-driven fine-tuning on multi-GPU clusters |
| [Baseten Training](https://www.baseten.co/products/training) | No | Managed cloud, API | Teams already serving models on Baseten |
| [CoreWeave Serverless RL and SFT](https://www.coreweave.com/products/coreweave-forge/serverless-rl) | No | Managed cloud, API | Managed RL for agents without provisioning GPUs |
| [DeepSpeed](https://www.deepspeed.ai) | Yes (Apache-2.0) | Library | Full fine-tuning of models too large for one GPU |
| [Fireworks Training](https://fireworks.ai/product/training) | No | Managed cloud, API | Training and serving specialized open models together |
| [Kiln](https://kiln.tech) | Yes (MIT (libraries); Kiln Desktop EULA (app)) | Desktop app, Library | Building fine-tuning datasets without writing pipelines |
| [LitGPT](https://github.com/Lightning-AI/litgpt) | Yes (Apache-2.0) | Library, CLI | Readable LLM implementations with training recipes |
| [LlamaFactory](https://github.com/hiyouga/LlamaFactory) | Yes (Apache-2.0) | Library, CLI | Fine-tuning through a web UI without writing code |
| [Microsoft Foundry fine-tuning](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning) | No | Managed cloud, API | Fine-tuning OpenAI and open models on Azure |
| [Modal Training](https://modal.com/products/training) | No | Managed cloud | Running your own training scripts on serverless GPUs |
| [Model tuning (Gemini Enterprise Agent Platform)](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning) | No | Managed cloud, API | Customizing Gemini models on Google Cloud |
| [ms-swift](https://swift.readthedocs.io/en/latest/) | Yes (Apache-2.0) | Library, CLI | Tuning Qwen and other open models, including multimodal ones |
| [Nebius Token Factory post-training](https://docs.tokenfactory.nebius.com/post-training/overview) | No | Managed cloud, API | Fine-tuning and serving open models on Nebius |
| [NVIDIA NeMo Automodel](https://github.com/NVIDIA-NeMo/Automodel) | Yes (Apache-2.0) | Library | Fine-tuning Hugging Face models on NVIDIA clusters |
| [NVIDIA NeMo RL](https://docs.nvidia.com/nemo/rl/latest/index.html) | Yes (Apache-2.0) | Library | Reinforcement learning on NVIDIA GPU clusters |
| [OpenAI fine-tuning](https://developers.openai.com/api/docs/guides/model-optimization) | No | API, Managed cloud | Existing customers with fine-tuned OpenAI models |
| [OpenRLHF](https://openrlhf.readthedocs.io/en/latest/) | Yes (Apache-2.0) | Library | Ray-based RLHF and agentic RL training |
| [Oumi](https://oumi.ai) | Yes (Apache-2.0) | Library, CLI, Managed cloud | End-to-end open model training from one toolkit |
| [PEFT](https://huggingface.co/docs/peft) | Yes (Apache-2.0) | Library | Adding LoRA and other adapters to custom training code |
| [Prime Intellect post-training](https://www.primeintellect.ai) | No | Managed cloud, API | RL post-training of open models on managed compute |
| [Tinker](https://thinkingmachines.ai/tinker/) | No | API, Managed cloud | Custom training loops without managing clusters |
| [Together AI Fine-Tuning](https://www.together.ai/fine-tuning) | No | Managed cloud, API | Fine-tuning and serving open models on one platform |
| [torchtitan](https://github.com/pytorch/torchtitan) | Yes (BSD-3-Clause) | Library | Large-scale PyTorch training and post-training |
| [torchtune](https://github.com/meta-pytorch/torchtune) | Yes (BSD-3-Clause) | Library, CLI | Existing projects built on PyTorch-native recipes |
| [TRL](https://huggingface.co/docs/trl) | Yes (Apache-2.0) | Library, CLI | Building SFT and RL pipelines on Hugging Face models |
| [Unsloth](https://unsloth.ai) | Yes (Apache-2.0 (Unsloth Studio: AGPL-3.0)) | Library, Desktop app, Local | Fine-tuning open models on a single GPU |
| [verl](https://verl.readthedocs.io/en/latest/) | Yes (Apache-2.0) | Library | Scalable RL post-training for reasoning models |

## Profiles

### Amazon Bedrock model customization

Amazon Bedrock customizes foundation models through supervised fine-tuning, model distillation, and reinforcement fine-tuning, including open-weight models through OpenAI-compatible APIs, and serves the resulting custom models alongside Bedrock Knowledge Bases and Guardrails.

- Company: Amazon Web Services
- Website: https://aws.amazon.com/bedrock/customize/
- Open source: No
- Deployment: Managed cloud, API
- Best for: Customizing models within AWS governance controls
- Permalink: https://theaiengineering.com/platforms/fine-tuning#bedrock-model-customization

### Amazon SageMaker AI

Amazon SageMaker AI provides serverless model customization, managed training jobs, and HyperPod clusters for fine-tuning open and Amazon Nova models, with JumpStart model catalogs and deployment to SageMaker endpoints or Amazon Bedrock.

- Company: Amazon Web Services
- Website: https://aws.amazon.com/sagemaker/ai/
- Open source: No
- Deployment: Managed cloud
- Best for: Full control over training infrastructure on AWS
- Permalink: https://theaiengineering.com/platforms/fine-tuning#sagemaker-ai

### ART (Agent Reinforcement Trainer)

ART is an Apache-2.0 framework for training multi-step agents with GRPO on their own trajectories, using the RULER LLM-as-judge reward function, Unsloth and vLLM under the hood, and CoreWeave Serverless RL as an optional managed backend.

CoreWeave acquired OpenPipe in September 2025; the OpenPipe platform migrated to Weights & Biases and CoreWeave, and ART continues as an open-source project.

- Company: OpenPipe (CoreWeave)
- Website: https://art.openpipe.ai
- GitHub: https://github.com/OpenPipe/ART
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Reinforcement learning for tool-using agents
- Permalink: https://theaiengineering.com/platforms/fine-tuning#openpipe-art

### Axolotl

Axolotl is an Apache-2.0 fine-tuning framework driven by YAML configs that supports full fine-tuning, LoRA, QLoRA, DPO, and GRPO on single GPUs or multi-GPU clusters with FSDP2 and DeepSpeed, including multimodal models.

- Company: Axolotl AI
- Website: https://axolotl.ai
- GitHub: https://github.com/axolotl-ai-cloud/axolotl
- Open source: Yes (Apache-2.0)
- Deployment: Library, CLI
- Languages: Python
- Best for: Config-driven fine-tuning on multi-GPU clusters
- Permalink: https://theaiengineering.com/platforms/fine-tuning#axolotl

### Baseten Training

Baseten Training runs fine-tuning jobs on Baseten's managed GPUs with a team's own code and frameworks such as TRL, storing checkpoints that deploy directly to Baseten's inference platform.

- Company: Baseten
- Website: https://www.baseten.co/products/training
- Open source: No
- Deployment: Managed cloud, API
- Best for: Teams already serving models on Baseten
- Permalink: https://theaiengineering.com/platforms/fine-tuning#baseten-training

### CoreWeave Serverless RL and SFT

CoreWeave Serverless RL and Serverless SFT are managed post-training services in CoreWeave Forge that use the ART API to train open models from scored agent trajectories or curated examples, while CoreWeave manages GPUs and distributed training.

- Company: CoreWeave
- Website: https://www.coreweave.com/products/coreweave-forge/serverless-rl
- Open source: No
- Deployment: Managed cloud, API
- Best for: Managed RL for agents without provisioning GPUs
- Permalink: https://theaiengineering.com/platforms/fine-tuning#coreweave-serverless-rl

### DeepSpeed

DeepSpeed is an Apache-2.0 optimization library for distributed training whose ZeRO memory partitioning and CPU and NVMe offloading make full fine-tuning of large models possible, and it serves as a backend in Axolotl, LlamaFactory, and Hugging Face Accelerate.

- Website: https://www.deepspeed.ai
- GitHub: https://github.com/deepspeedai/DeepSpeed
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Full fine-tuning of models too large for one GPU
- Permalink: https://theaiengineering.com/platforms/fine-tuning#deepspeed

### Fireworks Training

Fireworks Training customizes open models with supervised fine-tuning, DPO, and reinforcement learning, started from the Fireworks app, the API, or a coding agent, and then serves the tuned models on Fireworks inference.

- Company: Fireworks AI
- Website: https://fireworks.ai/product/training
- Open source: No
- Deployment: Managed cloud, API
- Best for: Training and serving specialized open models together
- Permalink: https://theaiengineering.com/platforms/fine-tuning#fireworks-training

### Kiln

Kiln is a free desktop app and MIT-licensed Python library for building evals, generating synthetic training data, and launching fine-tuning jobs on providers such as Fireworks and Together AI, alongside RAG and prompt optimization.

- Company: Chesterfield Laboratories
- Website: https://kiln.tech
- GitHub: https://github.com/Kiln-AI/Kiln
- Open source: Yes (MIT (libraries); Kiln Desktop EULA (app))
- Deployment: Desktop app, Library
- Languages: Python
- Best for: Building fine-tuning datasets without writing pipelines
- Permalink: https://theaiengineering.com/platforms/fine-tuning#kiln

### LitGPT

LitGPT is Lightning AI's Apache-2.0 library of from-scratch implementations of more than 20 open LLMs, with command-line recipes to pretrain, fine-tune with LoRA or QLoRA, evaluate, and deploy them.

- Company: Lightning AI
- Website: https://github.com/Lightning-AI/litgpt
- GitHub: https://github.com/Lightning-AI/litgpt
- Open source: Yes (Apache-2.0)
- Deployment: Library, CLI
- Languages: Python
- Best for: Readable LLM implementations with training recipes
- Permalink: https://theaiengineering.com/platforms/fine-tuning#litgpt

### LlamaFactory

LlamaFactory is an Apache-2.0 framework for fine-tuning more than 100 language and vision-language models from a CLI or the LlamaBoard web UI, supporting full and freeze tuning, LoRA, QLoRA, and preference methods such as DPO, KTO, and ORPO.

- Website: https://github.com/hiyouga/LlamaFactory
- GitHub: https://github.com/hiyouga/LlamaFactory
- Open source: Yes (Apache-2.0)
- Deployment: Library, CLI
- Languages: Python
- Best for: Fine-tuning through a web UI without writing code
- Permalink: https://theaiengineering.com/platforms/fine-tuning#llamafactory

### Microsoft Foundry fine-tuning

Microsoft Foundry offers fine-tuning with supervised fine-tuning, direct preference optimization, and reinforcement fine-tuning for Azure OpenAI models and selected open models such as Llama, Phi, and Mistral, on serverless or managed compute.

- Company: Microsoft
- Website: https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/fine-tuning
- Open source: No
- Deployment: Managed cloud, API
- Best for: Fine-tuning OpenAI and open models on Azure
- Permalink: https://theaiengineering.com/platforms/fine-tuning#foundry-fine-tuning

### Modal Training

Modal runs fine-tuning, reinforcement learning, and evaluation jobs on serverless GPUs that start in seconds, with the infrastructure defined in Python so teams can bring their own training code or open-source frameworks.

- Company: Modal
- Website: https://modal.com/products/training
- Open source: No
- Deployment: Managed cloud
- Languages: Python
- Best for: Running your own training scripts on serverless GPUs
- Permalink: https://theaiengineering.com/platforms/fine-tuning#modal-training

### Model tuning (Gemini Enterprise Agent Platform)

Google Cloud's Agent Platform tunes Gemini models with supervised fine-tuning on text, documents, images, audio, video, and function calls, along with preference tuning and reinforcement learning fine-tuning, and also tunes open models from Model Garden.

Formerly Vertex AI tuning; Google replaced the Vertex AI brand with Gemini Enterprise Agent Platform in April 2026.

- Company: Google Cloud
- Website: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning
- Open source: No
- Deployment: Managed cloud, API
- Best for: Customizing Gemini models on Google Cloud
- Permalink: https://theaiengineering.com/platforms/fine-tuning#google-model-tuning

### ms-swift

ms-swift is ModelScope's Apache-2.0 framework for continued pretraining, fine-tuning, and reinforcement learning on several hundred text and multimodal models, covering LoRA and full-parameter training, DPO, GRPO, and Megatron-based parallelism.

- Company: ModelScope (Alibaba)
- Website: https://swift.readthedocs.io/en/latest/
- GitHub: https://github.com/modelscope/ms-swift
- Open source: Yes (Apache-2.0)
- Deployment: Library, CLI
- Languages: Python
- Best for: Tuning Qwen and other open models, including multimodal ones
- Permalink: https://theaiengineering.com/platforms/fine-tuning#ms-swift

### Nebius Token Factory post-training

Nebius Token Factory includes post-training for open models, with supervised fine-tuning, LoRA adapter merging, custom speculative decoding models, and a Data Lab for preparing datasets, deployed to Token Factory inference endpoints.

- Company: Nebius
- Website: https://docs.tokenfactory.nebius.com/post-training/overview
- Open source: No
- Deployment: Managed cloud, API
- Best for: Fine-tuning and serving open models on Nebius
- Permalink: https://theaiengineering.com/platforms/fine-tuning#nebius-post-training

### NVIDIA NeMo Automodel

NeMo Automodel is NVIDIA's Apache-2.0 PyTorch library for supervised fine-tuning, parameter-efficient fine-tuning, and pretraining of Hugging Face language and vision-language models, using native DTensor-based parallelism across GPUs and nodes.

- Company: NVIDIA
- Website: https://github.com/NVIDIA-NeMo/Automodel
- GitHub: https://github.com/NVIDIA-NeMo/Automodel
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Fine-tuning Hugging Face models on NVIDIA clusters
- Permalink: https://theaiengineering.com/platforms/fine-tuning#nemo-automodel

### NVIDIA NeMo RL

NeMo RL is NVIDIA's Apache-2.0 post-training library for GRPO, DPO, and supervised fine-tuning that scales from one GPU to large clusters with Ray orchestration, Megatron or PyTorch DTensor backends, and vLLM generation.

- Company: NVIDIA
- Website: https://docs.nvidia.com/nemo/rl/latest/index.html
- GitHub: https://github.com/NVIDIA-NeMo/RL
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Reinforcement learning on NVIDIA GPU clusters
- Permalink: https://theaiengineering.com/platforms/fine-tuning#nemo-rl

### OpenAI fine-tuning

OpenAI's self-serve fine-tuning platform supports supervised, vision, direct preference optimization, and reinforcement fine-tuning jobs on selected OpenAI models through the API and dashboard, with graders for scoring results.

OpenAI is winding down self-serve fine-tuning: new organizations lost access in May 2026, active customers cannot create jobs after January 6, 2027, and tuned models stay available until their base models are deprecated.

- Company: OpenAI
- Website: https://developers.openai.com/api/docs/guides/model-optimization
- Open source: No
- Deployment: API, Managed cloud
- Best for: Existing customers with fine-tuned OpenAI models
- Permalink: https://theaiengineering.com/platforms/fine-tuning#openai-fine-tuning

### OpenRLHF

OpenRLHF is an Apache-2.0 reinforcement learning framework built on Ray and vLLM that distributes actor, critic, and reward models across GPUs and supports PPO, REINFORCE++, GRPO, and DAPO for reasoning and agentic post-training.

- Website: https://openrlhf.readthedocs.io/en/latest/
- GitHub: https://github.com/OpenRLHF/OpenRLHF
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Ray-based RLHF and agentic RL training
- Permalink: https://theaiengineering.com/platforms/fine-tuning#openrlhf

### Oumi

Oumi is an Apache-2.0 platform for training, evaluating, and deploying open models that covers data synthesis, SFT, LoRA, RL, evaluation, and vLLM inference from one configuration system, with a hosted product that automates model building.

- Website: https://oumi.ai
- GitHub: https://github.com/oumi-ai/oumi
- Open source: Yes (Apache-2.0)
- Deployment: Library, CLI, Managed cloud
- Languages: Python
- Best for: End-to-end open model training from one toolkit
- Permalink: https://theaiengineering.com/platforms/fine-tuning#oumi

### PEFT

PEFT is Hugging Face's Apache-2.0 library of parameter-efficient fine-tuning methods, including LoRA, DoRA, prompt tuning, and IA3, which train small adapter weights instead of full models and plug into Transformers, TRL, and Diffusers.

- Company: Hugging Face
- Website: https://huggingface.co/docs/peft
- GitHub: https://github.com/huggingface/peft
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Adding LoRA and other adapters to custom training code
- Permalink: https://theaiengineering.com/platforms/fine-tuning#peft

### Prime Intellect post-training

Prime Intellect runs a platform for post-training, inference, and compute where teams fine-tune open models with SFT and then RL against rewards they define, built around its open-source prime-rl framework and an environments library.

- Company: Prime Intellect
- Website: https://www.primeintellect.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: RL post-training of open models on managed compute
- Permalink: https://theaiengineering.com/platforms/fine-tuning#prime-intellect-post-training

### Tinker

Tinker is Thinking Machines Lab's managed API for fine-tuning open-weight models with LoRA, where users write training loops in Python against low-level primitives while the service runs distributed training; the Apache-2.0 Tinker Cookbook adds SFT and RL recipes.

- Company: Thinking Machines Lab
- Website: https://thinkingmachines.ai/tinker/
- Open source: No
- Deployment: API, Managed cloud
- Languages: Python
- Best for: Custom training loops without managing clusters
- Permalink: https://theaiengineering.com/platforms/fine-tuning#tinker

### Together AI Fine-Tuning

Together AI runs managed supervised fine-tuning and DPO jobs on open models, with LoRA or full fine-tuning, plus a separate post-training service where teams define their own SFT and RL loops on Together's GPU infrastructure.

- Company: Together AI
- Website: https://www.together.ai/fine-tuning
- Open source: No
- Deployment: Managed cloud, API
- Best for: Fine-tuning and serving open models on one platform
- Permalink: https://theaiengineering.com/platforms/fine-tuning#together-fine-tuning

### torchtitan

torchtitan is a PyTorch-native platform for pretraining and post-training generative models at scale that composes FSDP, tensor, pipeline, and context parallelism, with recipes for supervised fine-tuning, LoRA, and reinforcement learning through TitanRL.

- Company: Meta (PyTorch)
- Website: https://github.com/pytorch/torchtitan
- GitHub: https://github.com/pytorch/torchtitan
- Open source: Yes (BSD-3-Clause)
- Deployment: Library
- Languages: Python
- Best for: Large-scale PyTorch training and post-training
- Permalink: https://theaiengineering.com/platforms/fine-tuning#torchtitan

### torchtune

torchtune is a PyTorch-native post-training library with recipes for full fine-tuning, LoRA, QLoRA, DPO, and knowledge distillation on popular open models, configured through YAML files and launched from its tune command-line tool.

torchtune is no longer actively maintained; development wound down in 2025, and PyTorch is consolidating LLM training in torchtitan.

- Company: Meta (PyTorch)
- Website: https://github.com/meta-pytorch/torchtune
- GitHub: https://github.com/meta-pytorch/torchtune
- Open source: Yes (BSD-3-Clause)
- Deployment: Library, CLI
- Languages: Python
- Best for: Existing projects built on PyTorch-native recipes
- Permalink: https://theaiengineering.com/platforms/fine-tuning#torchtune

### TRL

TRL is Hugging Face's Apache-2.0 library for post-training transformer models, with trainers for supervised fine-tuning, DPO, GRPO, and reward modeling that integrate with PEFT adapters, Accelerate, and vLLM for generation.

- Company: Hugging Face
- Website: https://huggingface.co/docs/trl
- GitHub: https://github.com/huggingface/trl
- Open source: Yes (Apache-2.0)
- Deployment: Library, CLI
- Languages: Python
- Best for: Building SFT and RL pipelines on Hugging Face models
- Permalink: https://theaiengineering.com/platforms/fine-tuning#trl

### Unsloth

Unsloth is an open-source toolkit for fine-tuning and reinforcement learning on open models with custom GPU kernels that reduce memory use, available as a Python library, the Unsloth Studio web UI, and Unsloth Desktop for local training and inference.

- Company: Unsloth AI
- Website: https://unsloth.ai
- GitHub: https://github.com/unslothai/unsloth
- Open source: Yes (Apache-2.0 (Unsloth Studio: AGPL-3.0))
- Deployment: Library, Desktop app, Local
- Languages: Python
- Best for: Fine-tuning open models on a single GPU
- Permalink: https://theaiengineering.com/platforms/fine-tuning#unsloth

### verl

verl is an Apache-2.0 reinforcement learning library for LLM post-training, started by ByteDance Seed and based on the HybridFlow design, that runs PPO, GRPO, and DAPO with FSDP or Megatron-LM training and vLLM or SGLang rollouts.

- Website: https://verl.readthedocs.io/en/latest/
- GitHub: https://github.com/verl-project/verl
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: Scalable RL post-training for reasoning models
- Permalink: https://theaiengineering.com/platforms/fine-tuning#verl

## How to choose

1. **Methods**: Supervised fine-tuning, preference optimization such as DPO, and reinforcement learning such as GRPO, with full-parameter or adapter-based training.
2. **Model support**: Which open-weight or proprietary base models can be tuned, including multimodal and mixture-of-experts architectures.
3. **Hardware and scale**: Single-GPU memory savings from quantization and custom kernels versus multi-node parallelism with FSDP, DeepSpeed, or Megatron.
4. **Interface**: Python APIs, YAML configs, command-line tools, web UIs, or hosted APIs that hide the infrastructure.
5. **Ownership and export**: Whether you can download tuned weights or adapters, export formats such as GGUF, and serve the model on another platform.
6. **License and status**: The open-source license, and whether the project or service is still actively maintained, since several tools and APIs were deprecated recently.

## Frequently asked questions

### What is fine-tuning in LLMs?

Fine-tuning in LLMs is additional training of a pretrained model on a smaller, targeted dataset so that it follows a format, adopts a style, or performs a narrow task more reliably. It can update all weights or only small adapter layers such as LoRA. Fine-tuning changes model behavior; it is a weak way to add facts that change often, which retrieval handles better.

### Fine-tuning vs RAG: which should you use?

Use RAG when answers depend on knowledge that changes, must be cited, or is too large for a prompt, because retrieval updates without retraining. Use fine-tuning when the model needs a consistent output format, tone, or skill that prompting cannot deliver, or when a smaller tuned model can replace a larger one. Many production systems combine both: a tuned model reading retrieved context.

### What is the difference between LoRA and QLoRA?

LoRA freezes the base model and trains small low-rank adapter matrices added to selected layers, which cuts the memory and storage needed for fine-tuning. QLoRA applies the same adapters while keeping the frozen base model quantized to 4-bit precision, so larger models fit on a single GPU. QLoRA saves more memory, while LoRA on a full-precision base trains somewhat faster.

### What is the difference between SFT and RL in post-training?

Supervised fine-tuning trains a model to imitate example responses, so it needs curated input and output pairs. Reinforcement learning, using methods such as PPO or GRPO, lets the model generate its own attempts and updates it based on a reward, such as a verifier, a test suite, or an LLM judge. Teams often run SFT first, then RL to improve reasoning or agent behavior.

### Can you fine-tune OpenAI models?

Only in limited cases. OpenAI is winding down its self-serve fine-tuning platform: new organizations lost access in May 2026, and active customers can create jobs only until January 6, 2027, while existing fine-tuned models keep serving until their base models are deprecated. Microsoft Foundry continues to offer fine-tuning for selected Azure OpenAI models, and open-weight models such as gpt-oss can be tuned with open-source tools.

## Related guides

- [RAG vs fine-tuning: how to choose for your LLM application](https://theaiengineering.com/articles/rag-vs-fine-tuning): RAG vs fine-tuning: RAG adds knowledge at query time, while fine-tuning changes model behavior. Compare freshness, cost, latency, data needs, LoRA, and hybrids.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.

Product details change quickly; check each vendor's site for current features, licensing and pricing.

## Cite this page

The AI Engineering Editors. "Fine-tuning and post-training." The AI Engineering. https://theaiengineering.com/platforms/fine-tuning

- Title: Fine-tuning and post-training
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- URL: https://theaiengineering.com/platforms/fine-tuning
- Markdown: https://theaiengineering.com/platforms/fine-tuning.md
