Fine-tuning and post-training
Fine-tuning tools adapt a pretrained language model to a task, domain, or style by continuing training on new examples, through supervised fine-tuning, preference optimization such as DPO, or reinforcement learning with methods such as GRPO. Post-training is the broader term for these steps after pretraining.
Open-source libraries such as Unsloth, Axolotl, TRL, LlamaFactory, and ms-swift run on your own GPUs, often with LoRA or QLoRA adapters that reduce memory needs, while RL frameworks such as verl, OpenRLHF, NeMo RL, and ART target reasoning and agent training. Managed services, including Tinker, Together AI, Fireworks, Amazon Bedrock, and Google Cloud, run jobs on provider infrastructure and usually serve the tuned model too.
The field changed quickly in 2025 and 2026: OpenAI is winding down self-serve fine-tuning, Mistral AI deprecated its fine-tuning API, Databricks ended Foundation Model Fine-tuning, and Predibase and OpenPipe were acquired.
Fine-tuning and post-training compared
| Platform | Open source | Deployment | Best for |
|---|---|---|---|
| Amazon Bedrock model customization | No | Managed cloud, API | Customizing models within AWS governance controls |
| Amazon SageMaker AI | No | Managed cloud | Full control over training infrastructure on AWS |
| ART (Agent Reinforcement Trainer) | Yes (Apache-2.0) | Library | Reinforcement learning for tool-using agents |
| Axolotl | Yes (Apache-2.0) | Library, CLI | Config-driven fine-tuning on multi-GPU clusters |
| Baseten Training | No | Managed cloud, API | Teams already serving models on Baseten |
| CoreWeave Serverless RL and SFT | No | Managed cloud, API | Managed RL for agents without provisioning GPUs |
| DeepSpeed | Yes (Apache-2.0) | Library | Full fine-tuning of models too large for one GPU |
| Fireworks Training | No | Managed cloud, API | Training and serving specialized open models together |
| Kiln | Yes (MIT (libraries); Kiln Desktop EULA (app)) | Desktop app, Library | Building fine-tuning datasets without writing pipelines |
| LitGPT | Yes (Apache-2.0) | Library, CLI | Readable LLM implementations with training recipes |
| LlamaFactory | Yes (Apache-2.0) | Library, CLI | Fine-tuning through a web UI without writing code |
| Microsoft Foundry fine-tuning | No | Managed cloud, API | Fine-tuning OpenAI and open models on Azure |
| Modal Training | No | Managed cloud | Running your own training scripts on serverless GPUs |
| Model tuning (Gemini Enterprise Agent Platform) | No | Managed cloud, API | Customizing Gemini models on Google Cloud |
| ms-swift | Yes (Apache-2.0) | Library, CLI | Tuning Qwen and other open models, including multimodal ones |
| Nebius Token Factory post-training | No | Managed cloud, API | Fine-tuning and serving open models on Nebius |
| NVIDIA NeMo Automodel | Yes (Apache-2.0) | Library | Fine-tuning Hugging Face models on NVIDIA clusters |
| NVIDIA NeMo RL | Yes (Apache-2.0) | Library | Reinforcement learning on NVIDIA GPU clusters |
| OpenAI fine-tuning | No | API, Managed cloud | Existing customers with fine-tuned OpenAI models |
| OpenRLHF | Yes (Apache-2.0) | Library | Ray-based RLHF and agentic RL training |
| Oumi | Yes (Apache-2.0) | Library, CLI, Managed cloud | End-to-end open model training from one toolkit |
| PEFT | Yes (Apache-2.0) | Library | Adding LoRA and other adapters to custom training code |
| Prime Intellect post-training | No | Managed cloud, API | RL post-training of open models on managed compute |
| Tinker | No | API, Managed cloud | Custom training loops without managing clusters |
| Together AI Fine-Tuning | No | Managed cloud, API | Fine-tuning and serving open models on one platform |
| torchtitan | Yes (BSD-3-Clause) | Library | Large-scale PyTorch training and post-training |
| torchtune | Yes (BSD-3-Clause) | Library, CLI | Existing projects built on PyTorch-native recipes |
| TRL | Yes (Apache-2.0) | Library, CLI | Building SFT and RL pipelines on Hugging Face models |
| Unsloth | Yes (Apache-2.0 (Unsloth Studio: AGPL-3.0)) | Library, Desktop app, Local | Fine-tuning open models on a single GPU |
| verl | Yes (Apache-2.0) | Library | Scalable RL post-training for reasoning models |
Profiles

Amazon Bedrock model customization
Amazon Web Services · aws.amazon.com
Amazon Bedrock customizes foundation models through supervised fine-tuning, model distillation, and reinforcement fine-tuning, including open-weight models through OpenAI-compatible APIs, and serves the resulting custom models alongside Bedrock Knowledge Bases and Guardrails.

Amazon SageMaker AI
Amazon Web Services · aws.amazon.com
Amazon SageMaker AI provides serverless model customization, managed training jobs, and HyperPod clusters for fine-tuning open and Amazon Nova models, with JumpStart model catalogs and deployment to SageMaker endpoints or Amazon Bedrock.

ART (Agent Reinforcement Trainer)
OpenPipe (CoreWeave) · art.openpipe.ai
ART is an Apache-2.0 framework for training multi-step agents with GRPO on their own trajectories, using the RULER LLM-as-judge reward function, Unsloth and vLLM under the hood, and CoreWeave Serverless RL as an optional managed backend.
CoreWeave acquired OpenPipe in September 2025; the OpenPipe platform migrated to Weights & Biases and CoreWeave, and ART continues as an open-source project.

Axolotl
Axolotl AI · axolotl.ai
Axolotl is an Apache-2.0 fine-tuning framework driven by YAML configs that supports full fine-tuning, LoRA, QLoRA, DPO, and GRPO on single GPUs or multi-GPU clusters with FSDP2 and DeepSpeed, including multimodal models.

Baseten Training
Baseten · baseten.co
Baseten Training runs fine-tuning jobs on Baseten's managed GPUs with a team's own code and frameworks such as TRL, storing checkpoints that deploy directly to Baseten's inference platform.

CoreWeave Serverless RL and SFT
CoreWeave · coreweave.com
CoreWeave Serverless RL and Serverless SFT are managed post-training services in CoreWeave Forge that use the ART API to train open models from scored agent trajectories or curated examples, while CoreWeave manages GPUs and distributed training.

DeepSpeed
deepspeed.ai
DeepSpeed is an Apache-2.0 optimization library for distributed training whose ZeRO memory partitioning and CPU and NVMe offloading make full fine-tuning of large models possible, and it serves as a backend in Axolotl, LlamaFactory, and Hugging Face Accelerate.
Fireworks Training
Fireworks AI · fireworks.ai
Fireworks Training customizes open models with supervised fine-tuning, DPO, and reinforcement learning, started from the Fireworks app, the API, or a coding agent, and then serves the tuned models on Fireworks inference.

Kiln
Chesterfield Laboratories · kiln.tech
Kiln is a free desktop app and MIT-licensed Python library for building evals, generating synthetic training data, and launching fine-tuning jobs on providers such as Fireworks and Together AI, alongside RAG and prompt optimization.

LitGPT
Lightning AI · github.com
LitGPT is Lightning AI's Apache-2.0 library of from-scratch implementations of more than 20 open LLMs, with command-line recipes to pretrain, fine-tune with LoRA or QLoRA, evaluate, and deploy them.

LlamaFactory
github.com
LlamaFactory is an Apache-2.0 framework for fine-tuning more than 100 language and vision-language models from a CLI or the LlamaBoard web UI, supporting full and freeze tuning, LoRA, QLoRA, and preference methods such as DPO, KTO, and ORPO.
Microsoft Foundry fine-tuning
Microsoft · learn.microsoft.com
Microsoft Foundry offers fine-tuning with supervised fine-tuning, direct preference optimization, and reinforcement fine-tuning for Azure OpenAI models and selected open models such as Llama, Phi, and Mistral, on serverless or managed compute.
Modal Training
Modal · modal.com
Modal runs fine-tuning, reinforcement learning, and evaluation jobs on serverless GPUs that start in seconds, with the infrastructure defined in Python so teams can bring their own training code or open-source frameworks.

Model tuning (Gemini Enterprise Agent Platform)
Google Cloud · docs.cloud.google.com
Google Cloud's Agent Platform tunes Gemini models with supervised fine-tuning on text, documents, images, audio, video, and function calls, along with preference tuning and reinforcement learning fine-tuning, and also tunes open models from Model Garden.
Formerly Vertex AI tuning; Google replaced the Vertex AI brand with Gemini Enterprise Agent Platform in April 2026.

ms-swift
ModelScope (Alibaba) · swift.readthedocs.io
ms-swift is ModelScope's Apache-2.0 framework for continued pretraining, fine-tuning, and reinforcement learning on several hundred text and multimodal models, covering LoRA and full-parameter training, DPO, GRPO, and Megatron-based parallelism.
Nebius Token Factory post-training
Nebius · docs.tokenfactory.nebius.com
Nebius Token Factory includes post-training for open models, with supervised fine-tuning, LoRA adapter merging, custom speculative decoding models, and a Data Lab for preparing datasets, deployed to Token Factory inference endpoints.

NVIDIA NeMo Automodel
NVIDIA · github.com
NeMo Automodel is NVIDIA's Apache-2.0 PyTorch library for supervised fine-tuning, parameter-efficient fine-tuning, and pretraining of Hugging Face language and vision-language models, using native DTensor-based parallelism across GPUs and nodes.

NVIDIA NeMo RL
NVIDIA · docs.nvidia.com
NeMo RL is NVIDIA's Apache-2.0 post-training library for GRPO, DPO, and supervised fine-tuning that scales from one GPU to large clusters with Ray orchestration, Megatron or PyTorch DTensor backends, and vLLM generation.
OpenAI fine-tuning
OpenAI · developers.openai.com
OpenAI's self-serve fine-tuning platform supports supervised, vision, direct preference optimization, and reinforcement fine-tuning jobs on selected OpenAI models through the API and dashboard, with graders for scoring results.
OpenAI is winding down self-serve fine-tuning: new organizations lost access in May 2026, active customers cannot create jobs after January 6, 2027, and tuned models stay available until their base models are deprecated.

OpenRLHF
openrlhf.readthedocs.io
OpenRLHF is an Apache-2.0 reinforcement learning framework built on Ray and vLLM that distributes actor, critic, and reward models across GPUs and supports PPO, REINFORCE++, GRPO, and DAPO for reasoning and agentic post-training.

Oumi
oumi.ai
Oumi is an Apache-2.0 platform for training, evaluating, and deploying open models that covers data synthesis, SFT, LoRA, RL, evaluation, and vLLM inference from one configuration system, with a hosted product that automates model building.

PEFT
Hugging Face · huggingface.co
PEFT is Hugging Face's Apache-2.0 library of parameter-efficient fine-tuning methods, including LoRA, DoRA, prompt tuning, and IA3, which train small adapter weights instead of full models and plug into Transformers, TRL, and Diffusers.

Prime Intellect post-training
Prime Intellect · primeintellect.ai
Prime Intellect runs a platform for post-training, inference, and compute where teams fine-tune open models with SFT and then RL against rewards they define, built around its open-source prime-rl framework and an environments library.

Tinker
Thinking Machines Lab · thinkingmachines.ai
Tinker is Thinking Machines Lab's managed API for fine-tuning open-weight models with LoRA, where users write training loops in Python against low-level primitives while the service runs distributed training; the Apache-2.0 Tinker Cookbook adds SFT and RL recipes.

Together AI Fine-Tuning
Together AI · together.ai
Together AI runs managed supervised fine-tuning and DPO jobs on open models, with LoRA or full fine-tuning, plus a separate post-training service where teams define their own SFT and RL loops on Together's GPU infrastructure.

torchtitan
Meta (PyTorch) · github.com
torchtitan is a PyTorch-native platform for pretraining and post-training generative models at scale that composes FSDP, tensor, pipeline, and context parallelism, with recipes for supervised fine-tuning, LoRA, and reinforcement learning through TitanRL.

torchtune
Meta (PyTorch) · github.com
torchtune is a PyTorch-native post-training library with recipes for full fine-tuning, LoRA, QLoRA, DPO, and knowledge distillation on popular open models, configured through YAML files and launched from its tune command-line tool.
torchtune is no longer actively maintained; development wound down in 2025, and PyTorch is consolidating LLM training in torchtitan.

TRL
Hugging Face · huggingface.co
TRL is Hugging Face's Apache-2.0 library for post-training transformer models, with trainers for supervised fine-tuning, DPO, GRPO, and reward modeling that integrate with PEFT adapters, Accelerate, and vLLM for generation.

Unsloth
Unsloth AI · unsloth.ai
Unsloth is an open-source toolkit for fine-tuning and reinforcement learning on open models with custom GPU kernels that reduce memory use, available as a Python library, the Unsloth Studio web UI, and Unsloth Desktop for local training and inference.

verl
verl.readthedocs.io
verl is an Apache-2.0 reinforcement learning library for LLM post-training, started by ByteDance Seed and based on the HybridFlow design, that runs PPO, GRPO, and DAPO with FSDP or Megatron-LM training and vLLM or SGLang rollouts.
How to choose
- 01Methods
- Supervised fine-tuning, preference optimization such as DPO, and reinforcement learning such as GRPO, with full-parameter or adapter-based training.
- 02Model support
- Which open-weight or proprietary base models can be tuned, including multimodal and mixture-of-experts architectures.
- 03Hardware and scale
- Single-GPU memory savings from quantization and custom kernels versus multi-node parallelism with FSDP, DeepSpeed, or Megatron.
- 04Interface
- Python APIs, YAML configs, command-line tools, web UIs, or hosted APIs that hide the infrastructure.
- 05Ownership and export
- Whether you can download tuned weights or adapters, export formats such as GGUF, and serve the model on another platform.
- 06License and status
- The open-source license, and whether the project or service is still actively maintained, since several tools and APIs were deprecated recently.
Frequently asked questions
What is fine-tuning in LLMs?
- Fine-tuning in LLMs is additional training of a pretrained model on a smaller, targeted dataset so that it follows a format, adopts a style, or performs a narrow task more reliably. It can update all weights or only small adapter layers such as LoRA. Fine-tuning changes model behavior; it is a weak way to add facts that change often, which retrieval handles better.
Fine-tuning vs RAG: which should you use?
- Use RAG when answers depend on knowledge that changes, must be cited, or is too large for a prompt, because retrieval updates without retraining. Use fine-tuning when the model needs a consistent output format, tone, or skill that prompting cannot deliver, or when a smaller tuned model can replace a larger one. Many production systems combine both: a tuned model reading retrieved context.
What is the difference between LoRA and QLoRA?
- LoRA freezes the base model and trains small low-rank adapter matrices added to selected layers, which cuts the memory and storage needed for fine-tuning. QLoRA applies the same adapters while keeping the frozen base model quantized to 4-bit precision, so larger models fit on a single GPU. QLoRA saves more memory, while LoRA on a full-precision base trains somewhat faster.
What is the difference between SFT and RL in post-training?
- Supervised fine-tuning trains a model to imitate example responses, so it needs curated input and output pairs. Reinforcement learning, using methods such as PPO or GRPO, lets the model generate its own attempts and updates it based on a reward, such as a verifier, a test suite, or an LLM judge. Teams often run SFT first, then RL to improve reasoning or agent behavior.
Can you fine-tune OpenAI models?
- Only in limited cases. OpenAI is winding down its self-serve fine-tuning platform: new organizations lost access in May 2026, and active customers can create jobs only until January 6, 2027, while existing fine-tuned models keep serving until their base models are deprecated. Microsoft Foundry continues to offer fine-tuning for selected Azure OpenAI models, and open-weight models such as gpt-oss can be tuned with open-source tools.
Related guides
LLM models and benchmarks hubOther categories
Product details change quickly; check each vendor’s site for current features, licensing and pricing.