Fine-tuning and post-training

Fine-tuning tools adapt a pretrained language model to a task, domain, or style by continuing training on new examples, through supervised fine-tuning, preference optimization such as DPO, or reinforcement learning with methods such as GRPO. Post-training is the broader term for these steps after pretraining.

Open-source libraries such as Unsloth, Axolotl, TRL, LlamaFactory, and ms-swift run on your own GPUs, often with LoRA or QLoRA adapters that reduce memory needs, while RL frameworks such as verl, OpenRLHF, NeMo RL, and ART target reasoning and agent training. Managed services, including Tinker, Together AI, Fireworks, Amazon Bedrock, and Google Cloud, run jobs on provider infrastructure and usually serve the tuned model too.

The field changed quickly in 2025 and 2026: OpenAI is winding down self-serve fine-tuning, Mistral AI deprecated its fine-tuning API, Databricks ended Foundation Model Fine-tuning, and Predibase and OpenPipe were acquired.

Fine-tuning and post-training compared

PlatformOpen sourceDeploymentBest for
Amazon Bedrock model customizationNoManaged cloud, APICustomizing models within AWS governance controls
Amazon SageMaker AINoManaged cloudFull control over training infrastructure on AWS
ART (Agent Reinforcement Trainer)Yes (Apache-2.0)LibraryReinforcement learning for tool-using agents
AxolotlYes (Apache-2.0)Library, CLIConfig-driven fine-tuning on multi-GPU clusters
Baseten TrainingNoManaged cloud, APITeams already serving models on Baseten
CoreWeave Serverless RL and SFTNoManaged cloud, APIManaged RL for agents without provisioning GPUs
DeepSpeedYes (Apache-2.0)LibraryFull fine-tuning of models too large for one GPU
Fireworks TrainingNoManaged cloud, APITraining and serving specialized open models together
KilnYes (MIT (libraries); Kiln Desktop EULA (app))Desktop app, LibraryBuilding fine-tuning datasets without writing pipelines
LitGPTYes (Apache-2.0)Library, CLIReadable LLM implementations with training recipes
LlamaFactoryYes (Apache-2.0)Library, CLIFine-tuning through a web UI without writing code
Microsoft Foundry fine-tuningNoManaged cloud, APIFine-tuning OpenAI and open models on Azure
Modal TrainingNoManaged cloudRunning your own training scripts on serverless GPUs
Model tuning (Gemini Enterprise Agent Platform)NoManaged cloud, APICustomizing Gemini models on Google Cloud
ms-swiftYes (Apache-2.0)Library, CLITuning Qwen and other open models, including multimodal ones
Nebius Token Factory post-trainingNoManaged cloud, APIFine-tuning and serving open models on Nebius
NVIDIA NeMo AutomodelYes (Apache-2.0)LibraryFine-tuning Hugging Face models on NVIDIA clusters
NVIDIA NeMo RLYes (Apache-2.0)LibraryReinforcement learning on NVIDIA GPU clusters
OpenAI fine-tuningNoAPI, Managed cloudExisting customers with fine-tuned OpenAI models
OpenRLHFYes (Apache-2.0)LibraryRay-based RLHF and agentic RL training
OumiYes (Apache-2.0)Library, CLI, Managed cloudEnd-to-end open model training from one toolkit
PEFTYes (Apache-2.0)LibraryAdding LoRA and other adapters to custom training code
Prime Intellect post-trainingNoManaged cloud, APIRL post-training of open models on managed compute
TinkerNoAPI, Managed cloudCustom training loops without managing clusters
Together AI Fine-TuningNoManaged cloud, APIFine-tuning and serving open models on one platform
torchtitanYes (BSD-3-Clause)LibraryLarge-scale PyTorch training and post-training
torchtuneYes (BSD-3-Clause)Library, CLIExisting projects built on PyTorch-native recipes
TRLYes (Apache-2.0)Library, CLIBuilding SFT and RL pipelines on Hugging Face models
UnslothYes (Apache-2.0 (Unsloth Studio: AGPL-3.0))Library, Desktop app, LocalFine-tuning open models on a single GPU
verlYes (Apache-2.0)LibraryScalable RL post-training for reasoning models

Profiles

  • Amazon Bedrock model customization logo

    Amazon Bedrock model customization

    Amazon Web Services · aws.amazon.com

    Amazon Bedrock customizes foundation models through supervised fine-tuning, model distillation, and reinforcement fine-tuning, including open-weight models through OpenAI-compatible APIs, and serves the resulting custom models alongside Bedrock Knowledge Bases and Guardrails.

    • Managed cloud
    • API
  • Amazon SageMaker AI logo

    Amazon SageMaker AI

    Amazon Web Services · aws.amazon.com

    Amazon SageMaker AI provides serverless model customization, managed training jobs, and HyperPod clusters for fine-tuning open and Amazon Nova models, with JumpStart model catalogs and deployment to SageMaker endpoints or Amazon Bedrock.

    • Managed cloud
  • ART (Agent Reinforcement Trainer) logo

    ART (Agent Reinforcement Trainer)

    OpenPipe (CoreWeave) · art.openpipe.ai

    ART is an Apache-2.0 framework for training multi-step agents with GRPO on their own trajectories, using the RULER LLM-as-judge reward function, Unsloth and vLLM under the hood, and CoreWeave Serverless RL as an optional managed backend.

    CoreWeave acquired OpenPipe in September 2025; the OpenPipe platform migrated to Weights & Biases and CoreWeave, and ART continues as an open-source project.

    • Open source · Apache-2.0
    • Library
    • Python
  • Axolotl logo

    Axolotl

    Axolotl AI · axolotl.ai

    Axolotl is an Apache-2.0 fine-tuning framework driven by YAML configs that supports full fine-tuning, LoRA, QLoRA, DPO, and GRPO on single GPUs or multi-GPU clusters with FSDP2 and DeepSpeed, including multimodal models.

    • Open source · Apache-2.0
    • Library
    • CLI
    • Python
  • Baseten Training logo

    Baseten Training

    Baseten · baseten.co

    Baseten Training runs fine-tuning jobs on Baseten's managed GPUs with a team's own code and frameworks such as TRL, storing checkpoints that deploy directly to Baseten's inference platform.

    • Managed cloud
    • API
  • CoreWeave Serverless RL and SFT logo

    CoreWeave Serverless RL and SFT

    CoreWeave · coreweave.com

    CoreWeave Serverless RL and Serverless SFT are managed post-training services in CoreWeave Forge that use the ART API to train open models from scored agent trajectories or curated examples, while CoreWeave manages GPUs and distributed training.

    • Managed cloud
    • API
  • DeepSpeed logo

    DeepSpeed

    deepspeed.ai

    DeepSpeed is an Apache-2.0 optimization library for distributed training whose ZeRO memory partitioning and CPU and NVMe offloading make full fine-tuning of large models possible, and it serves as a backend in Axolotl, LlamaFactory, and Hugging Face Accelerate.

    • Open source · Apache-2.0
    • Library
    • Python
  • Fireworks Training logo

    Fireworks Training

    Fireworks AI · fireworks.ai

    Fireworks Training customizes open models with supervised fine-tuning, DPO, and reinforcement learning, started from the Fireworks app, the API, or a coding agent, and then serves the tuned models on Fireworks inference.

    • Managed cloud
    • API
  • Kiln logo

    Kiln

    Chesterfield Laboratories · kiln.tech

    Kiln is a free desktop app and MIT-licensed Python library for building evals, generating synthetic training data, and launching fine-tuning jobs on providers such as Fireworks and Together AI, alongside RAG and prompt optimization.

    • Open source · MIT (libraries); Kiln Desktop EULA (app)
    • Desktop app
    • Library
    • Python
  • LitGPT logo

    LitGPT

    Lightning AI · github.com

    LitGPT is Lightning AI's Apache-2.0 library of from-scratch implementations of more than 20 open LLMs, with command-line recipes to pretrain, fine-tune with LoRA or QLoRA, evaluate, and deploy them.

    • Open source · Apache-2.0
    • Library
    • CLI
    • Python
  • LlamaFactory logo

    LlamaFactory

    github.com

    LlamaFactory is an Apache-2.0 framework for fine-tuning more than 100 language and vision-language models from a CLI or the LlamaBoard web UI, supporting full and freeze tuning, LoRA, QLoRA, and preference methods such as DPO, KTO, and ORPO.

    • Open source · Apache-2.0
    • Library
    • CLI
    • Python
  • Microsoft Foundry fine-tuning logo

    Microsoft Foundry fine-tuning

    Microsoft · learn.microsoft.com

    Microsoft Foundry offers fine-tuning with supervised fine-tuning, direct preference optimization, and reinforcement fine-tuning for Azure OpenAI models and selected open models such as Llama, Phi, and Mistral, on serverless or managed compute.

    • Managed cloud
    • API
  • Model tuning (Gemini Enterprise Agent Platform) logo

    Model tuning (Gemini Enterprise Agent Platform)

    Google Cloud · docs.cloud.google.com

    Google Cloud's Agent Platform tunes Gemini models with supervised fine-tuning on text, documents, images, audio, video, and function calls, along with preference tuning and reinforcement learning fine-tuning, and also tunes open models from Model Garden.

    Formerly Vertex AI tuning; Google replaced the Vertex AI brand with Gemini Enterprise Agent Platform in April 2026.

    • Managed cloud
    • API
  • ms-swift logo

    ms-swift

    ModelScope (Alibaba) · swift.readthedocs.io

    ms-swift is ModelScope's Apache-2.0 framework for continued pretraining, fine-tuning, and reinforcement learning on several hundred text and multimodal models, covering LoRA and full-parameter training, DPO, GRPO, and Megatron-based parallelism.

    • Open source · Apache-2.0
    • Library
    • CLI
    • Python
  • Nebius Token Factory post-training logo

    Nebius Token Factory post-training

    Nebius · docs.tokenfactory.nebius.com

    Nebius Token Factory includes post-training for open models, with supervised fine-tuning, LoRA adapter merging, custom speculative decoding models, and a Data Lab for preparing datasets, deployed to Token Factory inference endpoints.

    • Managed cloud
    • API
  • NVIDIA NeMo Automodel logo

    NVIDIA NeMo Automodel

    NVIDIA · github.com

    NeMo Automodel is NVIDIA's Apache-2.0 PyTorch library for supervised fine-tuning, parameter-efficient fine-tuning, and pretraining of Hugging Face language and vision-language models, using native DTensor-based parallelism across GPUs and nodes.

    • Open source · Apache-2.0
    • Library
    • Python
  • NVIDIA NeMo RL logo

    NVIDIA NeMo RL

    NVIDIA · docs.nvidia.com

    NeMo RL is NVIDIA's Apache-2.0 post-training library for GRPO, DPO, and supervised fine-tuning that scales from one GPU to large clusters with Ray orchestration, Megatron or PyTorch DTensor backends, and vLLM generation.

    • Open source · Apache-2.0
    • Library
    • Python
  • OpenAI fine-tuning logo

    OpenAI fine-tuning

    OpenAI · developers.openai.com

    OpenAI's self-serve fine-tuning platform supports supervised, vision, direct preference optimization, and reinforcement fine-tuning jobs on selected OpenAI models through the API and dashboard, with graders for scoring results.

    OpenAI is winding down self-serve fine-tuning: new organizations lost access in May 2026, active customers cannot create jobs after January 6, 2027, and tuned models stay available until their base models are deprecated.

    • API
    • Managed cloud
  • OpenRLHF logo

    OpenRLHF

    openrlhf.readthedocs.io

    OpenRLHF is an Apache-2.0 reinforcement learning framework built on Ray and vLLM that distributes actor, critic, and reward models across GPUs and supports PPO, REINFORCE++, GRPO, and DAPO for reasoning and agentic post-training.

    • Open source · Apache-2.0
    • Library
    • Python
  • Oumi logo

    Oumi

    oumi.ai

    Oumi is an Apache-2.0 platform for training, evaluating, and deploying open models that covers data synthesis, SFT, LoRA, RL, evaluation, and vLLM inference from one configuration system, with a hosted product that automates model building.

    • Open source · Apache-2.0
    • Library
    • CLI
    • Managed cloud
    • Python
  • PEFT logo

    PEFT

    Hugging Face · huggingface.co

    PEFT is Hugging Face's Apache-2.0 library of parameter-efficient fine-tuning methods, including LoRA, DoRA, prompt tuning, and IA3, which train small adapter weights instead of full models and plug into Transformers, TRL, and Diffusers.

    • Open source · Apache-2.0
    • Library
    • Python
  • Prime Intellect post-training logo

    Prime Intellect post-training

    Prime Intellect · primeintellect.ai

    Prime Intellect runs a platform for post-training, inference, and compute where teams fine-tune open models with SFT and then RL against rewards they define, built around its open-source prime-rl framework and an environments library.

    • Managed cloud
    • API
  • Tinker logo

    Tinker

    Thinking Machines Lab · thinkingmachines.ai

    Tinker is Thinking Machines Lab's managed API for fine-tuning open-weight models with LoRA, where users write training loops in Python against low-level primitives while the service runs distributed training; the Apache-2.0 Tinker Cookbook adds SFT and RL recipes.

    • API
    • Managed cloud
    • Python
  • Together AI Fine-Tuning logo

    Together AI Fine-Tuning

    Together AI · together.ai

    Together AI runs managed supervised fine-tuning and DPO jobs on open models, with LoRA or full fine-tuning, plus a separate post-training service where teams define their own SFT and RL loops on Together's GPU infrastructure.

    • Managed cloud
    • API
  • torchtitan logo

    torchtitan

    Meta (PyTorch) · github.com

    torchtitan is a PyTorch-native platform for pretraining and post-training generative models at scale that composes FSDP, tensor, pipeline, and context parallelism, with recipes for supervised fine-tuning, LoRA, and reinforcement learning through TitanRL.

    • Open source · BSD-3-Clause
    • Library
    • Python
  • torchtune logo

    torchtune

    Meta (PyTorch) · github.com

    torchtune is a PyTorch-native post-training library with recipes for full fine-tuning, LoRA, QLoRA, DPO, and knowledge distillation on popular open models, configured through YAML files and launched from its tune command-line tool.

    torchtune is no longer actively maintained; development wound down in 2025, and PyTorch is consolidating LLM training in torchtitan.

    • Open source · BSD-3-Clause
    • Library
    • CLI
    • Python
  • TRL logo

    TRL

    Hugging Face · huggingface.co

    TRL is Hugging Face's Apache-2.0 library for post-training transformer models, with trainers for supervised fine-tuning, DPO, GRPO, and reward modeling that integrate with PEFT adapters, Accelerate, and vLLM for generation.

    • Open source · Apache-2.0
    • Library
    • CLI
    • Python
  • Unsloth logo

    Unsloth

    Unsloth AI · unsloth.ai

    Unsloth is an open-source toolkit for fine-tuning and reinforcement learning on open models with custom GPU kernels that reduce memory use, available as a Python library, the Unsloth Studio web UI, and Unsloth Desktop for local training and inference.

    • Open source · Apache-2.0 (Unsloth Studio: AGPL-3.0)
    • Library
    • Desktop app
    • Local
    • Python
  • verl logo

    verl

    verl.readthedocs.io

    verl is an Apache-2.0 reinforcement learning library for LLM post-training, started by ByteDance Seed and based on the HybridFlow design, that runs PPO, GRPO, and DAPO with FSDP or Megatron-LM training and vLLM or SGLang rollouts.

    • Open source · Apache-2.0
    • Library
    • Python

How to choose

01Methods
Supervised fine-tuning, preference optimization such as DPO, and reinforcement learning such as GRPO, with full-parameter or adapter-based training.
02Model support
Which open-weight or proprietary base models can be tuned, including multimodal and mixture-of-experts architectures.
03Hardware and scale
Single-GPU memory savings from quantization and custom kernels versus multi-node parallelism with FSDP, DeepSpeed, or Megatron.
04Interface
Python APIs, YAML configs, command-line tools, web UIs, or hosted APIs that hide the infrastructure.
05Ownership and export
Whether you can download tuned weights or adapters, export formats such as GGUF, and serve the model on another platform.
06License and status
The open-source license, and whether the project or service is still actively maintained, since several tools and APIs were deprecated recently.

Frequently asked questions

What is fine-tuning in LLMs?

Fine-tuning in LLMs is additional training of a pretrained model on a smaller, targeted dataset so that it follows a format, adopts a style, or performs a narrow task more reliably. It can update all weights or only small adapter layers such as LoRA. Fine-tuning changes model behavior; it is a weak way to add facts that change often, which retrieval handles better.

Fine-tuning vs RAG: which should you use?

Use RAG when answers depend on knowledge that changes, must be cited, or is too large for a prompt, because retrieval updates without retraining. Use fine-tuning when the model needs a consistent output format, tone, or skill that prompting cannot deliver, or when a smaller tuned model can replace a larger one. Many production systems combine both: a tuned model reading retrieved context.

What is the difference between LoRA and QLoRA?

LoRA freezes the base model and trains small low-rank adapter matrices added to selected layers, which cuts the memory and storage needed for fine-tuning. QLoRA applies the same adapters while keeping the frozen base model quantized to 4-bit precision, so larger models fit on a single GPU. QLoRA saves more memory, while LoRA on a full-precision base trains somewhat faster.

What is the difference between SFT and RL in post-training?

Supervised fine-tuning trains a model to imitate example responses, so it needs curated input and output pairs. Reinforcement learning, using methods such as PPO or GRPO, lets the model generate its own attempts and updates it based on a reward, such as a verifier, a test suite, or an LLM judge. Teams often run SFT first, then RL to improve reasoning or agent behavior.

Can you fine-tune OpenAI models?

Only in limited cases. OpenAI is winding down its self-serve fine-tuning platform: new organizations lost access in May 2026, and active customers can create jobs only until January 6, 2027, while existing fine-tuned models keep serving until their base models are deprecated. Microsoft Foundry continues to offer fine-tuning for selected Azure OpenAI models, and open-weight models such as gpt-oss can be tuned with open-source tools.

Other categories

Product details change quickly; check each vendor’s site for current features, licensing and pricing.