---
title: "GPU clouds"
description: "Compare GPU cloud providers for AI, from CoreWeave, Lambda, and Nebius to Runpod, Vast.ai, and the hyperscalers, by hardware, cluster options, and access model."
type: "platform-category"
canonical_url: "https://theaiengineering.com/platforms/gpu-clouds"
markdown_url: "https://theaiengineering.com/platforms/gpu-clouds.md"
topic: "LLM inference and serving"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
citation: "The AI Engineering Editors. \"GPU clouds.\" The AI Engineering. https://theaiengineering.com/platforms/gpu-clouds"
---

# GPU clouds

A GPU cloud is a provider that rents GPUs and other AI accelerators by the hour, the month, or under multi-year contracts, so teams can train, fine-tune, and serve models without buying hardware. Unlike an inference provider, a GPU cloud hands over machines or clusters, and the customer runs the software stack.

The market has three broad groups. Hyperscalers such as AWS, Google Cloud, Microsoft Azure, and Oracle sell GPUs next to their full cloud services. Neoclouds such as CoreWeave, Lambda, Nebius, Crusoe, and Nscale specialize in AI capacity, often with InfiniBand-connected clusters and managed Kubernetes or Slurm. Marketplaces and developer clouds such as Vast.ai, Runpod, SF Compute, and Shadeform aggregate capacity or focus on quick self-serve access.

The right choice usually depends on the accelerator generation a workload needs, how long it needs it, whether jobs span many nodes, and where data must stay. AMD-focused clouds such as TensorWave and Hot Aisle, and European providers such as Verda, Scaleway, and OVHcloud, widen the options further.

## GPU clouds compared

| Platform | Open source | Deployment | Best for |
| --- | --- | --- | --- |
| [AWS (EC2 GPUs and Trainium)](https://aws.amazon.com/ec2/instance-types/accelerated-computing/) | No | Managed cloud, API, CLI | GPU and custom-chip capacity inside AWS |
| [CoreWeave](https://www.coreweave.com) | No | Managed cloud, API | Large NVIDIA clusters with managed Kubernetes |
| [Crusoe](https://www.crusoe.ai) | No | Managed cloud, API | Large clusters from a vertically integrated provider |
| [Denvr](https://www.denvr.com) | No | Managed cloud, API | Canadian-hosted GPU compute |
| [DigitalOcean (GPU Droplets and Paperspace)](https://www.digitalocean.com/products/gpu-droplets) | No | Managed cloud, API, CLI | Simple GPU instances next to a general developer cloud |
| [Fluidstack](https://fluidstack.io) | No | Managed cloud | Frontier-scale dedicated clusters |
| [GMI Cloud](https://www.gmicloud.ai) | No | Managed cloud, API | Moving from inference APIs to dedicated GPU clusters |
| [Google Cloud (GPUs and TPUs)](https://cloud.google.com/gpu) | No | Managed cloud, API, CLI | TPUs and GPUs for teams on Google Cloud |
| [Hot Aisle](https://hotaisle.xyz) | No | Managed cloud, API, CLI | Self-serve AMD Instinct GPUs |
| [Hyperstack](https://www.hyperstack.cloud) | No | Managed cloud, API | On-demand NVIDIA virtual machines and clusters |
| [IREN](https://www.iren.com) | No | Managed cloud | Large GPU clusters in renewable-powered data centers |
| [Jarvislabs](https://jarvislabs.ai) | No | Managed cloud, API | Quick single-GPU instances for experiments |
| [Lambda](https://lambda.com) | No | Managed cloud, API | On-demand NVIDIA instances and reserved clusters |
| [Massed Compute](https://massedcompute.com) | No | Managed cloud, API | Short-term GPU rentals without contracts |
| [Microsoft Azure (AI infrastructure)](https://azure.microsoft.com/en-us/solutions/high-performance-computing/ai-infrastructure/) | No | Managed cloud, API, CLI | GPU clusters for teams standardized on Azure |
| [Nebius](https://nebius.com) | No | Managed cloud, API | Full-stack AI cloud with European and US regions |
| [Nscale](https://www.nscale.com) | No | Managed cloud, API | Sovereign European GPU capacity |
| [NVIDIA DGX Cloud Lepton](https://www.nvidia.com/en-us/data-center/dgx-cloud-lepton/) | No | Managed cloud, API, CLI | Finding NVIDIA capacity across partner clouds |
| [Oracle Cloud Infrastructure (GPUs)](https://www.oracle.com/ai-infrastructure/) | No | Managed cloud, API, CLI | Bare-metal GPU superclusters from a hyperscaler |
| [OVHcloud GPU instances](https://www.ovhcloud.com/en/public-cloud/gpu/) | No | Managed cloud, API | European public cloud GPUs with managed AI tools |
| [Prime Intellect](https://www.primeintellect.ai) | No | Managed cloud, API, CLI | Teams training their own models with reinforcement learning |
| [Radiant (formerly Ori)](https://radiant.co/ori) | No | Managed cloud | Sovereign AI clouds built and operated end to end |
| [Runpod](https://www.runpod.io) | No | Managed cloud, API, CLI | Self-serve GPUs and serverless endpoints for developers |
| [SaladCloud](https://salad.com) | No | Managed cloud, API | Batch and inference jobs on low-cost consumer GPUs |
| [Scaleway GPU Instances](https://www.scaleway.com/en/gpu-instances/) | No | Managed cloud, API | EU-resident GPU instances |
| [SF Compute](https://sfcompute.com) | No | Managed cloud, CLI | Flexible short-term access to large clusters |
| [Shadeform](https://shadeform.com) | No | Managed cloud, API | Comparing and launching GPUs across many clouds |
| [TensorWave](https://tensorwave.com) | No | Managed cloud | AMD Instinct capacity for training and inference |
| [Thunder Compute](https://www.thundercompute.com) | No | Managed cloud, API, CLI | GPU sandboxes for agents and quick developer instances |
| [Vast.ai](https://vast.ai) | No | Managed cloud, API, CLI | Low-cost GPUs from a broad pool of hosts |
| [Verda (formerly DataCrunch)](https://verda.com) | No | Managed cloud, API | European GPU instances and instant clusters |
| [Voltage Park](https://voltagepark.com) | No | Managed cloud | Dedicated H100 and Blackwell clusters |
| [Vultr](https://www.vultr.com/products/cloud-gpu/) | No | Managed cloud, API, CLI | AMD and NVIDIA GPUs across many global regions |

## Profiles

### AWS (EC2 GPUs and Trainium)

Amazon Web Services rents NVIDIA GPUs through EC2 P-series and G-series instances, along with its own Trainium and Inferentia chips, and offers UltraServers and SageMaker HyperPod for large managed training clusters.

- Company: Amazon Web Services
- Website: https://aws.amazon.com/ec2/instance-types/accelerated-computing/
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: GPU and custom-chip capacity inside AWS
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#aws-ec2-gpu

### CoreWeave

CoreWeave is an AI cloud that rents NVIDIA GPU clusters through CoreWeave Kubernetes Service and Slurm, and, after acquiring Weights & Biases, adds experiment tracking, serverless inference, and fine-tuning in its CoreWeave Forge suite.

- Website: https://www.coreweave.com
- GitHub: https://github.com/coreweave
- Headquarters: Livingston, New Jersey, United States
- Founded: 2017
- Open source: No
- Deployment: Managed cloud, API
- Best for: Large NVIDIA clusters with managed Kubernetes
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#coreweave

### Crusoe

Crusoe builds AI data centers with an energy-first approach and runs Crusoe Cloud, which offers NVIDIA and AMD GPUs, managed Kubernetes and Slurm, fault-tolerant AutoClusters, and Managed Inference.

- Website: https://www.crusoe.ai
- Founded: 2018
- Open source: No
- Deployment: Managed cloud, API
- Best for: Large clusters from a vertically integrated provider
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#crusoe

### Denvr

Denvr runs an AI cloud from sovereign data centers in Canada and the United States, offering NVIDIA and Intel Gaudi accelerators as bare metal, VMs, or containers, plus OpenAI-compatible dedicated inference endpoints.

- Website: https://www.denvr.com
- Open source: No
- Deployment: Managed cloud, API
- Best for: Canadian-hosted GPU compute
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#denvr

### DigitalOcean (GPU Droplets and Paperspace)

DigitalOcean offers GPU Droplets and bare-metal GPU servers alongside its AI platform for serverless inference, agents, and knowledge bases, and also runs Paperspace, the GPU cloud it acquired in 2023.

- Company: DigitalOcean
- Website: https://www.digitalocean.com/products/gpu-droplets
- GitHub: https://github.com/digitalocean
- Founded: 2011
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Simple GPU instances next to a general developer cloud
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#digitalocean

### Fluidstack

Fluidstack builds and operates gigawatt-scale AI data centers and large GPU clusters in the United States, focusing on dedicated capacity for large AI training and inference workloads rather than self-serve instances.

- Website: https://fluidstack.io
- Open source: No
- Deployment: Managed cloud
- Best for: Frontier-scale dedicated clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#fluidstack

### GMI Cloud

GMI Cloud is an AI cloud that offers serverless inference for open models, dedicated GPU clusters with RDMA-ready networking, and bare-metal servers on one platform.

- Website: https://www.gmicloud.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Moving from inference APIs to dedicated GPU clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#gmi-cloud

### Google Cloud (GPUs and TPUs)

Google Cloud offers NVIDIA GPUs on Compute Engine and Google Kubernetes Engine, plus its own Tensor Processing Units, packaged in the AI Hypercomputer architecture for training and serving large models.

- Company: Google Cloud
- Website: https://cloud.google.com/gpu
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: TPUs and GPUs for teams on Google Cloud
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#google-cloud-gpu-tpu

### Hot Aisle

Hot Aisle is an AMD-only GPU cloud offering MI300X and MI355X accelerators, from a single-GPU virtual machine to full bare-metal clusters, provisioned through a terminal UI, API, or CLI with per-minute billing.

- Website: https://hotaisle.xyz
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Self-serve AMD Instinct GPUs
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#hot-aisle

### Hyperstack

Hyperstack is the GPU cloud platform of NexGen Cloud, offering on-demand and reserved NVIDIA GPUs as virtual machines and clusters with an API, Kubernetes, and storage for AI training and inference.

- Company: NexGen Cloud
- Website: https://www.hyperstack.cloud
- Open source: No
- Deployment: Managed cloud, API
- Best for: On-demand NVIDIA virtual machines and clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#hyperstack

### IREN

IREN builds data centers powered by renewable energy in British Columbia and Texas and operates IREN AI Cloud, which provides GPU clusters for AI training and inference along with colocation and build-to-suit facilities.

- Website: https://www.iren.com
- Open source: No
- Deployment: Managed cloud
- Best for: Large GPU clusters in renewable-powered data centers
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#iren

### Jarvislabs

Jarvislabs rents NVIDIA GPUs, including H100 and H200, as on-demand instances with per-minute billing and ready-made environments reached through JupyterLab, VS Code, or SSH for training, fine-tuning, and serving models.

- Website: https://jarvislabs.ai
- Open source: No
- Deployment: Managed cloud, API
- Best for: Quick single-GPU instances for experiments
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#jarvislabs

### Lambda

Lambda rents NVIDIA GPUs as on-demand instances and reserved clusters, from single machines to large multi-node deployments, and is winding down its serverless Inference API in favor of models self-hosted on its GPU instances.

- Website: https://lambda.com
- GitHub: https://github.com/lambdal
- Open source: No
- Deployment: Managed cloud, API
- Best for: On-demand NVIDIA instances and reserved clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#lambda

### Massed Compute

Massed Compute rents NVIDIA GPUs, from current data center accelerators to workstation cards, as on-demand virtual machines with CUDA preinstalled, hourly billing, and no long-term contracts, for training, fine-tuning, and inference.

- Website: https://massedcompute.com
- Open source: No
- Deployment: Managed cloud, API
- Best for: Short-term GPU rentals without contracts
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#massed-compute

### Microsoft Azure (AI infrastructure)

Microsoft Azure provides NVIDIA and AMD GPU virtual machines, including the ND-series built for AI training and inference with InfiniBand networking, managed through Azure Kubernetes Service, CycleCloud, and Azure Machine Learning.

- Company: Microsoft
- Website: https://azure.microsoft.com/en-us/solutions/high-performance-computing/ai-infrastructure/
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: GPU clusters for teams standardized on Azure
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#azure-ai-infrastructure

### Nebius

Nebius runs an AI cloud with NVIDIA GPU clusters, managed Kubernetes and Slurm, storage, and the Token Factory inference service, from data centers in Europe and the United States.

- Website: https://nebius.com
- GitHub: https://github.com/nebius
- Headquarters: Amsterdam, Netherlands
- Open source: No
- Deployment: Managed cloud, API
- Best for: Full-stack AI cloud with European and US regions
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#nebius

### Nscale

Nscale, a British AI infrastructure company, runs a full-stack AI cloud from its own data centers in Norway, the United Kingdom, and the United States, offering GPU instances, managed Kubernetes and Slurm, and inference endpoints.

- Website: https://www.nscale.com
- Headquarters: London, United Kingdom
- Founded: 2024
- Open source: No
- Deployment: Managed cloud, API
- Best for: Sovereign European GPU capacity
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#nscale

### NVIDIA DGX Cloud Lepton

NVIDIA DGX Cloud Lepton is a platform that connects developers to GPU capacity from NVIDIA Cloud Partners, GPU marketplaces, and other cloud providers through a single interface for development, training, and inference.

- Company: NVIDIA
- Website: https://www.nvidia.com/en-us/data-center/dgx-cloud-lepton/
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Finding NVIDIA capacity across partner clouds
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#nvidia-dgx-cloud-lepton

### Oracle Cloud Infrastructure (GPUs)

Oracle Cloud Infrastructure rents NVIDIA and AMD GPUs as bare-metal and virtual machine instances and builds OCI Supercluster deployments with RDMA networking for large-scale training and inference.

- Company: Oracle
- Website: https://www.oracle.com/ai-infrastructure/
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Bare-metal GPU superclusters from a hyperscaler
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#oracle-oci-gpu

### OVHcloud GPU instances

OVHcloud, a French cloud provider, offers NVIDIA H200, H100, L40S, and L4 GPU instances in its Public Cloud, along with AI Notebooks, AI Training, and AI Endpoints for managed AI workloads.

- Company: OVHcloud
- Website: https://www.ovhcloud.com/en/public-cloud/gpu/
- Headquarters: Roubaix, France
- Founded: 1999
- Open source: No
- Deployment: Managed cloud, API
- Best for: European public cloud GPUs with managed AI tools
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#ovhcloud-gpu

### Prime Intellect

Prime Intellect pairs GPU compute sourced from multiple providers with open-source tools for distributed and reinforcement learning training, inference, and sandboxes, for teams that train and improve their own models.

- Website: https://www.primeintellect.ai
- GitHub: https://github.com/PrimeIntellect-ai
- Headquarters: San Francisco, United States
- Founded: 2024
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Teams training their own models with reinforcement learning
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#prime-intellect-compute

### Radiant (formerly Ori)

Radiant, which absorbed the London-based GPU cloud Ori, builds and operates AI factories and delivers dedicated bare metal, isolated virtual machines, and managed Kubernetes and Slurm on NVIDIA hardware through its FlightDeck software.

Ori merged with Brookfield-backed Radiant in February 2026, and the Ori platform now continues as part of Radiant.

- Company: Radiant
- Website: https://radiant.co/ori
- Open source: No
- Deployment: Managed cloud
- Best for: Sovereign AI clouds built and operated end to end
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#radiant-ori

### Runpod

Runpod is a GPU cloud for developers that offers Pods for persistent GPU instances, Serverless endpoints that scale to zero, multi-node Clusters, and a Hub for deploying open-source models and templates.

- Website: https://www.runpod.io
- GitHub: https://github.com/runpod
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Self-serve GPUs and serverless endpoints for developers
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#runpod

### SaladCloud

SaladCloud orchestrates containerized AI workloads across a distributed network of idle consumer GPUs, and also offers Salad Dedicated reserved servers and an OpenAI-compatible inference gateway.

- Company: Salad Technologies
- Website: https://salad.com
- GitHub: https://github.com/SaladTechnologies
- Open source: No
- Deployment: Managed cloud, API
- Best for: Batch and inference jobs on low-cost consumer GPUs
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#saladcloud

### Scaleway GPU Instances

Scaleway, a French cloud provider, offers NVIDIA B300, H100, and L4 GPU instances in Paris and Warsaw, with data kept in Europe and pay-as-you-go billing without egress fees.

- Company: Scaleway
- Website: https://www.scaleway.com/en/gpu-instances/
- Headquarters: Paris, France
- Open source: No
- Deployment: Managed cloud, API
- Best for: EU-resident GPU instances
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#scaleway-gpu

### SF Compute

SF Compute is a marketplace for buying and reselling time on large, vetted GPU clusters, from short reservations to longer terms, managed from the command line.

- Company: The San Francisco Compute Company
- Website: https://sfcompute.com
- Open source: No
- Deployment: Managed cloud, CLI
- Best for: Flexible short-term access to large clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#sf-compute

### Shadeform

Shadeform is a GPU cloud marketplace that lets teams find, launch, and manage GPU instances across many cloud providers through one console and API instead of separate accounts.

- Website: https://shadeform.com
- Open source: No
- Deployment: Managed cloud, API
- Best for: Comparing and launching GPUs across many clouds
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#shadeform

### TensorWave

TensorWave is a GPU cloud built entirely on AMD Instinct accelerators, offering dedicated compute for training and inference to teams that want an alternative to NVIDIA GPUs.

- Website: https://tensorwave.com
- Open source: No
- Deployment: Managed cloud
- Best for: AMD Instinct capacity for training and inference
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#tensorwave

### Thunder Compute

Thunder Compute offers persistent GPU instances that developers reach over SSH and short-lived, isolated Ubuntu GPU sandboxes for AI agents that run ML experiments, with per-second and per-minute billing.

- Website: https://www.thundercompute.com
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: GPU sandboxes for agents and quick developer instances
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#thunder-compute

### Vast.ai

Vast.ai is a GPU marketplace where independent hosts and data centers rent out hardware, from consumer cards to data center accelerators, as on-demand or interruptible instances that run Docker containers.

- Website: https://vast.ai
- GitHub: https://github.com/vast-ai
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: Low-cost GPUs from a broad pool of hosts
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#vast-ai

### Verda (formerly DataCrunch)

Verda is a Helsinki-based AI cloud that offers NVIDIA GPU instances, self-service instant clusters with InfiniBand, serverless containers, and a preview inference API for open-weight models.

DataCrunch renamed itself Verda in November 2025, and datacrunch.io now redirects to verda.com.

- Company: Verda
- Website: https://verda.com
- GitHub: https://github.com/verda-cloud
- Headquarters: Helsinki, Finland
- Founded: 2020
- Open source: No
- Deployment: Managed cloud, API
- Best for: European GPU instances and instant clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#verda

### Voltage Park

Voltage Park operates NVIDIA H100 and Blackwell GPU infrastructure in Tier 3+ data centers, offered as on-demand instances, reserved dedicated clusters, bare metal, virtual machines, and managed Kubernetes.

Voltage Park has merged with Lightning AI, and its GPU cloud continues to operate under the Voltage Park name.

- Website: https://voltagepark.com
- Open source: No
- Deployment: Managed cloud
- Best for: Dedicated H100 and Blackwell clusters
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#voltage-park

### Vultr

Vultr is a global cloud provider that offers AMD and NVIDIA GPUs on demand as virtual machines, bare metal, or self-service clusters, alongside its general compute, storage, and Kubernetes services.

- Website: https://www.vultr.com/products/cloud-gpu/
- GitHub: https://github.com/vultr
- Headquarters: West Palm Beach, Florida, United States
- Founded: 2014
- Open source: No
- Deployment: Managed cloud, API, CLI
- Best for: AMD and NVIDIA GPUs across many global regions
- Permalink: https://theaiengineering.com/platforms/gpu-clouds#vultr

## How to choose

1. **Accelerators**: Which NVIDIA, AMD, or custom chips are available, from consumer cards to the newest data center systems, and in what quantities.
2. **Capacity model**: On-demand, spot or interruptible, and reserved contracts each trade price, availability, and commitment differently.
3. **Cluster networking**: Multi-node training and large-model serving need InfiniBand or RDMA networking and topology-aware placement.
4. **Orchestration**: Managed Kubernetes, Slurm, containers, or bare metal determine how much infrastructure work stays with your team.
5. **Regions and sovereignty**: Data center locations, data residency guarantees, and certifications matter for regulated or EU-bound workloads.
6. **Managed AI services**: Storage, inference endpoints, fine-tuning, and observability on the same platform reduce the number of vendors to integrate.

## Frequently asked questions

### What is a GPU cloud provider?

A GPU cloud provider rents GPUs and other AI accelerators as virtual machines, bare-metal servers, or multi-node clusters, billed by the hour or under reserved contracts. Customers install their own drivers, frameworks, and serving software, or use the provider's managed Kubernetes, Slurm, or container services. CoreWeave, Lambda, Nebius, Runpod, and the GPU offerings of AWS, Google Cloud, and Azure are examples.

### What is a neocloud?

A neocloud is a cloud provider that specializes in GPU capacity for AI instead of offering the full catalog of a general-purpose cloud. CoreWeave, Lambda, Nebius, Crusoe, and Nscale are commonly described this way. Neoclouds focus on recent NVIDIA or AMD accelerators, high-bandwidth cluster networking, and AI-specific managed services, and many sell capacity to AI labs under multi-year contracts as well as on demand.

### Neocloud vs hyperscaler: what is the difference?

Hyperscalers such as AWS, Google Cloud, Microsoft Azure, and Oracle sell GPUs alongside databases, identity, networking, and hundreds of other services, which suits teams whose data and applications already run there. Neoclouds concentrate on AI accelerators and often offer simpler access to large clusters. Many organizations use both, for example training on a neocloud and serving inside their primary cloud.

### Should you rent a GPU for LLM inference or use an API?

An inference API is simpler when traffic is variable or the models you need are already hosted, because billing is per token and there are no servers to manage. Renting GPUs makes sense for custom or fine-tuned weights, steady high throughput, full control over data and software, or models that providers do not host. Many teams prototype on an API and move to rented GPUs later.

### Cloud GPU vs local: which should you use?

Local hardware, such as a workstation GPU or an Apple Silicon Mac, suits development, privacy-sensitive experiments, and small quantized models, with no hourly cost after purchase. Cloud GPUs suit training, large models that need data center memory and interconnects, and workloads that must scale or run around the clock. A common pattern is to develop locally with llama.cpp or Ollama and deploy to cloud GPUs.

## Related guides

- [LLM inference optimization: a practical guide for engineers](https://theaiengineering.com/articles/llm-inference-optimization): LLM inference optimization guide: prefill vs decode, TTFT, KV cache and PagedAttention, continuous batching, quantization, speculative decoding, and GPUs.
- [RAG vs fine-tuning: how to choose for your LLM application](https://theaiengineering.com/articles/rag-vs-fine-tuning): RAG vs fine-tuning: RAG adds knowledge at query time, while fine-tuning changes model behavior. Compare freshness, cost, latency, data needs, LoRA, and hybrids.

Product details change quickly; check each vendor's site for current features, licensing and pricing.

## Cite this page

The AI Engineering Editors. "GPU clouds." The AI Engineering. https://theaiengineering.com/platforms/gpu-clouds

- Title: GPU clouds
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- URL: https://theaiengineering.com/platforms/gpu-clouds
- Markdown: https://theaiengineering.com/platforms/gpu-clouds.md
