Topic hub 06

AI gateways and LLM routing

The control layer between applications and models: routing, keys, caching and policy.

Overview

An AI gateway is a proxy layer that sits between applications and model providers. It gives every application one API while centralizing routing, fallbacks, rate limits, budgets, caching, guardrails and logging.

The guides here explain what gateways do, how routing strategies and semantic caching work, and how to evaluate open source and managed options.

AI gateways and LLM routing guides

Key terms

Full glossary

Tools and platforms

Platform directory

Other topics