What is a system prompt in LLMs?

Also called system message, system instructions, developer message

Definition

A system prompt is the set of developer-written instructions sent to a language model ahead of the conversation, defining its role, rules, output format, and tool use for every turn that follows.

A system prompt is the block of instructions an application places ahead of the conversation to tell a model what role it plays, what it may do, and how to respond. Developers write it once and send it with every request, while user messages change from turn to turn. A typical system prompt states the role, the scope, the constraints, the output format, and when to call tools or hand off to a person. Providers name it differently: Anthropic's Messages API takes a top-level system parameter, and OpenAI's Responses API takes an instructions parameter or messages with the developer role.

How system prompts work#

A system prompt sits at the start of the model's input and takes priority over user messages when the two conflict. It counts against the context window like any other input and is billed on every request, so a long system prompt adds cost and time to first token to each call.

OpenAI's Model Spec formalizes the priority as a chain of command. Rules from the Model Spec and OpenAI come first, then developer instructions, then user instructions. Tool outputs, quoted text, file attachments, and images carry no authority by default, so instructions inside them count as information, not commands.

Whether a system prompt persists across turns depends on the API. OpenAI's text generation guide notes that the instructions parameter applies only to the current request; when a conversation continues with previous_response_id, earlier instructions are not carried over and must be sent again.

System prompt vs user prompt#

The system prompt carries the developer's standing rules, and the user prompt carries the request for the current turn. The system prompt is written in advance, changes only with a release, and is trusted. User prompts arrive at runtime, differ on every request, and must be treated as untrusted input.

System prompt User prompt
Written by The developer The end user or upstream code
Changes With a release Every turn
Priority Higher Lower
Typical content Role, rules, format, tool policy, examples The question or task

System prompt security and caching#

A system prompt is neither a secret nor a security boundary. The OWASP Top 10 for LLM Applications lists system prompt leakage as LLM07:2025 and says the system prompt should not be treated as a secret or used as a security control. Attackers can often infer its rules by using the application.

Instructions in a system prompt also do not stop prompt injection on their own, because injected text reaches the model through the same token stream. Keep credentials and internal data out of the prompt, and enforce permissions in application code, for example by requiring approval before any tool call that writes data.

A stable system prompt is the main beneficiary of prompt caching, which reuses work only for an identical prefix. Anthropic's prompt caching documentation builds the cache in the order tools, system, then messages, and editing the system prompt invalidates the system and message caches. Put timestamps, user names, and other per-request values in the messages that follow, not in the system prompt.

The context engineering explainer shows how the system prompt fits with tools, memory, and retrieved content in an agent's context window.

Cite this definition

The AI Engineering Editors. "What is a system prompt in LLMs?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/system-prompt

Title
What is a system prompt in LLMs?
Author
The AI Engineering Editors
Publisher
The AI Engineering
Updated
View as Markdown

Part of the Prompt and context engineering hub · Updated