Messages

When you call an LLM API, you don't send a single string of text. You send a structured list of messages, and each message has a role and some content. The shape of that list decides how the model behaves, what it remembers and how much each request costs.

The three roles#

Every message carries one of three roles. Together they describe who is speaking.

A typical request has the system message first, then alternating user and assistant turns, then the latest user message at the bottom.

As data, that list looks like this.

messages = [
    {"role": "system", "content": "You are a support agent for Acme Corp."},
    {"role": "user", "content": "How do I reset my password?"},
    {"role": "assistant", "content": "Open Settings, then choose Reset."},
    # The newest user message always goes last.
    {"role": "user", "content": "And if I no longer have my old email?"},
]
reply = call_model(messages)  # generic helper: sends the list, returns text

What the system prompt controls#

The system prompt defines the model's behavior for your application. It does not add knowledge. It shapes how the model acts.

Typical uses:

• The persona or tone to use.
• Which topics to stay on or avoid.
• The format to respond in.
• What to do when the model doesn't know something.

For example, a customer support bot might have this system prompt: "You are a support agent for Acme Corp. Answer only questions about our products. If you don't know, say so and offer to escalate. Always respond in three sentences or fewer."

Every request includes the system prompt, even though the user never sees it. That is how the model behaves consistently across a whole conversation.

Why the model has no memory#

The model keeps nothing between requests. Each time you send one, it starts completely fresh and knows only what is in the message list you send with it.

That is why your app resends the whole conversation every time. The alternating user and assistant turns are the model's only way to know what was said before. Without them, every message would look like the start of a new conversation. The next note on stateless design goes deeper on this.

Why the order matters#

The model reads the messages from top to bottom, and later messages tend to carry more weight. If the latest user turn contradicts the system prompt, the model often sides with the more recent instruction. This is also why the latest user message always goes last.

The cost of resending#

Resending the history means a long conversation is expensive. Suppose the chat history is 5,000 tokens and the user sends a 10-token message. You pay for 5,000 + 10 = 5,010 tokens of input, not 10.

Every turn adds to the history, so every turn adds to the bill. Tokens are how that cost is counted (see Tokens), and the history also has to fit inside the limit described in Context window.

Short line#

An API request is a list of system, user and assistant messages, sent in full every time because the model has no memory. The system prompt shapes behavior, and it is not a place for secrets.

Key takeaways#

• A request is a structured list of messages, each with a role and content.
• The system role sets standing behavior, the user role is the human, and the assistant role is the model's earlier replies.
• The system prompt controls behavior, not knowledge, and it is not a security boundary, so never put secrets in it.
• The model has no memory, so your app resends the full conversation on every request.
• Order matters: the model reads top to bottom, later messages weigh more, and the latest user message goes last.
• You pay for the whole history on each turn: 5,000 tokens of history plus a 10-token message is 5,010 input tokens.