An LLM API is stateless. The provider reads your request, sends a reply and keeps nothing. That means your app owns the conversation, and it also means your costs grow as the conversation gets longer.
What stateless means#
Between calls there is no session, no memory and nothing stored on the provider's side. Every request stands alone.
This is a deliberate choice, not a limitation. It keeps the API simple, easy to scale and predictable, because the provider does not have to hold state for millions of users at once.
The model also has no idea it is talking to the same person it spoke to a minute ago. It only sees what you put in the message list. This matches the idea of frozen weights: the model itself does not change as you use it.
What you are responsible for#
Because the provider forgets, your app takes on three jobs.
- Store the conversation history.
- Decide what to include in each request.
- Decide what to drop when the context fills up. The context window note covers strategies for this.
A chat feels continuous to the user only because your app feeds the history back in on every call.
How the history is resent#
Each call carries the whole conversation so far, plus the new message. The messages list is plain data that your app keeps and extends.
messages = [
{"role": "user", "content": "My name is Sam."},
{"role": "assistant", "content": "Nice to meet you, Sam."},
{"role": "user", "content": "What is my name?"},
]Send this list and the model can answer "Sam", because the earlier turns are in the request. Send only the last message and it has no way to know.
The cost gotcha#
Because you resend the full history every time, the input for each call grows with the length of the conversation. The next table assumes each turn adds 500 tokens, and counts the cumulative history sent as input.
| Turn | Input tokens for this call |
|---|---|
| 1 | 500 |
| 2 | 1,000 |
| 10 | 5,000 |
| 20 | 10,000 |
By turn 20 you pay for 10,000 input tokens to send one new message. Most of those tokens are history the model already processed in earlier turns.
The cost also compounds across the whole conversation. Adding up turns 1 to 20 gives 105,000 input tokens (500 x 210), even though the conversation only holds 10,000 tokens of text. Input is billed per token, so every resent turn counts again.
Managing history is a cost decision#
Context management is not only a technical detail. It directly controls your bill. These are the main tools:
- Drop old turns.
- Summarize earlier turns.
- Retrieve only the history that is relevant to the current question.
Plan a strategy for long conversations from day one. It is hard to add later, once real users and real bills exist.
Trade-offs#
Statelessness gives you simplicity, scale and full control over what the model sees. The price is that you carry the work and the cost of state. Every turn you keep makes every later call more expensive, so you choose what is worth resending.
Short line#
The API is stateless, so your app stores the history and resends it with every call. Cost therefore grows with conversation length, and trimming history is a cost decision.
Key takeaways#
- The provider keeps no session or memory between calls. Each request is independent.
- The model only sees what is in the message list you send.
- Your app stores the history, picks what to include and drops what no longer fits.
- Input tokens for a call grow with every turn, so cost grows with conversation length.
- Dropping, summarizing or retrieving history controls your bill.