Cost · bigonotes

Every API call is billed on two numbers: the input tokens you send and the output tokens the model writes. Each has its own price, and you pay on every request. Knowing where the tokens go tells you where to save.

Two prices, one bill#

Input and output tokens are priced separately, and output tokens almost always cost more. Typically it is two to five times more, depending on the model.

The reason comes from how the model works. Input tokens are processed in one parallel pass (see Tokens). Output tokens are generated one at a time, and each one needs a full pass through the model (see Next-token prediction).

The four levers#

There are four main ways to control cost. Each one either sends fewer tokens in, asks for fewer tokens out, or pays less per token.

Lever	What to do	What it cuts
Shorter prompts	Trim any system prompt text or document that does no real work.	Input tokens
Cap output	Set `max_tokens` on every request.	Output tokens
Cheaper models	Send easy tasks to a cheaper model.	Price per token
Caching	Cache a prompt prefix that repeats across requests.	Input cost

Input tokens and output tokens each carry a price. Shorter prompts and caching reduce input cost, max_tokens limits output, and a cheaper model lowers the price on both

Using each lever#

Shorter prompts matter because every word in your system prompt and every document you include is billed on every request. A sentence that adds nothing still costs money each time.

Capping output matters because an uncapped response to a vague prompt can run much longer than you need. Every API has a max_tokens parameter, so set it.

For easy tasks such as classification or simple extraction, a cheaper model is often enough. Save the expensive model for hard reasoning, and the savings can be large.

Caching is for prompts that start the same way on many requests. Some providers let you cache that shared beginning, so you do not pay full price for those tokens each time.

Short line#

Every token costs money on every request, in both directions. Send fewer tokens in, ask for fewer tokens out, or pay less per token.

Key takeaways#

Calls are billed on input tokens and output tokens, each with its own price.
Output typically costs two to five times more than input, because output is generated one token at a time.
Trim prompts, because every word is paid for on every request.
Set max_tokens so a vague prompt cannot produce a very long answer.
Route easy tasks to a cheaper model and keep the expensive one for hard reasoning.
Cache a repeated prompt prefix where the provider supports it.

Core idea#

Each API call is billed on input tokens and output tokens. You pay for both on every request.

Key points#

Input and output tokens have separate prices.
Output costs two to five times more than input, depending on the model.
Input is processed in one parallel pass.
Output is generated one token at a time. Each token needs a full pass.
The levers: fewer tokens in, fewer tokens out, lower price per token.

The four levers#

Lever	What to do
Shorter prompts	Trim text that does no real work.
Cap output	Set `max_tokens`.
Cheaper models	Use them for easy tasks.
Caching	Cache a repeated prompt prefix.

Pitfalls#

Do not leave output uncapped. A vague prompt can give a very long answer.
Do not keep unused text in the system prompt. You pay for it every request.
Do not use the expensive model for classification or simple extraction.

Short line#

"Every token costs money on every request. Cut tokens in, tokens out, or the price per token."