When a task has several steps, a model that answers straight away often gets it wrong. If you ask it to reason first, the same model often gets it right. This note explains why that works, how to ask for it so the output stays easy to parse, and when the extra cost is worth paying.
Why reasoning first helps#
A model writes left to right and cannot go back (see Next-Token Prediction). If you ask a hard question and expect an immediate answer, the first answer token is conditioned on nothing but the question. The model has to guess the whole result in one step.
When the model reasons first, the reasoning tokens are written before the answer. By the time it produces the answer, its context holds a chain of worked-out steps as well as the question. Each step is easier to get right than the whole problem, and the answer follows from them.
How to ask for it#
The simplest trigger is a short phrase in your prompt, such as "think step by step" or "reason through this before answering". That works, but the reasoning and the answer then come back mixed together.
To make the output parseable, ask for two separate sections. Your code can then read the answer tag and ignore the reasoning. This is the tag pattern from Delimiters used on the output side.
Reason through the problem step by step in <reasoning> tags.
Then give your final answer in <answer> tags.Put the reasoning section first. If the answer comes before the reasoning, the answer is back to being a snap guess, and the "reasoning" only explains it afterwards.
When to use it#
Step-by-step prompting pays off when the task needs several dependent steps and accuracy matters more than speed.
| Use it for | Skip it for |
|---|---|
| Math and arithmetic word problems | Simple lookups and short factual answers |
| Logic and multi-step deduction | Rewording, formatting or translation |
| Multi-step extraction from messy input | Classification with an obvious label |
| Any task where a wrong answer costs more than a slow one | Latency-sensitive replies, such as autocomplete |
The cost#
Reasoning is not free. Reasoning tokens are output tokens, which are the slow and expensive kind (see Cost). A long chain of thought makes the reply slower and the bill bigger on every request.
So reserve it for the tasks that need it. Reasoning models (such as o3) do this thinking internally and automatically, so you rarely need to ask them. Explicit step-by-step prompting matters most on standard models.
Add step-by-step only where a plain prompt gets the answer wrong. If a direct answer is already right, the extra tokens buy you nothing.
Short line#
Reasoning before the answer puts worked-out steps into the context, so the answer is conditioned on them instead of a guess. It costs extra output tokens, so use it for hard tasks only.
Key takeaways#
- The model cannot go back, so a direct answer is conditioned on the question alone.
- Reasoning tokens come first, so the answer is conditioned on worked-out steps.
- Trigger it with "think step by step", or ask for
<reasoning>then<answer>tags. - Separate sections let your code parse the answer and drop the reasoning.
- Reasoning tokens are output tokens: slower and more expensive.
- Use it for math, logic and multi-step extraction. Reasoning models do it automatically.