Routing is a two-step pattern. First, classify the input. Then send it to a specialized prompt or model based on that class.
An example#
A customer support system receives a message.
- A classifier labels the message as billing, technical or general.
- Your code sends it to the prompt for that class.
| Route | What its prompt knows |
|---|---|
| Billing | How to handle refunds and charges |
| Technical | The product's error codes |
| General | Everything else, in a short friendly reply |
Why specialize#
One general prompt has to carry instructions for every case. A specialized prompt only carries what its class needs, so it is:
- Sharper. The model is not distracted by rules for other cases.
- Shorter. Fewer instructions means fewer input tokens.
- Cheaper. Shorter prompts cost less on every call.
Route by difficulty#
You can also route by how hard the question is, not only by topic. Send easy questions to a small, cheap model and hard ones to a larger, more expensive one. Most traffic is usually easy, so this cuts cost without hurting the hard cases.
The classifier itself can be a small model call, or plain code when the classes are easy to tell apart.
Short line#
Routing classifies the input first, then sends it to a specialized prompt or model. Specialized prompts are sharper, shorter and cheaper than one general prompt, and you can route easy questions to a small model and hard ones to a large one.
Key takeaways#
- Routing has two steps: classify, then dispatch.
- Each class gets its own specialized prompt or model.
- A specialized prompt is sharper, shorter and cheaper than a general one.
- Route easy questions to a small cheap model and hard ones to a larger model.