Using LLM APIs
Messages, stateless design, cost, streaming and error handling when you call a model from code.
5 notes · 16 min total
- 1MessagesHow an LLM API request is built from system, user and assistant messages, and why your app resends the whole conversation.4 min
- 2Stateless DesignWhy the API remembers nothing between calls, and what that means for your history, your code and your bill.3 min
- 3CostHow API calls are billed in input and output tokens, and the four levers that bring the bill down.2 min
- 4StreamingHow streaming sends a response piece by piece so the app feels faster, and the two gotchas to handle in code.3 min
- 5Error HandlingThe three kinds of API failure, which ones to retry, and how timeouts and fallbacks keep your app usable.4 min