Questions about the same state belong in one request: they are evaluated in parallel and independently, so extra questions add a few tokens and almost no time. Here is what we measured with 13 questions, why you should ask speculative questions you might not need, and the few cases where a second request is genuinely warranted.
Key takeaways
- Every question in a request sees the same state and is evaluated independently and in parallel.
- In our test, 13 questions in one request took 0.27 seconds and 1,828 tokens; 13 separate requests took 3.3 seconds and 18,940 tokens, with the same answers.
- Ask speculative questions: everything your code might need, including ones that only matter on some branches, and let code ignore the rest.
- A second request is warranted only when an earlier answer changes the state or the options of the next question.
- Coding agents tend to write one question per call; check their integrations for this.
Try it with your agent
Paste this into your coding agent:
Find every place we call a decision model (or an LLM classifier) more than once about the same input. Merge the questions into one request, adding speculative questions that later branches might need. Keep a second request only where the next question's state or options depend on an earlier answer, and add a comment explaining why. Measure latency and input tokens before and after on 20 real inputs.
Transcript
Intro
Once you're asking many small questions, the next question is how to send them. The answer is: all at once. Let's see why, with numbers.
Parallel and independent
In one request, every question sees the same state, and each is evaluated on its own, in parallel. One answer is never hidden context for another. So adding a question costs a few tokens for its text, and almost no time.
What we measured
We tested it. One support ticket, the customer's record, and a seven hundred word policy, about eighteen hundred tokens of state. Thirteen questions: the category, a refund request, a duplicate charge, whether policy allows it, a login problem, a repeat contact, a churn threat, and more. All thirteen came back together.
In one request, thirteen questions took a quarter of a second and eighteen hundred tokens. As thirteen separate requests, one after another, they took three point three seconds and almost nineteen thousand tokens, because the state was sent thirteen times. Firing the thirteen in parallel fixes the time, but not the bill. And the answers? The same, within point oh four.
Speculative questions
So ask speculatively. Include every question your code might need, even ones that only matter on some branches. Bug severity only matters if it's a bug report. A refund question only matters for billing. Ask them all anyway, and let the code use what applies.
When a second request is right
When is a second request justified? Only when an earlier answer changes what you'd send next. TypeSafe's cookbooks have three real cases. Skill suggestion ranks a hundred eighty two skills, then fetches the full text of the top three and judges them again. Structure recovery first decides which line breaks split sentences, then classifies the paragraphs that didn't exist before. And hierarchical classification uses each level's answer to pick the next level's options.
Watch your agent
One warning from the TypeSafe docs: coding agents fall into the one question per call habit more than people do. If an agent writes your integration, look for a loop that sends one question at a time, and merge it.
Quick check
Quick check. You need to know a ticket's category, and if it's billing, whether a refund was requested. What's the best way to ask?
Try it
Try it. Have your agent find and merge your one question per call loops, and measure the difference.
Recap
So: questions about the same state go in one request. They're evaluated in parallel and independently, so extra questions are nearly free. Ask speculatively, and only make a second request when the next question depends on the first. Next: routing requests to the right handler.