{"id":"13-fan-out","number":13,"slug":"speculative-fan-out","title":"Ask everything at once","short":"Fan-out","module":"Composing judgments","summary":"Questions about the same state belong in one request: they are evaluated in parallel and independently, so extra questions add a few tokens and almost no time. Here is what we measured with 13 questions, why you should ask speculative questions you might not need, and the few cases where a second request is genuinely warranted.","voice":"Jessica","scenes":[{"template":"title","chapter":"Intro","narration":"Once you're asking many small questions, the next question is how to send them. The answer is: all at once. Let's see why, with numbers.","params":{"kicker":"Lesson 13","module":"Module 4 · Composing judgments","title":"Ask *everything* at once","subtitle":"Many questions, one state, one request.","cast":["snap","curly"]},"start":0.5,"dur":10.761,"words":[{"text":"Once","s":0.8,"e":1.04},{"text":"you're","s":1.08,"e":1.2},{"text":"asking","s":1.4,"e":1.6},{"text":"many","s":1.68,"e":1.92},{"text":"small","s":2.08,"e":2.32},{"text":"questions,","s":2.4,"e":3.28},{"text":"the","s":3.36,"e":3.44},{"text":"next","s":3.52,"e":3.76},{"text":"question","s":3.84,"e":4.24},{"text":"is","s":4.4,"e":4.56},{"text":"how","s":4.72,"e":4.88},{"text":"to","s":4.92,"e":4.96},{"text":"send","s":5.12,"e":5.36},{"text":"them.","s":5.48,"e":5.68},{"text":"The","s":6.16,"e":6.24},{"text":"answer","s":6.32,"e":6.56},{"text":"is:","s":6.64,"e":7.08},{"text":"all","s":7.28,"e":7.44},{"text":"at","s":7.52,"e":7.6},{"text":"once.","s":7.68,"e":8.28},{"text":"Let's","s":8.48,"e":8.68},{"text":"see","s":8.72,"e":8.8},{"text":"why,","s":8.88,"e":9.36},{"text":"with","s":9.44,"e":9.52},{"text":"numbers.","s":9.6,"e":10.4}]},{"template":"statement","chapter":"Parallel and independent","narration":"In one request, every question sees the same state, and each is evaluated on its own, in parallel. One answer is never hidden context for another. So adding a question costs a few tokens for its text, and almost no time.","params":{"eyebrow":"How a request is evaluated","text":"Every question sees the same state. *None sees another's answer.*","sub":"So adding a question costs a few tokens and almost no time.","sub_at":"adding a question","art":"layers","tone":"agent","punch":"on its own"},"start":11.261,"dur":15.866,"words":[{"text":"In","s":11.561,"e":11.681},{"text":"one","s":11.721,"e":11.908},{"text":"request,","s":11.961,"e":12.521},{"text":"every","s":12.761,"e":13.049},{"text":"question","s":13.081,"e":13.511},{"text":"sees","s":13.561,"e":13.801},{"text":"the","s":13.881,"e":14.041},{"text":"same","s":14.161,"e":14.281},{"text":"state,","s":14.441,"e":14.921},{"text":"and","s":15.041,"e":15.161},{"text":"each","s":15.241,"e":15.401},{"text":"is","s":15.481,"e":15.561},{"text":"evaluated","s":15.721,"e":16.201},{"text":"on","s":16.281,"e":16.361},{"text":"its","s":16.481,"e":16.601},{"text":"own,","s":16.681,"e":17.161},{"text":"in","s":17.241,"e":17.321},{"text":"parallel.","s":17.401,"e":18.361},{"text":"One","s":18.441,"e":18.521},{"text":"answer","s":18.601,"e":19.001},{"text":"is","s":19.081,"e":19.321},{"text":"never","s":19.401,"e":19.641},{"text":"hidden","s":19.721,"e":19.961},{"text":"context","s":20.241,"e":20.521},{"text":"for","s":20.561,"e":20.601},{"text":"another.","s":20.681,"e":21.561},{"text":"So","s":21.681,"e":21.801},{"text":"adding","s":21.881,"e":22.121},{"text":"a","s":22.161,"e":22.201},{"text":"question","s":22.281,"e":22.681},{"text":"costs","s":22.761,"e":23.081},{"text":"a","s":23.161,"e":23.241},{"text":"few","s":23.321,"e":23.401},{"text":"tokens","s":23.481,"e":23.801},{"text":"for","s":23.881,"e":23.961},{"text":"its","s":24.081,"e":24.201},{"text":"text,","s":24.401,"e":24.761},{"text":"and","s":24.921,"e":25.081},{"text":"almost","s":25.281,"e":25.481},{"text":"no","s":25.641,"e":25.801},{"text":"time.","s":25.881,"e":26.521}]},{"template":"request","chapter":"What we measured","narration":"We tested it. One support ticket, the customer's record, and a seven hundred word policy, about eighteen hundred tokens of state. Thirteen questions: the category, a refund request, a duplicate charge, whether policy allows it, a login problem, a repeat contact, a churn threat, and more. All thirteen came back together.","params":{"model":"jev-1.13.0","latency":0.27,"state":"Ticket: “Hi, I was charged twice for order A-104 last Thursday and I still can't log in since your update. This is the third time I'm writing. If this isn't fixed today I'm cancelling…” + customer record + 745-word support policy","questions":[{"id":"c","type":"choice","text":"Main request?","ans":{"choice":"billing","p":0.93},"say_at":"the category"},{"id":"r","type":"noul","text":"Explicitly asks for a refund?","ans":{"v":0.13},"say_at":"a refund request"},{"id":"d","type":"noul","text":"Duplicate charge?","ans":{"v":0.98}},{"id":"p","type":"noul","text":"Policy allows a refund?","ans":{"v":0.99}},{"id":"l","type":"noul","text":"Login problem?","ans":{"v":0.99}},{"id":"rep","type":"noul","text":"Repeat contact?","ans":{"v":0.96}},{"id":"ch","type":"noul","text":"Threatens to cancel?","ans":{"v":0.98},"say_at":"a churn threat"}],"send_at":"Thirteen questions","ans_at":"All thirteen","footer":"7 of the 13 shown. Real answers, jev-1.13.0, 2026-10-15."},"start":27.127,"dur":22.841,"words":[{"text":"We","s":27.427,"e":27.547},{"text":"tested","s":27.587,"e":27.954},{"text":"it.","s":27.987,"e":28.227},{"text":"One","s":28.307,"e":28.867},{"text":"support","s":28.947,"e":29.267},{"text":"ticket,","s":29.467,"e":29.907},{"text":"the","s":29.947,"e":29.987},{"text":"customer's","s":30.067,"e":30.467},{"text":"record,","s":30.547,"e":31.107},{"text":"and","s":31.227,"e":31.347},{"text":"a","s":31.427,"e":31.587},{"text":"seven","s":31.667,"e":31.907},{"text":"hundred","s":31.987,"e":32.227},{"text":"word","s":32.307,"e":32.547},{"text":"policy,","s":32.627,"e":33.427},{"text":"about","s":33.507,"e":33.827},{"text":"eighteen","s":33.907,"e":34.227},{"text":"hundred","s":34.387,"e":34.707},{"text":"tokens","s":34.787,"e":35.107},{"text":"of","s":35.187,"e":35.267},{"text":"state.","s":35.347,"e":36.147},{"text":"Thirteen","s":36.187,"e":36.547},{"text":"questions:","s":36.707,"e":37.667},{"text":"the","s":37.707,"e":37.747},{"text":"category,","s":37.827,"e":38.627},{"text":"a","s":38.667,"e":38.707},{"text":"refund","s":38.787,"e":39.107},{"text":"request,","s":39.187,"e":39.747},{"text":"a","s":39.827,"e":39.867},{"text":"duplicate","s":39.907,"e":40.307},{"text":"charge,","s":40.387,"e":41.027},{"text":"whether","s":41.147,"e":41.267},{"text":"policy","s":41.347,"e":41.667},{"text":"allows","s":41.747,"e":42.067},{"text":"it,","s":42.147,"e":42.627},{"text":"a","s":42.667,"e":42.707},{"text":"login","s":42.787,"e":43.107},{"text":"problem,","s":43.187,"e":43.747},{"text":"a","s":43.787,"e":43.827},{"text":"repeat","s":43.907,"e":44.227},{"text":"contact,","s":44.307,"e":45.027},{"text":"a","s":45.067,"e":45.107},{"text":"churn","s":45.187,"e":45.427},{"text":"threat,","s":45.507,"e":45.987},{"text":"and","s":46.067,"e":46.147},{"text":"more.","s":46.227,"e":46.547},{"text":"All","s":47.027,"e":47.267},{"text":"thirteen","s":47.427,"e":47.987},{"text":"came","s":48.067,"e":48.227},{"text":"back","s":48.307,"e":48.387},{"text":"together.","s":48.467,"e":49.267}]},{"template":"stat","chapter":"What we measured","narration":"In one request, thirteen questions took a quarter of a second and eighteen hundred tokens. As thirteen separate requests, one after another, they took three point three seconds and almost nineteen thousand tokens, because the state was sent thirteen times. Firing the thirteen in parallel fixes the time, but not the bill. And the answers? The same, within point oh four.","params":{"title":"13 questions, same state","items":[{"value":0.27,"decimals":2,"suffix":" s","label":"one request · 1,828 tokens","tone":"good","at":"In one request","hero":true},{"value":3.3,"decimals":1,"suffix":" s","label":"13 requests in sequence · 18,940 tokens","tone":"bad","at":"three point three"},{"value":0.37,"decimals":2,"suffix":" s","label":"13 requests in parallel · 18,940 tokens","tone":"s2","at":"in parallel"}],"src":"Medians of 5 runs each, jev-1.13.0, measured 2026-10-15. Answers differed by at most 0.04."},"start":49.968,"dur":25.714,"words":[{"text":"In","s":50.268,"e":50.388},{"text":"one","s":50.428,"e":50.615},{"text":"request,","s":50.668,"e":51.308},{"text":"thirteen","s":51.548,"e":51.908},{"text":"questions","s":51.948,"e":52.508},{"text":"took","s":52.588,"e":52.748},{"text":"a","s":52.828,"e":52.908},{"text":"quarter","s":52.988,"e":53.308},{"text":"of","s":53.348,"e":53.388},{"text":"a","s":53.468,"e":53.508},{"text":"second","s":53.548,"e":53.948},{"text":"and","s":54.028,"e":54.268},{"text":"eighteen","s":54.348,"e":54.748},{"text":"hundred","s":54.988,"e":55.308},{"text":"tokens.","s":55.388,"e":56.028},{"text":"As","s":56.508,"e":56.668},{"text":"thirteen","s":56.828,"e":57.228},{"text":"separate","s":57.388,"e":57.708},{"text":"requests,","s":57.788,"e":58.508},{"text":"one","s":58.588,"e":58.668},{"text":"after","s":58.748,"e":58.988},{"text":"another,","s":59.188,"e":59.788},{"text":"they","s":59.868,"e":59.948},{"text":"took","s":60.028,"e":60.108},{"text":"three","s":60.268,"e":60.428},{"text":"point","s":60.508,"e":60.748},{"text":"three","s":60.868,"e":60.988},{"text":"seconds","s":61.148,"e":61.548},{"text":"and","s":61.628,"e":61.708},{"text":"almost","s":61.788,"e":62.108},{"text":"nineteen","s":62.268,"e":62.828},{"text":"thousand","s":62.988,"e":63.388},{"text":"tokens,","s":63.468,"e":64.348},{"text":"because","s":64.468,"e":64.588},{"text":"the","s":64.628,"e":64.668},{"text":"state","s":64.748,"e":64.988},{"text":"was","s":65.068,"e":65.148},{"text":"sent","s":65.268,"e":65.388},{"text":"thirteen","s":65.468,"e":65.868},{"text":"times.","s":65.948,"e":66.748},{"text":"Firing","s":66.988,"e":67.388},{"text":"the","s":67.468,"e":67.548},{"text":"thirteen","s":67.628,"e":67.948},{"text":"in","s":68.028,"e":68.108},{"text":"parallel","s":68.188,"e":68.748},{"text":"fixes","s":68.828,"e":69.228},{"text":"the","s":69.308,"e":69.388},{"text":"time,","s":69.628,"e":70.108},{"text":"but","s":70.228,"e":70.348},{"text":"not","s":70.508,"e":70.668},{"text":"the","s":70.748,"e":70.828},{"text":"bill.","s":70.908,"e":71.468},{"text":"And","s":71.708,"e":71.788},{"text":"the","s":71.828,"e":71.868},{"text":"answers?","s":71.948,"e":72.508},{"text":"The","s":72.828,"e":72.908},{"text":"same,","s":73.068,"e":73.548},{"text":"within","s":73.708,"e":73.868},{"text":"point","s":73.948,"e":74.108},{"text":"oh","s":74.268,"e":74.428},{"text":"four.","s":74.588,"e":75.068}]},{"template":"code","chapter":"Speculative questions","narration":"So ask speculatively. Include every question your code might need, even ones that only matter on some branches. Bug severity only matters if it's a bug report. A refund question only matters for billing. Ask them all anyway, and let the code use what applies.","params":{"file":"triage.py","lang":"py","lines":["questions = {","    \"category\": Choice(...),          # always needed","    \"bug_severity\": Score(...),       # only if category == bug","    \"has_repro_steps\": Noul(...),    # only if category == bug","    \"refund_requested\": Noul(...),   # only if category == billing","    \"frustration\": Score(...),       # always useful","}","a = client.system_one(state=ticket, questions=questions).answers","if a[\"category\"].choice == \"bug\" and a[\"bug_severity\"].score > 1.5:","    escalate(ticket)                  # the speculative answers were ready"],"hl":[{"from":3,"to":5,"label":"speculative: maybe needed","tone":"agent","at":"Ask them all anyway"},{"from":9,"to":10,"label":"code uses what applies","tone":"good","at":"let the code use"}]},"start":75.682,"dur":17.956,"words":[{"text":"So","s":75.982,"e":76.222},{"text":"ask","s":76.382,"e":76.702},{"text":"speculatively.","s":76.782,"e":77.942},{"text":"Include","s":78.142,"e":78.622},{"text":"every","s":78.702,"e":78.942},{"text":"question","s":79.102,"e":79.422},{"text":"your","s":79.502,"e":79.582},{"text":"code","s":79.662,"e":79.822},{"text":"might","s":79.902,"e":80.062},{"text":"need,","s":80.142,"e":80.702},{"text":"even","s":80.822,"e":80.942},{"text":"ones","s":81.022,"e":81.182},{"text":"that","s":81.262,"e":81.422},{"text":"only","s":81.502,"e":81.662},{"text":"matter","s":81.742,"e":81.982},{"text":"on","s":82.062,"e":82.142},{"text":"some","s":82.222,"e":82.382},{"text":"branches.","s":82.462,"e":83.302},{"text":"Bug","s":83.582,"e":83.742},{"text":"severity","s":83.822,"e":84.302},{"text":"only","s":84.382,"e":84.702},{"text":"matters","s":84.782,"e":85.182},{"text":"if","s":85.262,"e":85.342},{"text":"it's","s":85.382,"e":85.502},{"text":"a","s":85.542,"e":85.582},{"text":"bug","s":85.662,"e":85.822},{"text":"report.","s":85.902,"e":86.382},{"text":"A","s":86.622,"e":86.702},{"text":"refund","s":86.902,"e":87.102},{"text":"question","s":87.342,"e":87.582},{"text":"only","s":87.742,"e":87.982},{"text":"matters","s":88.062,"e":88.382},{"text":"for","s":88.422,"e":88.462},{"text":"billing.","s":88.542,"e":89.022},{"text":"Ask","s":89.502,"e":89.742},{"text":"them","s":89.822,"e":89.902},{"text":"all","s":90.062,"e":90.302},{"text":"anyway,","s":90.382,"e":91.022},{"text":"and","s":91.062,"e":91.102},{"text":"let","s":91.182,"e":91.262},{"text":"the","s":91.342,"e":91.382},{"text":"code","s":91.422,"e":91.662},{"text":"use","s":91.822,"e":91.982},{"text":"what","s":92.062,"e":92.142},{"text":"applies.","s":92.182,"e":93.022}]},{"template":"list","chapter":"When a second request is right","narration":"When is a second request justified? Only when an earlier answer changes what you'd send next. TypeSafe's cookbooks have three real cases. Skill suggestion ranks a hundred eighty two skills, then fetches the full text of the top three and judges them again. Structure recovery first decides which line breaks split sentences, then classifies the paragraphs that didn't exist before. And hierarchical classification uses each level's answer to pick the next level's options.","params":{"title":"A second request only when the next question depends on the first","style":"num","items":[{"text":"**New evidence:** rank 182 skills, then fetch the top 3 and re-judge","at":"Skill suggestion"},{"text":"**New state:** rebuild paragraphs, then classify them","at":"Structure recovery"},{"text":"**New options:** each level's answer picks the next level's choices","at":"hierarchical classification"}]},"start":93.638,"dur":30.834,"words":[{"text":"When","s":93.938,"e":94.148},{"text":"is","s":94.178,"e":94.238},{"text":"a","s":94.258,"e":94.338},{"text":"second","s":94.418,"e":94.765},{"text":"request","s":94.818,"e":95.298},{"text":"justified?","s":95.378,"e":96.498},{"text":"Only","s":96.698,"e":96.898},{"text":"when","s":96.978,"e":97.058},{"text":"an","s":97.138,"e":97.218},{"text":"earlier","s":97.298,"e":97.618},{"text":"answer","s":97.698,"e":98.098},{"text":"changes","s":98.258,"e":98.658},{"text":"what","s":98.738,"e":98.818},{"text":"you'd","s":98.898,"e":99.058},{"text":"send","s":99.178,"e":99.298},{"text":"next.","s":99.378,"e":100.258},{"text":"TypeSafe's","s":100.378,"e":100.818},{"text":"cookbooks","s":100.938,"e":101.298},{"text":"have","s":101.378,"e":101.618},{"text":"three","s":101.778,"e":101.938},{"text":"real","s":102.098,"e":102.338},{"text":"cases.","s":102.418,"e":103.258},{"text":"Skill","s":103.538,"e":103.778},{"text":"suggestion","s":103.858,"e":104.418},{"text":"ranks","s":104.498,"e":104.738},{"text":"a","s":104.818,"e":104.898},{"text":"hundred","s":104.978,"e":105.218},{"text":"eighty","s":105.298,"e":105.498},{"text":"two","s":105.538,"e":105.618},{"text":"skills,","s":105.778,"e":106.578},{"text":"then","s":106.698,"e":106.818},{"text":"fetches","s":106.898,"e":107.298},{"text":"the","s":107.378,"e":107.458},{"text":"full","s":107.538,"e":107.698},{"text":"text","s":107.778,"e":108.098},{"text":"of","s":108.138,"e":108.178},{"text":"the","s":108.258,"e":108.298},{"text":"top","s":108.338,"e":108.578},{"text":"three","s":108.658,"e":109.058},{"text":"and","s":109.138,"e":109.298},{"text":"judges","s":109.378,"e":109.618},{"text":"them","s":109.698,"e":109.778},{"text":"again.","s":109.978,"e":110.458},{"text":"Structure","s":110.738,"e":111.138},{"text":"recovery","s":111.218,"e":111.778},{"text":"first","s":111.938,"e":112.178},{"text":"decides","s":112.258,"e":112.658},{"text":"which","s":112.818,"e":112.978},{"text":"line","s":113.058,"e":113.298},{"text":"breaks","s":113.418,"e":113.578},{"text":"split","s":113.618,"e":113.858},{"text":"sentences,","s":113.938,"e":114.978},{"text":"then","s":115.138,"e":115.298},{"text":"classifies","s":115.378,"e":116.018},{"text":"the","s":116.058,"e":116.098},{"text":"paragraphs","s":116.338,"e":116.658},{"text":"that","s":116.738,"e":116.818},{"text":"didn't","s":116.898,"e":117.098},{"text":"exist","s":117.138,"e":117.458},{"text":"before.","s":117.538,"e":118.178},{"text":"And","s":118.418,"e":118.498},{"text":"hierarchical","s":118.578,"e":119.138},{"text":"classification","s":119.218,"e":120.098},{"text":"uses","s":120.178,"e":120.578},{"text":"each","s":120.658,"e":120.818},{"text":"level's","s":120.898,"e":121.218},{"text":"answer","s":121.298,"e":121.618},{"text":"to","s":121.698,"e":121.778},{"text":"pick","s":121.858,"e":122.018},{"text":"the","s":122.058,"e":122.098},{"text":"next","s":122.258,"e":122.498},{"text":"level's","s":122.578,"e":122.818},{"text":"options.","s":122.898,"e":123.858}]},{"template":"statement","chapter":"Watch your agent","narration":"One warning from the TypeSafe docs: coding agents fall into the one question per call habit more than people do. If an agent writes your integration, look for a loop that sends one question at a time, and merge it.","params":{"eyebrow":"TypeSafe docs","text":"Coding agents fall into the *one question per call* habit.","sub":"Look for loops that send one question at a time, and merge them into one request.","sub_at":"look for a loop","art":"bot","tone":"bad","punch":"one question per call"},"start":124.472,"dur":14.116,"words":[{"text":"One","s":124.772,"e":125.012},{"text":"warning","s":125.092,"e":125.492},{"text":"from","s":125.572,"e":125.652},{"text":"the","s":125.732,"e":125.772},{"text":"TypeSafe","s":125.812,"e":126.292},{"text":"docs:","s":126.372,"e":127.252},{"text":"coding","s":127.412,"e":127.572},{"text":"agents","s":127.652,"e":128.052},{"text":"fall","s":128.132,"e":128.292},{"text":"into","s":128.372,"e":128.532},{"text":"the","s":128.572,"e":128.612},{"text":"one","s":128.772,"e":128.932},{"text":"question","s":129.012,"e":129.412},{"text":"per","s":129.492,"e":129.572},{"text":"call","s":129.652,"e":129.892},{"text":"habit","s":129.972,"e":130.372},{"text":"more","s":130.532,"e":130.692},{"text":"than","s":130.772,"e":130.852},{"text":"people","s":131.012,"e":131.252},{"text":"do.","s":131.332,"e":131.652},{"text":"If","s":132.132,"e":132.212},{"text":"an","s":132.292,"e":132.372},{"text":"agent","s":132.452,"e":132.692},{"text":"writes","s":132.772,"e":133.012},{"text":"your","s":133.052,"e":133.092},{"text":"integration,","s":133.172,"e":134.132},{"text":"look","s":134.252,"e":134.372},{"text":"for","s":134.452,"e":134.532},{"text":"a","s":134.572,"e":134.612},{"text":"loop","s":134.692,"e":134.852},{"text":"that","s":134.932,"e":135.012},{"text":"sends","s":135.092,"e":135.332},{"text":"one","s":135.492,"e":135.652},{"text":"question","s":135.732,"e":136.132},{"text":"at","s":136.172,"e":136.212},{"text":"a","s":136.292,"e":136.332},{"text":"time,","s":136.372,"e":137.012},{"text":"and","s":137.092,"e":137.172},{"text":"merge","s":137.252,"e":137.492},{"text":"it.","s":137.612,"e":137.972}]},{"template":"quiz","chapter":"Quick check","narration":"Quick check. You need to know a ticket's category, and if it's billing, whether a refund was requested. What's the best way to ask?","params":{"q":"You need the **category**, and **if billing**, whether a refund was asked for. Best approach?","options":["Ask the category, then send a second request for billing tickets","Ask both in one request; ignore the refund answer for non-billing tickets","Ask one combined question: is this a billing refund request?","Ask an LLM to write both answers as JSON"],"answer":1,"explain":"The refund question doesn't depend on the category answer; it can be asked of any ticket. Ask it speculatively in the same request and let code decide whether to use it."},"start":138.588,"dur":13.938,"words":[{"text":"Quick","s":138.888,"e":139.048},{"text":"check.","s":139.128,"e":139.528},{"text":"You","s":139.848,"e":139.928},{"text":"need","s":140.008,"e":140.168},{"text":"to","s":140.208,"e":140.248},{"text":"know","s":140.328,"e":140.488},{"text":"a","s":140.528,"e":140.568},{"text":"ticket's","s":140.648,"e":141.048},{"text":"category,","s":141.368,"e":141.848},{"text":"and","s":141.928,"e":142.248},{"text":"if","s":142.328,"e":142.408},{"text":"it's","s":142.488,"e":142.648},{"text":"billing,","s":142.728,"e":143.288},{"text":"whether","s":143.448,"e":143.608},{"text":"a","s":143.648,"e":143.688},{"text":"refund","s":143.848,"e":144.248},{"text":"was","s":144.328,"e":144.408},{"text":"requested.","s":144.728,"e":145.288},{"text":"What's","s":145.528,"e":145.768},{"text":"the","s":145.808,"e":145.848},{"text":"best","s":146.088,"e":146.328},{"text":"way","s":146.368,"e":146.408},{"text":"to","s":146.488,"e":146.568},{"text":"ask?","s":146.648,"e":147.368}]},{"template":"exercise","chapter":"Try it","narration":"Try it. Have your agent find and merge your one question per call loops, and measure the difference.","params":{"title":"Merge your calls","intro":"Paste this into your coding agent:","prompt":"Find every place we call a decision model (or an LLM classifier) more\nthan once about the same input. Merge the questions into one request,\nadding speculative questions that later branches might need. Keep a\nsecond request only where the next question's state or options depend\non an earlier answer, and add a comment explaining why. Measure latency\nand input tokens before and after on 20 real inputs."},"start":152.526,"dur":6.906,"words":[{"text":"Try","s":152.826,"e":153.066},{"text":"it.","s":153.146,"e":153.386},{"text":"Have","s":153.706,"e":153.866},{"text":"your","s":153.946,"e":154.026},{"text":"agent","s":154.106,"e":154.346},{"text":"find","s":154.506,"e":154.826},{"text":"and","s":154.906,"e":154.986},{"text":"merge","s":155.066,"e":155.386},{"text":"your","s":155.506,"e":155.626},{"text":"one","s":155.706,"e":155.786},{"text":"question","s":155.946,"e":156.266},{"text":"per","s":156.346,"e":156.426},{"text":"call","s":156.506,"e":156.746},{"text":"loops,","s":156.826,"e":157.306},{"text":"and","s":157.386,"e":157.466},{"text":"measure","s":157.546,"e":157.786},{"text":"the","s":157.866,"e":157.906},{"text":"difference.","s":157.946,"e":158.826}]},{"template":"recap","chapter":"Recap","narration":"So: questions about the same state go in one request. They're evaluated in parallel and independently, so extra questions are nearly free. Ask speculatively, and only make a second request when the next question depends on the first. Next: routing requests to the right handler.","params":{"items":["One state, many questions, one request.","13 questions: 0.27 s and 1.8k tokens vs 3.3 s and 18.9k.","Ask speculative questions; let code ignore the rest.","Second request only when the next question depends on the first."],"next":"Route and cascade","next_at":"Next"},"start":159.432,"dur":20.577,"words":[{"text":"So:","s":159.732,"e":160.372},{"text":"questions","s":160.612,"e":160.852},{"text":"about","s":160.972,"e":161.092},{"text":"the","s":161.172,"e":161.252},{"text":"same","s":161.332,"e":161.572},{"text":"state","s":161.652,"e":162.052},{"text":"go","s":162.132,"e":162.212},{"text":"in","s":162.332,"e":162.452},{"text":"one","s":162.532,"e":162.612},{"text":"request.","s":162.932,"e":163.572},{"text":"They're","s":163.612,"e":163.732},{"text":"evaluated","s":163.892,"e":164.372},{"text":"in","s":164.452,"e":164.612},{"text":"parallel","s":164.892,"e":165.172},{"text":"and","s":165.292,"e":165.412},{"text":"independently,","s":165.492,"e":166.532},{"text":"so","s":166.692,"e":166.852},{"text":"extra","s":166.932,"e":167.172},{"text":"questions","s":167.252,"e":167.652},{"text":"are","s":167.732,"e":167.892},{"text":"nearly","s":167.972,"e":168.292},{"text":"free.","s":168.372,"e":169.252},{"text":"Ask","s":169.412,"e":169.572},{"text":"speculatively,","s":169.652,"e":170.692},{"text":"and","s":170.812,"e":170.932},{"text":"only","s":171.012,"e":171.252},{"text":"make","s":171.332,"e":171.412},{"text":"a","s":171.465,"e":171.519},{"text":"second","s":171.572,"e":171.812},{"text":"request","s":172.052,"e":172.292},{"text":"when","s":172.372,"e":172.452},{"text":"the","s":172.532,"e":172.612},{"text":"next","s":172.692,"e":172.932},{"text":"question","s":173.012,"e":173.412},{"text":"depends","s":173.652,"e":173.892},{"text":"on","s":173.932,"e":173.972},{"text":"the","s":174.052,"e":174.132},{"text":"first.","s":174.372,"e":175.172},{"text":"Next:","s":175.252,"e":175.972},{"text":"routing","s":176.052,"e":176.372},{"text":"requests","s":176.452,"e":176.932},{"text":"to","s":177.012,"e":177.172},{"text":"the","s":177.212,"e":177.252},{"text":"right","s":177.332,"e":177.572},{"text":"handler.","s":177.652,"e":178.452}]}],"total":180.409,"audio":"/audio/13-fan-out.mp3"}