{"id":"15-select","number":15,"slug":"select-dont-generate","title":"Select, don't generate","short":"Select, don't generate","module":"Composing judgments","summary":"A decision model can't write a value, but most extraction is really selection: the value is already in the text. Let code find the candidates and let the model pick one, then copy it verbatim. Here is that pattern for phone numbers, dates, search reranking, line-level search and deep taxonomies.","voice":"Jessica","scenes":[{"template":"title","chapter":"Intro","narration":"A decision model can't write. So how do you get a phone number, a date, or the right paragraph out of a document? You don't ask it to write the value. You let it choose.","params":{"kicker":"Lesson 15","module":"Module 4 · Composing judgments","title":"Select, *don't generate*","subtitle":"Code finds the candidates. The model picks. Code copies.","cast":["curly","snap"]},"start":0.5,"dur":12.041,"words":[{"text":"A","s":0.8,"e":0.88},{"text":"decision","s":1.12,"e":1.36},{"text":"model","s":1.44,"e":1.76},{"text":"can't","s":1.84,"e":2.16},{"text":"write.","s":2.24,"e":3.12},{"text":"So","s":3.28,"e":3.44},{"text":"how","s":3.6,"e":3.68},{"text":"do","s":3.72,"e":3.76},{"text":"you","s":3.8,"e":3.84},{"text":"get","s":3.92,"e":4},{"text":"a","s":4.08,"e":4.16},{"text":"phone","s":4.28,"e":4.4},{"text":"number,","s":4.48,"e":4.96},{"text":"a","s":5,"e":5.04},{"text":"date,","s":5.2,"e":5.68},{"text":"or","s":5.76,"e":5.84},{"text":"the","s":5.92,"e":6},{"text":"right","s":6.12,"e":6.24},{"text":"paragraph","s":6.52,"e":6.8},{"text":"out","s":6.88,"e":6.96},{"text":"of","s":7,"e":7.04},{"text":"a","s":7.08,"e":7.12},{"text":"document?","s":7.4,"e":8.16},{"text":"You","s":8.2,"e":8.24},{"text":"don't","s":8.28,"e":8.48},{"text":"ask","s":8.64,"e":8.8},{"text":"it","s":8.88,"e":8.96},{"text":"to","s":9,"e":9.04},{"text":"write","s":9.12,"e":9.2},{"text":"the","s":9.28,"e":9.32},{"text":"value.","s":9.36,"e":10.16},{"text":"You","s":10.24,"e":10.32},{"text":"let","s":10.44,"e":10.56},{"text":"it","s":10.64,"e":10.88},{"text":"choose.","s":11.12,"e":11.68}]},{"template":"flow","chapter":"The pattern","narration":"Here's the pattern. Most extraction is really selection, because the value is already sitting in the text. So code finds the candidates, with a regular expression, a parser or a search. The model picks the one you asked for. And code copies it verbatim. Nothing can be invented, because the model only ever chooses.","params":{"title":"Find, choose, copy","nodes":[{"label":"Code finds candidates","sub":"regex, parser, search","kind":"curly","edge":"candidates","at":"code finds"},{"label":"Model picks one","sub":"a Choice, plus “none”","kind":"snap","edge":"a choice","at":"The model picks"},{"label":"Code copies it","sub":"verbatim, then normalizes","kind":"curly","at":"copies it verbatim"}]},"start":12.541,"dur":22.188,"words":[{"text":"Here's","s":12.841,"e":13.161},{"text":"the","s":13.201,"e":13.241},{"text":"pattern.","s":13.321,"e":13.801},{"text":"Most","s":14.201,"e":14.441},{"text":"extraction","s":14.521,"e":15.241},{"text":"is","s":15.321,"e":15.481},{"text":"really","s":15.561,"e":15.881},{"text":"selection,","s":16.041,"e":16.921},{"text":"because","s":17.041,"e":17.161},{"text":"the","s":17.241,"e":17.281},{"text":"value","s":17.321,"e":17.721},{"text":"is","s":17.801,"e":18.041},{"text":"already","s":18.241,"e":18.441},{"text":"sitting","s":18.601,"e":18.841},{"text":"in","s":18.881,"e":18.921},{"text":"the","s":18.961,"e":19.001},{"text":"text.","s":19.081,"e":19.641},{"text":"So","s":20.041,"e":20.201},{"text":"code","s":20.441,"e":20.681},{"text":"finds","s":20.841,"e":21.121},{"text":"the","s":21.161,"e":21.241},{"text":"candidates,","s":21.561,"e":22.041},{"text":"with","s":22.121,"e":22.201},{"text":"a","s":22.228,"e":22.254},{"text":"regular","s":22.281,"e":22.601},{"text":"expression,","s":22.961,"e":23.401},{"text":"a","s":23.481,"e":23.521},{"text":"parser","s":23.561,"e":24.121},{"text":"or","s":24.201,"e":24.281},{"text":"a","s":24.361,"e":24.441},{"text":"search.","s":24.521,"e":25.001},{"text":"The","s":25.401,"e":25.481},{"text":"model","s":25.561,"e":25.881},{"text":"picks","s":26.041,"e":26.281},{"text":"the","s":26.361,"e":26.401},{"text":"one","s":26.441,"e":26.601},{"text":"you","s":26.681,"e":26.761},{"text":"asked","s":26.881,"e":27.081},{"text":"for.","s":27.161,"e":27.601},{"text":"And","s":27.801,"e":27.881},{"text":"code","s":27.961,"e":28.281},{"text":"copies","s":28.361,"e":28.761},{"text":"it","s":28.841,"e":28.921},{"text":"verbatim.","s":29.001,"e":29.721},{"text":"Nothing","s":30.121,"e":30.441},{"text":"can","s":30.521,"e":30.601},{"text":"be","s":30.681,"e":30.721},{"text":"invented,","s":30.761,"e":31.481},{"text":"because","s":31.601,"e":31.721},{"text":"the","s":31.761,"e":31.801},{"text":"model","s":31.961,"e":32.201},{"text":"only","s":32.441,"e":32.681},{"text":"ever","s":32.841,"e":33.001},{"text":"chooses.","s":33.161,"e":34.121}]},{"template":"request","chapter":"Pick a value","narration":"This email signature has three phone numbers: a finance desk, a direct line, and a support hotline. A regex finds all three. Then one Choice: which candidate is the twenty four seven support number? The hotline, at one point oh. Your code copies that exact string, and there's no way to get a digit wrong.","params":{"model":"jev-1.13.0","latency":0.31,"state":{"text":"…billing questions call our finance desk at +1 (415) 555-0142. My direct line is 415-555-0199… Our 24/7 support hotline is (800) 555-0123.","candidates":["+1 (415) 555-0142","415-555-0199","(800) 555-0123"]},"questions":[{"id":"phone","type":"choice","text":"Which value in `candidates` is the 24/7 support number in `text`?","ans":{"choice":"(800) 555-0123","p":1},"say_at":"one Choice"}],"send_at":"Then one Choice","ans_at":"The hotline","footer":"Options: the three candidates + “none”. Real answer, jev-1.13.0."},"start":34.729,"dur":20.986,"words":[{"text":"This","s":35.029,"e":35.269},{"text":"email","s":35.349,"e":35.589},{"text":"signature","s":35.669,"e":36.149},{"text":"has","s":36.269,"e":36.389},{"text":"three","s":36.469,"e":36.709},{"text":"phone","s":36.789,"e":36.949},{"text":"numbers:","s":37.029,"e":37.749},{"text":"a","s":37.789,"e":37.829},{"text":"finance","s":37.909,"e":38.309},{"text":"desk,","s":38.389,"e":38.869},{"text":"a","s":38.909,"e":38.949},{"text":"direct","s":39.029,"e":39.349},{"text":"line,","s":39.429,"e":39.909},{"text":"and","s":40.029,"e":40.149},{"text":"a","s":40.229,"e":40.269},{"text":"support","s":40.309,"e":40.709},{"text":"hotline.","s":40.829,"e":41.429},{"text":"A","s":41.669,"e":41.749},{"text":"regex","s":41.989,"e":42.229},{"text":"finds","s":42.389,"e":42.709},{"text":"all","s":42.829,"e":42.949},{"text":"three.","s":43.029,"e":43.749},{"text":"Then","s":43.869,"e":43.989},{"text":"one","s":44.309,"e":44.469},{"text":"Choice:","s":44.549,"e":45.349},{"text":"which","s":45.469,"e":45.589},{"text":"candidate","s":45.669,"e":46.069},{"text":"is","s":46.149,"e":46.229},{"text":"the","s":46.309,"e":46.389},{"text":"twenty","s":46.469,"e":46.709},{"text":"four","s":46.789,"e":46.949},{"text":"seven","s":47.029,"e":47.269},{"text":"support","s":47.429,"e":47.589},{"text":"number?","s":47.669,"e":48.389},{"text":"The","s":48.429,"e":48.469},{"text":"hotline,","s":48.549,"e":49.189},{"text":"at","s":49.269,"e":49.349},{"text":"one","s":49.429,"e":49.509},{"text":"point","s":49.669,"e":49.749},{"text":"oh.","s":49.909,"e":50.469},{"text":"Your","s":50.549,"e":50.709},{"text":"code","s":50.829,"e":50.949},{"text":"copies","s":51.029,"e":51.349},{"text":"that","s":51.429,"e":51.589},{"text":"exact","s":51.869,"e":52.149},{"text":"string,","s":52.229,"e":52.789},{"text":"and","s":52.829,"e":52.869},{"text":"there's","s":52.949,"e":53.269},{"text":"no","s":53.349,"e":53.509},{"text":"way","s":53.589,"e":53.749},{"text":"to","s":53.789,"e":53.829},{"text":"get","s":53.909,"e":53.989},{"text":"a","s":54.029,"e":54.069},{"text":"digit","s":54.149,"e":54.389},{"text":"wrong.","s":54.469,"e":55.109}]},{"template":"request","chapter":"Dates are parts","narration":"Dates work the same way, part by part. Jev reads dates as text, not as numbers, so don't ask it which date is earlier. Ask three small Choices: which month, which day, which year, each with a not stated option. Here, March, the third, twenty twenty six. Then code builds the real date, and does every comparison.","params":{"model":"jev-1.13.0","latency":0.25,"state":"This agreement starts on the date of signature, the third of March 2026, and renews each year unless cancelled.","questions":[{"id":"m","type":"choice","text":"Start month? (12 months + not_stated)","ans":{"choice":"March","p":1},"at":"which month"},{"id":"d","type":"choice","text":"Start day? (1–31 + not_stated)","ans":{"choice":"3","p":1},"at":"which day"},{"id":"y","type":"choice","text":"Start year? (2020–2030 + not_stated)","ans":{"choice":"2026","p":1},"at":"which year"}],"send_at":"Here, March","ans_at":"Here, March"},"start":55.715,"dur":24.016,"words":[{"text":"Dates","s":56.015,"e":56.335},{"text":"work","s":56.415,"e":56.575},{"text":"the","s":56.655,"e":56.735},{"text":"same","s":56.815,"e":57.055},{"text":"way,","s":57.135,"e":57.695},{"text":"part","s":57.815,"e":57.935},{"text":"by","s":58.015,"e":58.175},{"text":"part.","s":58.335,"e":58.975},{"text":"Jev","s":59.055,"e":59.295},{"text":"reads","s":59.375,"e":59.615},{"text":"dates","s":59.775,"e":59.935},{"text":"as","s":60.055,"e":60.175},{"text":"text,","s":60.415,"e":60.895},{"text":"not","s":60.975,"e":61.055},{"text":"as","s":61.135,"e":61.215},{"text":"numbers,","s":61.295,"e":61.935},{"text":"so","s":62.095,"e":62.255},{"text":"don't","s":62.335,"e":62.495},{"text":"ask","s":62.575,"e":62.815},{"text":"it","s":62.895,"e":62.975},{"text":"which","s":63.055,"e":63.135},{"text":"date","s":63.215,"e":63.375},{"text":"is","s":63.495,"e":63.615},{"text":"earlier.","s":63.695,"e":64.575},{"text":"Ask","s":64.775,"e":64.975},{"text":"three","s":65.095,"e":65.215},{"text":"small","s":65.375,"e":65.615},{"text":"Choices:","s":65.695,"e":66.655},{"text":"which","s":66.775,"e":66.895},{"text":"month,","s":66.975,"e":67.615},{"text":"which","s":67.735,"e":67.855},{"text":"day,","s":67.935,"e":68.575},{"text":"which","s":68.695,"e":68.815},{"text":"year,","s":68.895,"e":69.615},{"text":"each","s":69.735,"e":69.855},{"text":"with","s":69.935,"e":70.015},{"text":"a","s":70.055,"e":70.095},{"text":"not","s":70.175,"e":70.415},{"text":"stated","s":70.495,"e":70.815},{"text":"option.","s":70.895,"e":71.775},{"text":"Here,","s":71.935,"e":72.495},{"text":"March,","s":72.575,"e":73.135},{"text":"the","s":73.175,"e":73.215},{"text":"third,","s":73.295,"e":73.855},{"text":"twenty","s":73.975,"e":74.095},{"text":"twenty","s":74.175,"e":74.335},{"text":"six.","s":74.415,"e":75.375},{"text":"Then","s":75.455,"e":75.695},{"text":"code","s":75.935,"e":76.175},{"text":"builds","s":76.255,"e":76.495},{"text":"the","s":76.575,"e":76.655},{"text":"real","s":76.735,"e":76.975},{"text":"date,","s":77.055,"e":77.428},{"text":"and","s":77.482,"e":77.535},{"text":"does","s":77.655,"e":77.775},{"text":"every","s":77.935,"e":78.095},{"text":"comparison.","s":78.175,"e":79.135}]},{"template":"code","chapter":"Dates are parts","narration":"The model's job ends at the parts. Building the date, checking it's valid, computing a renewal or a deadline, comparing two dates: that's arithmetic, and arithmetic belongs in code. If any part comes back not stated, or with low confidence, ask a person.","params":{"file":"dates.py","lang":"py","lines":["m, d, y = (a[k].choice for k in (\"start_month\", \"start_day\", \"start_year\"))","if \"not_stated\" in (m, d, y) or min(a[k].confidence for k in a) < 0.8:","    return send_to_review(doc)","start = date(int(y), MONTHS.index(m) + 1, int(d))   # validates, too","renewal = start.replace(year=start.year + 1)","days_left = (renewal - today).days                  # code, not the model"],"hl":[{"from":2,"to":3,"label":"missing or unsure: a person","tone":"s2","at":"not stated"},{"from":4,"to":6,"label":"arithmetic stays in code","tone":"good","at":"belongs in code"}]},"start":79.731,"dur":18.191,"words":[{"text":"The","s":80.031,"e":80.191},{"text":"model's","s":80.351,"e":80.591},{"text":"job","s":80.671,"e":80.911},{"text":"ends","s":81.071,"e":81.311},{"text":"at","s":81.391,"e":81.471},{"text":"the","s":81.511,"e":81.551},{"text":"parts.","s":81.631,"e":82.191},{"text":"Building","s":82.591,"e":82.911},{"text":"the","s":82.951,"e":82.991},{"text":"date,","s":83.071,"e":83.631},{"text":"checking","s":83.791,"e":83.951},{"text":"it's","s":83.991,"e":84.111},{"text":"valid,","s":84.191,"e":84.831},{"text":"computing","s":84.991,"e":85.311},{"text":"a","s":85.338,"e":85.364},{"text":"renewal","s":85.391,"e":85.951},{"text":"or","s":85.991,"e":86.031},{"text":"a","s":86.111,"e":86.151},{"text":"deadline,","s":86.191,"e":87.071},{"text":"comparing","s":87.151,"e":87.551},{"text":"two","s":87.631,"e":87.711},{"text":"dates:","s":87.791,"e":88.591},{"text":"that's","s":88.671,"e":88.911},{"text":"arithmetic,","s":88.991,"e":89.951},{"text":"and","s":89.991,"e":90.031},{"text":"arithmetic","s":90.111,"e":90.751},{"text":"belongs","s":90.831,"e":91.231},{"text":"in","s":91.271,"e":91.311},{"text":"code.","s":91.471,"e":91.871},{"text":"If","s":92.351,"e":92.511},{"text":"any","s":92.591,"e":92.751},{"text":"part","s":92.831,"e":93.071},{"text":"comes","s":93.191,"e":93.311},{"text":"back","s":93.391,"e":93.551},{"text":"not","s":93.631,"e":93.871},{"text":"stated,","s":93.951,"e":94.511},{"text":"or","s":94.591,"e":94.751},{"text":"with","s":94.831,"e":94.991},{"text":"low","s":95.071,"e":95.231},{"text":"confidence,","s":95.311,"e":96.191},{"text":"ask","s":96.271,"e":96.431},{"text":"a","s":96.511,"e":96.551},{"text":"person.","s":96.591,"e":97.311}]},{"template":"chart","chapter":"Rerank","narration":"Search is selection too. In a TypeSafe cookbook, a cheap keyword search found thirty candidate passages for each of forty legal questions. The right one was always in the list, but it came first only five percent of the time. One Noul per candidate, is this passage relevant, then sort by probability. Now the right passage came first eighteen percent of the time, and in the top ten sixty two percent.","params":{"title":"40 legal queries · 30 BM25 candidates each","kind":"bar","ymax":0.75,"data":[{"x":"top-1, search only","y":0.05,"label":"5%","at":"five percent"},{"x":"top-1, reranked","y":0.18,"label":"18%","hl":true,"at":"eighteen percent"},{"x":"top-10, search only","y":0.38,"label":"38%","tone":"s2"},{"x":"top-10, reranked","y":0.62,"label":"62%","hl":true,"at":"sixty two"}],"src":"TypeSafe cookbook: Re-ranking (CLERC legal retrieval)"},"start":97.922,"dur":26.759,"words":[{"text":"Search","s":98.222,"e":98.662},{"text":"is","s":98.702,"e":98.822},{"text":"selection","s":98.862,"e":99.315},{"text":"too.","s":99.342,"e":99.742},{"text":"In","s":99.822,"e":100.262},{"text":"a","s":100.302,"e":100.422},{"text":"TypeSafe","s":100.462,"e":101.022},{"text":"cookbook,","s":101.102,"e":101.742},{"text":"a","s":101.822,"e":101.902},{"text":"cheap","s":102.022,"e":102.142},{"text":"keyword","s":102.222,"e":102.622},{"text":"search","s":102.702,"e":102.942},{"text":"found","s":103.022,"e":103.342},{"text":"thirty","s":103.502,"e":103.742},{"text":"candidate","s":103.902,"e":104.222},{"text":"passages","s":104.302,"e":104.862},{"text":"for","s":104.982,"e":105.102},{"text":"each","s":105.182,"e":105.342},{"text":"of","s":105.422,"e":105.582},{"text":"forty","s":105.662,"e":105.982},{"text":"legal","s":106.062,"e":106.302},{"text":"questions.","s":106.382,"e":107.422},{"text":"The","s":107.502,"e":107.582},{"text":"right","s":107.702,"e":107.822},{"text":"one","s":107.942,"e":108.062},{"text":"was","s":108.182,"e":108.302},{"text":"always","s":108.382,"e":108.702},{"text":"in","s":108.782,"e":108.862},{"text":"the","s":108.902,"e":108.942},{"text":"list,","s":109.022,"e":109.662},{"text":"but","s":109.742,"e":109.822},{"text":"it","s":109.862,"e":109.902},{"text":"came","s":109.982,"e":110.222},{"text":"first","s":110.382,"e":110.542},{"text":"only","s":110.702,"e":110.862},{"text":"five","s":110.942,"e":111.102},{"text":"percent","s":111.262,"e":111.502},{"text":"of","s":111.542,"e":111.582},{"text":"the","s":111.662,"e":111.742},{"text":"time.","s":111.942,"e":112.702},{"text":"One","s":112.782,"e":112.862},{"text":"Noul","s":112.942,"e":113.182},{"text":"per","s":113.262,"e":113.342},{"text":"candidate,","s":113.422,"e":114.142},{"text":"is","s":114.222,"e":114.302},{"text":"this","s":114.382,"e":114.542},{"text":"passage","s":114.662,"e":114.942},{"text":"relevant,","s":115.182,"e":115.742},{"text":"then","s":115.862,"e":115.982},{"text":"sort","s":116.062,"e":116.302},{"text":"by","s":116.382,"e":116.462},{"text":"probability.","s":116.542,"e":117.742},{"text":"Now","s":117.942,"e":118.142},{"text":"the","s":118.222,"e":118.302},{"text":"right","s":118.382,"e":118.542},{"text":"passage","s":118.622,"e":119.022},{"text":"came","s":119.142,"e":119.262},{"text":"first","s":119.462,"e":119.662},{"text":"eighteen","s":119.742,"e":120.142},{"text":"percent","s":120.302,"e":120.542},{"text":"of","s":120.582,"e":120.622},{"text":"the","s":120.702,"e":120.782},{"text":"time,","s":120.942,"e":121.342},{"text":"and","s":121.382,"e":121.422},{"text":"in","s":121.502,"e":121.542},{"text":"the","s":121.582,"e":121.662},{"text":"top","s":121.782,"e":121.902},{"text":"ten","s":121.982,"e":122.302},{"text":"sixty","s":122.542,"e":123.022},{"text":"two","s":123.102,"e":123.182},{"text":"percent.","s":123.262,"e":123.982}]},{"template":"statement","chapter":"Find the line","narration":"You can even point at lines. Another cookbook numbers the two hundred eighteen lines of GitHub's terms of service, and uses them as the options of a single Choice: which line answers this question? Plus one Noul: does the document answer it at all? Search without generating a word.","params":{"eyebrow":"Cookbook: line-by-line search","text":"218 numbered lines become *the options of one Choice*.","sub":"Plus one Noul: “Does the document contain an answer?”","sub_at":"Plus one Noul","art":"search","tone":"agent","punch":"two hundred eighteen"},"start":124.681,"dur":19.157,"words":[{"text":"You","s":124.981,"e":125.141},{"text":"can","s":125.181,"e":125.221},{"text":"even","s":125.301,"e":125.541},{"text":"point","s":125.701,"e":126.021},{"text":"at","s":126.101,"e":126.181},{"text":"lines.","s":126.461,"e":126.981},{"text":"Another","s":127.221,"e":127.541},{"text":"cookbook","s":127.621,"e":128.021},{"text":"numbers","s":128.101,"e":128.421},{"text":"the","s":128.501,"e":128.581},{"text":"two","s":128.661,"e":128.741},{"text":"hundred","s":128.901,"e":129.061},{"text":"eighteen","s":129.221,"e":129.621},{"text":"lines","s":129.701,"e":130.021},{"text":"of","s":130.101,"e":130.181},{"text":"GitHub's","s":130.261,"e":130.661},{"text":"terms","s":130.741,"e":130.981},{"text":"of","s":131.061,"e":131.141},{"text":"service,","s":131.221,"e":132.021},{"text":"and","s":132.101,"e":132.181},{"text":"uses","s":132.261,"e":132.581},{"text":"them","s":132.661,"e":132.821},{"text":"as","s":132.901,"e":132.981},{"text":"the","s":133.061,"e":133.141},{"text":"options","s":133.221,"e":133.541},{"text":"of","s":133.621,"e":133.701},{"text":"a","s":133.741,"e":133.781},{"text":"single","s":133.941,"e":134.181},{"text":"Choice:","s":134.341,"e":135.301},{"text":"which","s":135.421,"e":135.541},{"text":"line","s":135.621,"e":135.781},{"text":"answers","s":135.861,"e":136.181},{"text":"this","s":136.261,"e":136.421},{"text":"question?","s":136.501,"e":137.061},{"text":"Plus","s":137.461,"e":137.621},{"text":"one","s":137.701,"e":137.861},{"text":"Noul:","s":137.941,"e":138.661},{"text":"does","s":138.741,"e":138.821},{"text":"the","s":138.861,"e":138.901},{"text":"document","s":138.981,"e":139.381},{"text":"answer","s":139.461,"e":139.701},{"text":"it","s":139.781,"e":139.861},{"text":"at","s":139.941,"e":140.021},{"text":"all?","s":140.101,"e":140.781},{"text":"Search","s":141.141,"e":141.381},{"text":"without","s":141.461,"e":141.861},{"text":"generating","s":141.941,"e":142.421},{"text":"a","s":142.461,"e":142.501},{"text":"word.","s":142.581,"e":143.221}]},{"template":"flow","chapter":"Deep taxonomies","narration":"And for taxonomies with thousands of leaves, like patent classes or product categories, go level by level. At each level, a Choice picks among the children. Keep the best few paths, not just one, so a near miss early on can recover. That's a beam search over the model's probabilities.","params":{"title":"Level by level, keep the best few paths","nodes":[{"label":"Level 1","sub":"Choice among top classes","kind":"snap","edge":"top 3 paths","at":"level by level"},{"label":"Level 2","sub":"Choice among children","kind":"snap","edge":"top 3 paths","at":"Keep the best few"},{"label":"Leaf","sub":"best full path wins","kind":"curly","at":"beam search"}]},"start":143.838,"dur":19.549,"words":[{"text":"And","s":144.138,"e":144.298},{"text":"for","s":144.378,"e":144.458},{"text":"taxonomies","s":144.618,"e":145.178},{"text":"with","s":145.298,"e":145.418},{"text":"thousands","s":145.498,"e":145.978},{"text":"of","s":146.058,"e":146.138},{"text":"leaves,","s":146.218,"e":146.698},{"text":"like","s":146.778,"e":147.018},{"text":"patent","s":147.098,"e":147.418},{"text":"classes","s":147.738,"e":148.058},{"text":"or","s":148.138,"e":148.218},{"text":"product","s":148.378,"e":148.698},{"text":"categories,","s":148.778,"e":149.738},{"text":"go","s":149.818,"e":149.978},{"text":"level","s":150.138,"e":150.458},{"text":"by","s":150.538,"e":150.618},{"text":"level.","s":150.778,"e":151.578},{"text":"At","s":151.658,"e":151.738},{"text":"each","s":151.818,"e":151.978},{"text":"level,","s":152.058,"e":152.538},{"text":"a","s":152.618,"e":152.698},{"text":"Choice","s":152.898,"e":153.098},{"text":"picks","s":153.178,"e":153.418},{"text":"among","s":153.538,"e":153.658},{"text":"the","s":153.698,"e":153.738},{"text":"children.","s":153.818,"e":154.698},{"text":"Keep","s":154.778,"e":154.858},{"text":"the","s":154.938,"e":155.098},{"text":"best","s":155.258,"e":155.418},{"text":"few","s":155.498,"e":155.658},{"text":"paths,","s":155.898,"e":156.218},{"text":"not","s":156.298,"e":156.458},{"text":"just","s":156.578,"e":156.698},{"text":"one,","s":156.778,"e":157.258},{"text":"so","s":157.338,"e":157.418},{"text":"a","s":157.498,"e":157.578},{"text":"near","s":157.698,"e":157.818},{"text":"miss","s":157.898,"e":158.138},{"text":"early","s":158.298,"e":158.458},{"text":"on","s":158.538,"e":158.778},{"text":"can","s":158.858,"e":158.938},{"text":"recover.","s":159.418,"e":159.951},{"text":"That's","s":160.005,"e":160.138},{"text":"a","s":160.218,"e":160.298},{"text":"beam","s":160.418,"e":160.538},{"text":"search","s":160.618,"e":160.938},{"text":"over","s":161.018,"e":161.178},{"text":"the","s":161.218,"e":161.258},{"text":"model's","s":161.338,"e":161.658},{"text":"probabilities.","s":161.738,"e":162.778}]},{"template":"quiz","chapter":"Quick check","narration":"Quick check. You need the invoice total from a PDF's text. What's the select-don't-generate way to get it?","params":{"q":"Get the **invoice total** from extracted PDF text. The select-don't-generate way?","options":["Ask a Score for the amount, from 0 to 10,000","Regex every money amount, ask a Choice which one is the total (plus “none”), copy it","Ask a Noul for each possible number from 1 to 10,000","Ask the model to type the total"],"answer":1,"explain":"Code finds the candidates, the model picks the one that's the total, and code copies it exactly. The model never has to produce digits."},"start":163.387,"dur":12.657,"words":[{"text":"Quick","s":163.687,"e":163.927},{"text":"check.","s":164.007,"e":164.407},{"text":"You","s":164.807,"e":164.887},{"text":"need","s":164.967,"e":165.127},{"text":"the","s":165.167,"e":165.207},{"text":"invoice","s":165.287,"e":165.687},{"text":"total","s":165.767,"e":166.007},{"text":"from","s":166.087,"e":166.247},{"text":"a","s":166.287,"e":166.327},{"text":"PDF's","s":166.407,"e":166.887},{"text":"text.","s":166.967,"e":167.647},{"text":"What's","s":167.847,"e":168.087},{"text":"the","s":168.167,"e":168.327},{"text":"select-don't-generate","s":168.487,"e":169.927},{"text":"way","s":170.007,"e":170.087},{"text":"to","s":170.167,"e":170.207},{"text":"get","s":170.247,"e":170.407},{"text":"it?","s":170.487,"e":170.887}]},{"template":"exercise","chapter":"Try it","narration":"Try it. Have your agent replace one generated extraction in your code with find, choose, copy.","params":{"title":"Find, choose, copy","intro":"Paste this into your coding agent:","prompt":"Find one place where we ask an LLM to extract a value (an amount, email,\nID, date, name) from text. Replace it with select-don't-generate:\ncandidates from regex/parsing in code, one Choice that picks the\nrequested candidate (always with a \"none\" option), and code that copies\nthe chosen string verbatim and normalizes it. For dates, ask month/day/\nyear as separate Choices with \"not_stated\". Compare both versions on\n50 real documents."},"start":176.044,"dur":7.872,"words":[{"text":"Try","s":176.344,"e":176.584},{"text":"it.","s":176.664,"e":176.904},{"text":"Have","s":177.224,"e":177.384},{"text":"your","s":177.464,"e":177.544},{"text":"agent","s":177.624,"e":177.864},{"text":"replace","s":178.104,"e":178.344},{"text":"one","s":178.504,"e":178.664},{"text":"generated","s":178.744,"e":179.224},{"text":"extraction","s":179.344,"e":179.864},{"text":"in","s":179.904,"e":179.944},{"text":"your","s":180.024,"e":180.104},{"text":"code","s":180.184,"e":180.504},{"text":"with","s":180.584,"e":180.824},{"text":"find,","s":181.144,"e":181.864},{"text":"choose,","s":182.104,"e":182.584},{"text":"copy.","s":182.784,"e":183.304}]},{"template":"recap","chapter":"Recap","narration":"So: when you need a value, select it, don't generate it. Code finds candidates, the model picks, code copies. Dates are parts. Reranking and line search are selection. And deep taxonomies go level by level. Next: using decision models to check other models' work.","params":{"items":["Most extraction is selection: the value is already in the text.","Code finds candidates; the model picks; code copies verbatim.","Dates: a Choice per part, arithmetic in code.","Rerank with a Noul per candidate; walk taxonomies level by level."],"next":"Verify and guard","next_at":"Next"},"start":183.916,"dur":21.857,"words":[{"text":"So:","s":184.216,"e":184.936},{"text":"when","s":185.016,"e":185.096},{"text":"you","s":185.176,"e":185.256},{"text":"need","s":185.336,"e":185.416},{"text":"a","s":185.456,"e":185.496},{"text":"value,","s":185.576,"e":186.296},{"text":"select","s":186.376,"e":186.776},{"text":"it,","s":186.856,"e":187.256},{"text":"don't","s":187.336,"e":187.576},{"text":"generate","s":187.656,"e":188.056},{"text":"it.","s":188.136,"e":188.536},{"text":"Code","s":188.776,"e":189.016},{"text":"finds","s":189.096,"e":189.376},{"text":"candidates,","s":189.416,"e":190.216},{"text":"the","s":190.296,"e":190.376},{"text":"model","s":190.496,"e":190.616},{"text":"picks,","s":190.776,"e":191.416},{"text":"code","s":191.496,"e":191.816},{"text":"copies.","s":191.896,"e":192.936},{"text":"Dates","s":193.136,"e":193.336},{"text":"are","s":193.416,"e":193.576},{"text":"parts.","s":193.656,"e":194.536},{"text":"Reranking","s":194.576,"e":195.176},{"text":"and","s":195.216,"e":195.256},{"text":"line","s":195.336,"e":195.576},{"text":"search","s":195.656,"e":195.976},{"text":"are","s":196.056,"e":196.136},{"text":"selection.","s":196.216,"e":197.176},{"text":"And","s":197.256,"e":197.336},{"text":"deep","s":197.416,"e":197.576},{"text":"taxonomies","s":197.696,"e":198.296},{"text":"go","s":198.456,"e":198.536},{"text":"level","s":198.696,"e":199.016},{"text":"by","s":199.096,"e":199.256},{"text":"level.","s":199.456,"e":200.216},{"text":"Next:","s":200.296,"e":201.176},{"text":"using","s":201.256,"e":201.496},{"text":"decision","s":201.576,"e":201.976},{"text":"models","s":202.056,"e":202.456},{"text":"to","s":202.496,"e":202.536},{"text":"check","s":202.616,"e":202.936},{"text":"other","s":203.056,"e":203.176},{"text":"models'","s":203.256,"e":203.616},{"text":"work.","s":203.656,"e":204.216}]}],"total":206.173,"audio":"/audio/15-select.mp3"}