{"id":"03-how-it-works","number":3,"slug":"how-decision-models-work","title":"How a decision model works","short":"How it works","module":"The idea","summary":"A language model writes its answer one token at a time. A decision model reads your state once and scores every allowed answer in a single pass. Here is the core trick in a few lines of Python, what production models add on top (trained readout heads, shared state, calibration training), and why typed output is not the same as a correct answer.","voice":"Jessica","scenes":[{"template":"title","chapter":"Intro","narration":"What's actually inside a decision model? Less magic than you'd think. In this lesson, you'll see the core trick in a few lines of code, and what the real models add on top.","params":{"kicker":"Lesson 3","module":"Module 1 · The idea","title":"How a decision model *works*","subtitle":"Read once. Score every allowed answer. Return a distribution.","cast":["snap","curly"]},"start":0.5,"dur":12.354,"words":[{"text":"What's","s":0.8,"e":1.12},{"text":"actually","s":1.28,"e":1.76},{"text":"inside","s":1.84,"e":2.4},{"text":"a","s":2.48,"e":2.52},{"text":"decision","s":2.56,"e":2.96},{"text":"model?","s":3.04,"e":3.44},{"text":"Less","s":3.92,"e":4.16},{"text":"magic","s":4.24,"e":4.56},{"text":"than","s":4.64,"e":4.72},{"text":"you'd","s":4.76,"e":4.96},{"text":"think.","s":5.16,"e":5.56},{"text":"In","s":5.76,"e":5.84},{"text":"this","s":5.92,"e":6.08},{"text":"lesson,","s":6.16,"e":6.72},{"text":"you'll","s":6.76,"e":6.96},{"text":"see","s":7.04,"e":7.12},{"text":"the","s":7.2,"e":7.28},{"text":"core","s":7.36,"e":7.6},{"text":"trick","s":7.68,"e":8},{"text":"in","s":8.04,"e":8.08},{"text":"a","s":8.16,"e":8.24},{"text":"few","s":8.32,"e":8.48},{"text":"lines","s":8.56,"e":8.8},{"text":"of","s":8.84,"e":8.88},{"text":"code,","s":8.96,"e":9.6},{"text":"and","s":9.68,"e":9.92},{"text":"what","s":10,"e":10.08},{"text":"the","s":10.16,"e":10.24},{"text":"real","s":10.32,"e":10.56},{"text":"models","s":10.64,"e":10.96},{"text":"add","s":11.04,"e":11.2},{"text":"on","s":11.28,"e":11.36},{"text":"top.","s":11.44,"e":12}]},{"template":"compare","chapter":"Generate vs score","narration":"Start with a normal language model. It writes its answer one token at a time, and every token is another full pass through the network. A fifty token answer is fifty passes, one after another. A decision model never writes. It reads your state once, and scores every allowed answer in a single pass.","params":{"left":{"title":"Language model","who":"mull","tone":"s2","items":[{"text":"pass 1 → `{`","at":"one token at a time"},{"text":"pass 2 → `\"team\"`"},{"text":"pass 3 → `:`"},{"text":"… pass 50 → `}`","at":"fifty passes"}]},"right":{"title":"Decision model","who":"snap","tone":"good","items":[{"text":"pass 1 → billing 0.95 · technical 0.03 · other 0.02","at":"never writes"}],"at":"never writes"},"vs":"vs"},"start":12.854,"dur":20.672,"words":[{"text":"Start","s":13.154,"e":13.474},{"text":"with","s":13.554,"e":13.634},{"text":"a","s":13.714,"e":13.794},{"text":"normal","s":13.874,"e":14.194},{"text":"language","s":14.394,"e":14.594},{"text":"model.","s":14.674,"e":15.234},{"text":"It","s":15.474,"e":15.554},{"text":"writes","s":15.674,"e":15.794},{"text":"its","s":15.874,"e":15.954},{"text":"answer","s":16.034,"e":16.434},{"text":"one","s":16.594,"e":16.754},{"text":"token","s":16.834,"e":17.154},{"text":"at","s":17.194,"e":17.234},{"text":"a","s":17.314,"e":17.354},{"text":"time,","s":17.394,"e":17.954},{"text":"and","s":18.074,"e":18.194},{"text":"every","s":18.274,"e":18.514},{"text":"token","s":18.594,"e":18.994},{"text":"is","s":19.074,"e":19.154},{"text":"another","s":19.354,"e":19.554},{"text":"full","s":19.794,"e":19.954},{"text":"pass","s":20.034,"e":20.354},{"text":"through","s":20.434,"e":20.514},{"text":"the","s":20.554,"e":20.594},{"text":"network.","s":20.674,"e":21.154},{"text":"A","s":21.554,"e":21.714},{"text":"fifty","s":21.794,"e":22.034},{"text":"token","s":22.114,"e":22.354},{"text":"answer","s":22.434,"e":22.834},{"text":"is","s":22.914,"e":23.074},{"text":"fifty","s":23.154,"e":23.474},{"text":"passes,","s":23.554,"e":24.354},{"text":"one","s":24.434,"e":24.674},{"text":"after","s":24.754,"e":24.994},{"text":"another.","s":25.074,"e":25.794},{"text":"A","s":26.114,"e":26.194},{"text":"decision","s":26.434,"e":26.674},{"text":"model","s":26.754,"e":26.994},{"text":"never","s":27.234,"e":27.474},{"text":"writes.","s":27.554,"e":28.234},{"text":"It","s":28.434,"e":28.514},{"text":"reads","s":28.594,"e":28.834},{"text":"your","s":28.874,"e":28.914},{"text":"state","s":28.994,"e":29.234},{"text":"once,","s":29.314,"e":29.794},{"text":"and","s":29.834,"e":29.874},{"text":"scores","s":29.954,"e":30.354},{"text":"every","s":30.434,"e":30.674},{"text":"allowed","s":30.754,"e":31.074},{"text":"answer","s":31.274,"e":31.474},{"text":"in","s":31.554,"e":31.634},{"text":"a","s":31.714,"e":31.794},{"text":"single","s":31.874,"e":32.114},{"text":"pass.","s":32.194,"e":32.914}]},{"template":"code","chapter":"The core trick","narration":"Here's the core trick, adapted from a popular post called Jev in twenty five lines of Python. Put the options in the prompt with letters. Run a small open model once. Then, instead of letting it write, read the probability it gives each letter as the next token, and normalize.","params":{"file":"tiny_decider.py","lang":"py","badge":"runs locally","lines":["labels  = [\"A\", \"B\", \"C\"]","choices = [\"Legitimate\", \"Spam\", \"Phishing\"]","email   = \"Payroll asks for your password on a fake page.\"","prompt  = f\"Choose one.\\nEmail: {email}\\nA. Legitimate\\nB. Spam\\nC. Phishing\"","model.eval(model.tokenize(prompt))","logits = model.scores[-1]","x = [logits[token_id(l)] for l in labels]","p = softmax(x)"],"hl":[{"from":1,"to":4,"label":"options in the prompt","tone":"agent","at":"with letters"},{"from":5,"to":5,"label":"run once","tone":"s2","at":"once"},{"from":6,"to":8,"label":"read, don't write","tone":"good","at":"instead of letting it write"}]},"start":33.526,"dur":18.583,"words":[{"text":"Here's","s":33.826,"e":34.146},{"text":"the","s":34.226,"e":34.306},{"text":"core","s":34.386,"e":34.626},{"text":"trick,","s":34.706,"e":35.266},{"text":"adapted","s":35.346,"e":35.746},{"text":"from","s":35.826,"e":35.906},{"text":"a","s":35.986,"e":36.066},{"text":"popular","s":36.306,"e":36.546},{"text":"post","s":36.706,"e":36.866},{"text":"called","s":36.946,"e":37.266},{"text":"Jev","s":37.506,"e":37.826},{"text":"in","s":37.906,"e":37.986},{"text":"twenty","s":38.066,"e":38.306},{"text":"five","s":38.386,"e":38.546},{"text":"lines","s":38.626,"e":38.866},{"text":"of","s":38.946,"e":39.026},{"text":"Python.","s":39.266,"e":39.586},{"text":"Put","s":40.066,"e":40.226},{"text":"the","s":40.266,"e":40.306},{"text":"options","s":40.386,"e":40.786},{"text":"in","s":40.826,"e":40.866},{"text":"the","s":40.906,"e":40.946},{"text":"prompt","s":41.026,"e":41.346},{"text":"with","s":41.426,"e":41.506},{"text":"letters.","s":41.586,"e":42.346},{"text":"Run","s":42.546,"e":42.706},{"text":"a","s":42.786,"e":42.866},{"text":"small","s":43.026,"e":43.186},{"text":"open","s":43.346,"e":43.586},{"text":"model","s":43.666,"e":43.906},{"text":"once.","s":44.066,"e":44.746},{"text":"Then,","s":44.946,"e":45.426},{"text":"instead","s":45.586,"e":45.746},{"text":"of","s":45.826,"e":45.906},{"text":"letting","s":45.946,"e":46.146},{"text":"it","s":46.186,"e":46.226},{"text":"write,","s":46.386,"e":47.106},{"text":"read","s":47.186,"e":47.346},{"text":"the","s":47.426,"e":47.466},{"text":"probability","s":47.506,"e":48.146},{"text":"it","s":48.226,"e":48.306},{"text":"gives","s":48.413,"e":48.626},{"text":"each","s":48.746,"e":48.866},{"text":"letter","s":48.946,"e":49.186},{"text":"as","s":49.266,"e":49.346},{"text":"the","s":49.426,"e":49.466},{"text":"next","s":49.506,"e":49.746},{"text":"token,","s":49.986,"e":50.386},{"text":"and","s":50.466,"e":50.546},{"text":"normalize.","s":50.626,"e":51.506}]},{"template":"dist","chapter":"The core trick","narration":"On a tiny model, that phishing email comes out about eighty nine percent phishing, eight percent spam, and three percent legitimate. A distribution over exactly your options. Nothing to parse, and no way to answer outside the list.","params":{"state":"Payroll asks for your password on a non-company sign-in page.","question":"Which option is this email?","options":[{"label":"Phishing","p":0.885},{"label":"Spam","p":0.084},{"label":"Legitimate","p":0.031}],"at":"eighty nine percent","gauge":false,"more":"Qwen3-0.6B, one forward pass (NobodyWho's example)","hnote":{"text":"only your options exist","at":"outside the list","p":[74,40],"pp":[62,66]}},"start":52.109,"dur":14.978,"words":[{"text":"On","s":52.409,"e":52.569},{"text":"a","s":52.649,"e":52.729},{"text":"tiny","s":52.929,"e":53.129},{"text":"model,","s":53.209,"e":53.769},{"text":"that","s":53.889,"e":54.009},{"text":"phishing","s":54.089,"e":54.329},{"text":"email","s":54.409,"e":54.729},{"text":"comes","s":54.809,"e":54.969},{"text":"out","s":55.049,"e":55.129},{"text":"about","s":55.209,"e":55.529},{"text":"eighty","s":55.609,"e":55.849},{"text":"nine","s":55.929,"e":56.009},{"text":"percent","s":56.089,"e":56.409},{"text":"phishing,","s":56.489,"e":57.209},{"text":"eight","s":57.289,"e":57.369},{"text":"percent","s":57.449,"e":57.769},{"text":"spam,","s":57.849,"e":58.409},{"text":"and","s":58.529,"e":58.649},{"text":"three","s":58.729,"e":58.889},{"text":"percent","s":59.049,"e":59.369},{"text":"legitimate.","s":59.449,"e":60.009},{"text":"A","s":60.409,"e":60.489},{"text":"distribution","s":60.809,"e":61.129},{"text":"over","s":61.209,"e":61.449},{"text":"exactly","s":61.529,"e":62.169},{"text":"your","s":62.249,"e":62.329},{"text":"options.","s":62.409,"e":63.129},{"text":"Nothing","s":63.289,"e":63.529},{"text":"to","s":63.569,"e":63.609},{"text":"parse,","s":63.689,"e":64.169},{"text":"and","s":64.249,"e":64.329},{"text":"no","s":64.489,"e":64.569},{"text":"way","s":64.649,"e":64.729},{"text":"to","s":64.809,"e":64.889},{"text":"answer","s":64.969,"e":65.209},{"text":"outside","s":65.289,"e":65.609},{"text":"the","s":65.649,"e":65.689},{"text":"list.","s":65.769,"e":66.489}]},{"template":"stack","chapter":"What real models add","narration":"Real decision models start from that idea and add three layers. A trained readout head, instead of guessing letter tokens. A shared state, read once, so many questions cost almost nothing extra. And training aimed at one thing: probabilities you can trust.","params":{"title":"What production models add","sub":"On top of a pretrained language model torso.","layers":[{"label":"A pretrained model torso","sub":"Qwen, Gemma or your own; the language head is removed","tone":"paper","icon":"brain","at":"that idea"},{"label":"A trained readout head","sub":"scores each allowed option; no text generation","tone":"ink","icon":"bolt","at":"readout head"},{"label":"Shared state, many questions","sub":"the state is read once; each question adds only its own tokens","tone":"s2","icon":"layers","at":"shared state"},{"label":"Calibration training","sub":"Brier loss, RLCD-style rewards, distillation","tone":"agent","icon":"target","at":"probabilities you can trust"}]},"start":67.087,"dur":18.191,"words":[{"text":"Real","s":67.387,"e":67.627},{"text":"decision","s":67.707,"e":68.107},{"text":"models","s":68.187,"e":68.587},{"text":"start","s":68.747,"e":68.907},{"text":"from","s":69.027,"e":69.147},{"text":"that","s":69.227,"e":69.307},{"text":"idea","s":69.387,"e":69.867},{"text":"and","s":69.947,"e":70.107},{"text":"add","s":70.227,"e":70.347},{"text":"three","s":70.427,"e":70.667},{"text":"layers.","s":70.747,"e":71.507},{"text":"A","s":71.707,"e":71.867},{"text":"trained","s":72.027,"e":72.187},{"text":"readout","s":72.267,"e":72.587},{"text":"head,","s":72.667,"e":73.067},{"text":"instead","s":73.147,"e":73.467},{"text":"of","s":73.507,"e":73.547},{"text":"guessing","s":73.707,"e":74.027},{"text":"letter","s":74.107,"e":74.267},{"text":"tokens.","s":74.347,"e":75.307},{"text":"A","s":75.387,"e":75.467},{"text":"shared","s":75.547,"e":75.787},{"text":"state,","s":75.947,"e":76.587},{"text":"read","s":76.707,"e":76.827},{"text":"once,","s":76.907,"e":77.467},{"text":"so","s":77.547,"e":77.627},{"text":"many","s":77.707,"e":77.867},{"text":"questions","s":77.947,"e":78.347},{"text":"cost","s":78.427,"e":78.667},{"text":"almost","s":78.867,"e":79.067},{"text":"nothing","s":79.147,"e":79.467},{"text":"extra.","s":79.547,"e":80.427},{"text":"And","s":80.507,"e":80.587},{"text":"training","s":80.827,"e":81.067},{"text":"aimed","s":81.147,"e":81.307},{"text":"at","s":81.387,"e":81.627},{"text":"one","s":81.747,"e":81.867},{"text":"thing:","s":81.947,"e":82.587},{"text":"probabilities","s":82.707,"e":83.467},{"text":"you","s":83.547,"e":83.627},{"text":"can","s":83.707,"e":83.867},{"text":"trust.","s":84.107,"e":84.667}]},{"template":"quote","chapter":"What real models add","narration":"AWS's open Strands Decider is a good look inside. They take a two billion parameter model, throw away the part that generates words, and add a tiny pointer head of about a million parameters that compares each option to the question. One forward pass, no generation, no decoding loop.","params":{"text":"One forward pass, *no generation, no decoding loop*.","who":"Strands Decider README","role":"AWS Strands Labs","src":"github.com/strands-labs/strands-decider"},"start":85.278,"dur":19.236,"words":[{"text":"AWS's","s":85.578,"e":86.458},{"text":"open","s":86.538,"e":86.858},{"text":"Strands","s":86.938,"e":87.258},{"text":"Decider","s":87.338,"e":87.898},{"text":"is","s":87.978,"e":88.058},{"text":"a","s":88.138,"e":88.218},{"text":"good","s":88.298,"e":88.458},{"text":"look","s":88.538,"e":88.698},{"text":"inside.","s":88.978,"e":89.738},{"text":"They","s":89.818,"e":89.898},{"text":"take","s":89.978,"e":90.058},{"text":"a","s":90.138,"e":90.298},{"text":"two","s":90.378,"e":90.458},{"text":"billion","s":90.698,"e":91.018},{"text":"parameter","s":91.098,"e":91.498},{"text":"model,","s":91.578,"e":92.218},{"text":"throw","s":92.298,"e":92.538},{"text":"away","s":92.658,"e":92.778},{"text":"the","s":92.818,"e":92.858},{"text":"part","s":92.938,"e":93.098},{"text":"that","s":93.178,"e":93.258},{"text":"generates","s":93.378,"e":93.658},{"text":"words,","s":93.738,"e":94.458},{"text":"and","s":94.538,"e":94.618},{"text":"add","s":94.698,"e":94.778},{"text":"a","s":94.858,"e":94.938},{"text":"tiny","s":95.138,"e":95.338},{"text":"pointer","s":95.498,"e":95.818},{"text":"head","s":95.898,"e":96.058},{"text":"of","s":96.138,"e":96.218},{"text":"about","s":96.338,"e":96.458},{"text":"a","s":96.538,"e":96.618},{"text":"million","s":96.698,"e":97.018},{"text":"parameters","s":97.338,"e":97.658},{"text":"that","s":97.738,"e":97.818},{"text":"compares","s":97.898,"e":98.298},{"text":"each","s":98.378,"e":98.538},{"text":"option","s":98.618,"e":99.018},{"text":"to","s":99.058,"e":99.098},{"text":"the","s":99.138,"e":99.178},{"text":"question.","s":99.258,"e":100.218},{"text":"One","s":100.298,"e":100.458},{"text":"forward","s":100.538,"e":100.778},{"text":"pass,","s":100.858,"e":101.498},{"text":"no","s":101.578,"e":101.658},{"text":"generation,","s":101.738,"e":102.698},{"text":"no","s":102.778,"e":102.858},{"text":"decoding","s":102.938,"e":103.338},{"text":"loop.","s":103.418,"e":103.898}]},{"template":"table","chapter":"Three training goals","narration":"The training goal is where these models differ most from chatbots. Reinforcement learning from human feedback, which made ChatGPT, rewards answers people prefer. Verifiable rewards made reasoning models good at math and code. TypeSafe calls its third path reinforcement learning for calibrated decisions: reward probabilities that match how often the answer turns out right.","params":{"title":"Three post-training goals","cols":["Method","Rewards","Gave us"],"rows":[{"cells":["RLHF","answers people prefer","chat assistants"],"at":"human feedback"},{"cells":["RLVR","answers a checker can verify","reasoning models"],"at":"Verifiable rewards"},{"cells":["RLCD","*probabilities that match outcomes*","decision models"],"hl":true,"at":"calibrated decisions"}],"note":"TypeSafe has not published how RLCD works. Cloudflare trains Clef with cross-entropy plus a Brier loss."},"start":104.514,"dur":24.826,"words":[{"text":"The","s":104.814,"e":104.974},{"text":"training","s":105.054,"e":105.374},{"text":"goal","s":105.454,"e":105.774},{"text":"is","s":105.854,"e":105.934},{"text":"where","s":106.014,"e":106.174},{"text":"these","s":106.254,"e":106.334},{"text":"models","s":106.414,"e":106.734},{"text":"differ","s":106.814,"e":107.054},{"text":"most","s":107.214,"e":107.534},{"text":"from","s":107.614,"e":107.774},{"text":"chatbots.","s":107.934,"e":108.574},{"text":"Reinforcement","s":109.054,"e":109.694},{"text":"learning","s":109.774,"e":110.014},{"text":"from","s":110.094,"e":110.254},{"text":"human","s":110.334,"e":110.654},{"text":"feedback,","s":110.734,"e":111.374},{"text":"which","s":111.454,"e":111.534},{"text":"made","s":111.614,"e":111.694},{"text":"ChatGPT,","s":111.774,"e":112.974},{"text":"rewards","s":113.054,"e":113.534},{"text":"answers","s":113.614,"e":114.014},{"text":"people","s":114.094,"e":114.334},{"text":"prefer.","s":114.414,"e":114.974},{"text":"Verifiable","s":115.534,"e":116.174},{"text":"rewards","s":116.254,"e":116.734},{"text":"made","s":116.814,"e":116.974},{"text":"reasoning","s":117.134,"e":117.454},{"text":"models","s":117.534,"e":117.854},{"text":"good","s":117.934,"e":118.094},{"text":"at","s":118.134,"e":118.174},{"text":"math","s":118.254,"e":118.654},{"text":"and","s":118.694,"e":118.734},{"text":"code.","s":118.894,"e":119.294},{"text":"TypeSafe","s":119.774,"e":120.414},{"text":"calls","s":120.494,"e":120.654},{"text":"its","s":120.734,"e":120.894},{"text":"third","s":120.974,"e":121.214},{"text":"path","s":121.294,"e":121.614},{"text":"reinforcement","s":121.774,"e":122.494},{"text":"learning","s":122.574,"e":122.814},{"text":"for","s":122.894,"e":122.974},{"text":"calibrated","s":123.054,"e":123.614},{"text":"decisions:","s":123.694,"e":124.894},{"text":"reward","s":125.094,"e":125.294},{"text":"probabilities","s":125.374,"e":126.094},{"text":"that","s":126.174,"e":126.254},{"text":"match","s":126.334,"e":126.574},{"text":"how","s":126.694,"e":126.814},{"text":"often","s":126.974,"e":127.134},{"text":"the","s":127.174,"e":127.214},{"text":"answer","s":127.294,"e":127.614},{"text":"turns","s":127.694,"e":127.934},{"text":"out","s":128.014,"e":128.094},{"text":"right.","s":128.174,"e":128.734}]},{"template":"statement","chapter":"Typed is not correct","narration":"Now, the big claim at launch was that Jev can't hallucinate. Hacker News pushed back fast. As one commenter put it, it can still forward a billing query to the dev department. Even TypeSafe's founder agreed in the thread: these models are probabilistic, so they can be confidently wrong.","params":{"eyebrow":"Hacker News, launch day","text":"It can still forward a billing query *to the dev department*.","sub":"Typed output guarantees the shape of the answer, not that it's right.","sub_at":"confidently wrong","art":"snap","mood":"unsure","react":{"at":"confidently wrong","mood":"unsure"}},"start":129.34,"dur":19.471,"words":[{"text":"Now,","s":129.64,"e":130.12},{"text":"the","s":130.2,"e":130.36},{"text":"big","s":130.48,"e":130.6},{"text":"claim","s":130.76,"e":130.92},{"text":"at","s":131,"e":131.08},{"text":"launch","s":131.28,"e":131.48},{"text":"was","s":131.56,"e":131.64},{"text":"that","s":131.72,"e":131.8},{"text":"Jev","s":131.88,"e":132.2},{"text":"can't","s":132.36,"e":132.68},{"text":"hallucinate.","s":132.76,"e":133.48},{"text":"Hacker","s":133.88,"e":134.2},{"text":"News","s":134.28,"e":134.52},{"text":"pushed","s":134.68,"e":134.92},{"text":"back","s":135,"e":135.24},{"text":"fast.","s":135.48,"e":136.04},{"text":"As","s":136.44,"e":136.6},{"text":"one","s":136.68,"e":136.76},{"text":"commenter","s":136.84,"e":137.32},{"text":"put","s":137.4,"e":137.48},{"text":"it,","s":137.56,"e":137.88},{"text":"it","s":137.92,"e":137.96},{"text":"can","s":138.04,"e":138.2},{"text":"still","s":138.32,"e":138.44},{"text":"forward","s":138.52,"e":138.84},{"text":"a","s":138.88,"e":138.92},{"text":"billing","s":139,"e":139.32},{"text":"query","s":139.4,"e":139.64},{"text":"to","s":139.72,"e":139.8},{"text":"the","s":139.84,"e":139.88},{"text":"dev","s":139.96,"e":140.12},{"text":"department.","s":140.2,"e":140.76},{"text":"Even","s":141.24,"e":141.48},{"text":"TypeSafe's","s":141.56,"e":142.12},{"text":"founder","s":142.28,"e":142.52},{"text":"agreed","s":142.6,"e":142.92},{"text":"in","s":142.96,"e":143},{"text":"the","s":143.04,"e":143.08},{"text":"thread:","s":143.16,"e":143.96},{"text":"these","s":144.04,"e":144.2},{"text":"models","s":144.28,"e":144.6},{"text":"are","s":144.68,"e":144.84},{"text":"probabilistic,","s":144.96,"e":145.96},{"text":"so","s":146.08,"e":146.2},{"text":"they","s":146.36,"e":146.52},{"text":"can","s":146.56,"e":146.6},{"text":"be","s":146.68,"e":146.76},{"text":"confidently","s":146.84,"e":147.4},{"text":"wrong.","s":147.48,"e":148.2}]},{"template":"noul","chapter":"Typed is not correct","narration":"So what you get is a probability, and the real promise is that the probability is honest. When a decision model says point nine, it should be right about nine times out of ten. Whether that holds on your data is something you'll learn to measure in module three.","params":{"question":"What a well calibrated 0.9 means","rows":[{"text":"Says 0.9 on 1,000 answers","p":0.9,"at":"point nine"},{"text":"About 900 should be right","p":0.9,"at":"nine times out of ten"},{"text":"And about 100 will be wrong","p":0.1,"at":"right about"}]},"start":148.811,"dur":16.336,"words":[{"text":"So","s":149.111,"e":149.271},{"text":"what","s":149.351,"e":149.431},{"text":"you","s":149.511,"e":149.671},{"text":"get","s":149.831,"e":149.991},{"text":"is","s":150.071,"e":150.151},{"text":"a","s":150.191,"e":150.231},{"text":"probability,","s":150.311,"e":151.431},{"text":"and","s":151.471,"e":151.511},{"text":"the","s":151.591,"e":151.671},{"text":"real","s":151.751,"e":151.991},{"text":"promise","s":152.071,"e":152.551},{"text":"is","s":152.631,"e":152.711},{"text":"that","s":152.791,"e":152.871},{"text":"the","s":152.951,"e":153.031},{"text":"probability","s":153.391,"e":153.751},{"text":"is","s":153.831,"e":154.071},{"text":"honest.","s":154.151,"e":155.111},{"text":"When","s":155.151,"e":155.191},{"text":"a","s":155.271,"e":155.351},{"text":"decision","s":155.551,"e":155.751},{"text":"model","s":155.831,"e":156.071},{"text":"says","s":156.151,"e":156.311},{"text":"point","s":156.471,"e":156.791},{"text":"nine,","s":156.991,"e":157.511},{"text":"it","s":157.591,"e":157.751},{"text":"should","s":157.831,"e":157.911},{"text":"be","s":157.991,"e":158.151},{"text":"right","s":158.271,"e":158.391},{"text":"about","s":158.471,"e":158.871},{"text":"nine","s":158.951,"e":159.031},{"text":"times","s":159.191,"e":159.431},{"text":"out","s":159.471,"e":159.511},{"text":"of","s":159.591,"e":159.671},{"text":"ten.","s":159.831,"e":160.471},{"text":"Whether","s":160.591,"e":160.711},{"text":"that","s":160.791,"e":160.871},{"text":"holds","s":160.951,"e":161.111},{"text":"on","s":161.191,"e":161.351},{"text":"your","s":161.431,"e":161.591},{"text":"data","s":161.671,"e":162.071},{"text":"is","s":162.151,"e":162.231},{"text":"something","s":162.391,"e":162.551},{"text":"you'll","s":162.591,"e":162.711},{"text":"learn","s":162.791,"e":162.871},{"text":"to","s":162.951,"e":163.031},{"text":"measure","s":163.191,"e":163.351},{"text":"in","s":163.431,"e":163.511},{"text":"module","s":163.671,"e":163.831},{"text":"three.","s":163.991,"e":164.551}]},{"template":"quiz","chapter":"Quick check","narration":"Quick check. Why can a decision model never return an answer outside your list?","params":{"q":"Why can't a decision model return an answer **outside your list**?","options":["It was fine-tuned on your categories","It only scores the options you supplied and returns a distribution over them","It regenerates until the output validates","A regex filters its output"],"answer":1,"explain":"The model never writes text. It assigns probability only to the answers you defined, so anything else simply doesn't exist for it. It can still pick the wrong one."},"start":165.147,"dur":10.829,"words":[{"text":"Quick","s":165.447,"e":165.687},{"text":"check.","s":165.767,"e":166.327},{"text":"Why","s":166.567,"e":166.727},{"text":"can","s":166.807,"e":166.887},{"text":"a","s":166.967,"e":167.007},{"text":"decision","s":167.047,"e":167.447},{"text":"model","s":167.527,"e":167.847},{"text":"never","s":168.087,"e":168.327},{"text":"return","s":168.407,"e":168.727},{"text":"an","s":168.807,"e":168.887},{"text":"answer","s":169.087,"e":169.287},{"text":"outside","s":169.367,"e":169.847},{"text":"your","s":169.927,"e":170.007},{"text":"list?","s":170.087,"e":170.727}]},{"template":"exercise","chapter":"Try it","narration":"Try it. Ask your coding agent to build the tiny decider from this lesson with any small open model, and compare it with a hosted decision model on twenty of your own examples.","params":{"title":"Build a tiny decider","intro":"Paste this into your coding agent (needs Python):","prompt":"Build tiny_decider.py: given a text and a list of options, put the\noptions in a prompt with letters A, B, C..., run a small open GGUF model\n(e.g. Qwen3-0.6B via llama-cpp-python) once, and return a probability\nfor each option from the next-token logits (softmax over the letters).\nThen run it on 20 real examples from this project and print the\nprobabilities next to the labels we expect."},"start":175.976,"dur":11.556,"words":[{"text":"Try","s":176.276,"e":176.516},{"text":"it.","s":176.596,"e":176.836},{"text":"Ask","s":177.156,"e":177.396},{"text":"your","s":177.476,"e":177.556},{"text":"coding","s":177.636,"e":177.876},{"text":"agent","s":177.956,"e":178.276},{"text":"to","s":178.356,"e":178.436},{"text":"build","s":178.516,"e":178.676},{"text":"the","s":178.756,"e":178.836},{"text":"tiny","s":178.996,"e":179.156},{"text":"decider","s":179.276,"e":179.716},{"text":"from","s":179.796,"e":179.876},{"text":"this","s":179.956,"e":180.116},{"text":"lesson","s":180.196,"e":180.596},{"text":"with","s":180.756,"e":180.916},{"text":"any","s":181.036,"e":181.156},{"text":"small","s":181.316,"e":181.556},{"text":"open","s":181.716,"e":181.876},{"text":"model,","s":181.956,"e":182.596},{"text":"and","s":182.676,"e":182.756},{"text":"compare","s":182.836,"e":183.156},{"text":"it","s":183.196,"e":183.236},{"text":"with","s":183.316,"e":183.396},{"text":"a","s":183.476,"e":183.556},{"text":"hosted","s":183.676,"e":183.956},{"text":"decision","s":184.156,"e":184.356},{"text":"model","s":184.436,"e":184.756},{"text":"on","s":184.876,"e":184.996},{"text":"twenty","s":185.196,"e":185.396},{"text":"of","s":185.476,"e":185.556},{"text":"your","s":185.636,"e":185.716},{"text":"own","s":185.796,"e":185.876},{"text":"examples.","s":185.956,"e":186.916}]},{"template":"recap","chapter":"Recap","narration":"So: decision models read once and score every allowed answer in one pass. The trick is reading probabilities instead of writing text. Real models add a trained head, shared state, and calibration training. And typed doesn't mean correct. Next: the four decision models you can use today.","params":{"items":["One pass that scores every allowed answer, no token-by-token writing.","Read the probability of each option instead of generating text.","Real models add a readout head, shared state, calibration training.","Typed isn't correct: it can still be confidently wrong."],"next":"Jev, Decisions, Clef, Strands: the landscape","next_at":"Next"},"start":187.532,"dur":22.249,"words":[{"text":"So:","s":187.832,"e":188.552},{"text":"decision","s":188.632,"e":189.032},{"text":"models","s":189.112,"e":189.432},{"text":"read","s":189.512,"e":189.752},{"text":"once","s":189.832,"e":190.232},{"text":"and","s":190.312,"e":190.392},{"text":"score","s":190.472,"e":190.872},{"text":"every","s":190.952,"e":191.192},{"text":"allowed","s":191.392,"e":191.592},{"text":"answer","s":191.792,"e":191.992},{"text":"in","s":192.072,"e":192.232},{"text":"one","s":192.312,"e":192.472},{"text":"pass.","s":192.552,"e":193.512},{"text":"The","s":193.592,"e":193.672},{"text":"trick","s":193.712,"e":194.072},{"text":"is","s":194.152,"e":194.312},{"text":"reading","s":194.472,"e":194.632},{"text":"probabilities","s":194.752,"e":195.512},{"text":"instead","s":195.592,"e":195.912},{"text":"of","s":195.952,"e":195.992},{"text":"writing","s":196.072,"e":196.392},{"text":"text.","s":196.632,"e":197.432},{"text":"Real","s":197.552,"e":197.672},{"text":"models","s":197.752,"e":198.152},{"text":"add","s":198.232,"e":198.312},{"text":"a","s":198.392,"e":198.472},{"text":"trained","s":198.632,"e":198.792},{"text":"head,","s":198.872,"e":199.352},{"text":"shared","s":199.432,"e":199.672},{"text":"state,","s":199.752,"e":200.152},{"text":"and","s":200.232,"e":200.312},{"text":"calibration","s":200.392,"e":200.952},{"text":"training.","s":201.032,"e":201.912},{"text":"And","s":202.032,"e":202.152},{"text":"typed","s":202.312,"e":202.872},{"text":"doesn't","s":202.952,"e":203.272},{"text":"mean","s":203.352,"e":203.512},{"text":"correct.","s":203.592,"e":204.792},{"text":"Next:","s":204.872,"e":205.672},{"text":"the","s":205.752,"e":205.832},{"text":"four","s":205.912,"e":206.072},{"text":"decision","s":206.272,"e":206.472},{"text":"models","s":206.552,"e":206.872},{"text":"you","s":206.952,"e":207.032},{"text":"can","s":207.112,"e":207.192},{"text":"use","s":207.272,"e":207.432},{"text":"today.","s":207.512,"e":208.232}]}],"total":210.181,"audio":"/audio/03-how-it-works.mp3"}