{"id":"21-responsible","number":21,"slug":"responsible-decisions","title":"Decide responsibly","short":"Decide responsibly","module":"Shipping it","summary":"A decision model makes the same call the same way, millions of times. That removes noise, and it makes any bias systematic. Here is how to audit decisions about people (including the name-swap test we ran), where the reasons come from when the model gives none, and what a responsible deployment logs, samples and leaves to a person.","voice":"Jessica","scenes":[{"template":"title","chapter":"Intro","narration":"A decision model will make the same call the same way, a million times a day. That's its strength. It's also why some of those calls deserve extra care.","params":{"kicker":"Lesson 21","module":"Module 6 · Shipping it","title":"Decide *responsibly*","subtitle":"Consistent is not the same as fair.","cast":["you",{"who":"snap","mood":"unsure"}]},"start":0.5,"dur":11.388,"words":[{"text":"A","s":0.8,"e":0.88},{"text":"decision","s":1.12,"e":1.36},{"text":"model","s":1.44,"e":1.76},{"text":"will","s":1.84,"e":1.92},{"text":"make","s":2,"e":2.16},{"text":"the","s":2.24,"e":2.32},{"text":"same","s":2.4,"e":2.64},{"text":"call","s":2.72,"e":3.12},{"text":"the","s":3.2,"e":3.36},{"text":"same","s":3.44,"e":3.68},{"text":"way,","s":3.76,"e":4.24},{"text":"a","s":4.36,"e":4.48},{"text":"million","s":4.56,"e":4.88},{"text":"times","s":4.96,"e":5.2},{"text":"a","s":5.24,"e":5.28},{"text":"day.","s":5.4,"e":6},{"text":"That's","s":6.12,"e":6.32},{"text":"its","s":6.44,"e":6.56},{"text":"strength.","s":6.84,"e":7.52},{"text":"It's","s":7.6,"e":7.76},{"text":"also","s":7.84,"e":8.08},{"text":"why","s":8.16,"e":8.32},{"text":"some","s":8.48,"e":8.64},{"text":"of","s":8.72,"e":8.8},{"text":"those","s":8.92,"e":9.04},{"text":"calls","s":9.12,"e":9.44},{"text":"deserve","s":9.52,"e":9.92},{"text":"extra","s":10.12,"e":10.32},{"text":"care.","s":10.52,"e":11.04}]},{"template":"compare","chapter":"Noise and bias","narration":"Olivier Sibony and Daniel Kahneman separate two kinds of error. Noise is scatter: in one audit at an investment firm, two analysts valuing the same security differed by forty four percent on average. Bias is a shared, predictable error. Decision models have very little noise: in our tests, repeated runs moved by four hundredths at most. So whatever errors they have are systematic. The same mistake, for the same kind of case, every time.","params":{"left":{"title":"People: noise","who":"you","tone":"s2","items":[{"text":"Scatter: same case, different answers","at":"Noise is scatter"},{"text":"Two analysts, same security: 44% apart"},{"text":"Fix: average independent judgments"}]},"right":{"title":"Decision models: bias","who":"snap","tone":"bad","items":[{"text":"Repeat runs moved ≤ 0.04","at":"very little noise"},{"text":"Errors are systematic","at":"systematic"},{"text":"Same mistake, every time, at scale"}]},"vs":"vs"},"start":11.888,"dur":30.912,"words":[{"text":"Olivier","s":12.188,"e":12.668},{"text":"Sibony","s":12.708,"e":13.228},{"text":"and","s":13.308,"e":13.468},{"text":"Daniel","s":13.548,"e":13.788},{"text":"Kahneman","s":13.868,"e":14.428},{"text":"separate","s":14.508,"e":14.988},{"text":"two","s":15.068,"e":15.228},{"text":"kinds","s":15.308,"e":15.548},{"text":"of","s":15.628,"e":15.708},{"text":"error.","s":15.788,"e":16.388},{"text":"Noise","s":16.668,"e":17.148},{"text":"is","s":17.228,"e":17.388},{"text":"scatter:","s":17.668,"e":18.348},{"text":"in","s":18.428,"e":18.588},{"text":"one","s":18.708,"e":18.828},{"text":"audit","s":18.908,"e":19.228},{"text":"at","s":19.308,"e":19.388},{"text":"an","s":19.428,"e":19.468},{"text":"investment","s":19.708,"e":19.948},{"text":"firm,","s":20.028,"e":20.588},{"text":"two","s":20.708,"e":20.828},{"text":"analysts","s":20.908,"e":21.388},{"text":"valuing","s":21.468,"e":21.868},{"text":"the","s":21.948,"e":22.028},{"text":"same","s":22.108,"e":22.268},{"text":"security","s":22.348,"e":22.908},{"text":"differed","s":22.988,"e":23.308},{"text":"by","s":23.428,"e":23.548},{"text":"forty","s":23.628,"e":23.948},{"text":"four","s":24.028,"e":24.108},{"text":"percent","s":24.268,"e":24.748},{"text":"on","s":24.828,"e":24.908},{"text":"average.","s":25.148,"e":25.948},{"text":"Bias","s":26.028,"e":26.508},{"text":"is","s":26.588,"e":26.668},{"text":"a","s":26.748,"e":26.908},{"text":"shared,","s":27.108,"e":27.468},{"text":"predictable","s":27.668,"e":28.108},{"text":"error.","s":28.188,"e":28.988},{"text":"Decision","s":29.188,"e":29.388},{"text":"models","s":29.468,"e":29.788},{"text":"have","s":29.868,"e":30.108},{"text":"very","s":30.268,"e":30.428},{"text":"little","s":30.548,"e":30.668},{"text":"noise:","s":30.908,"e":31.468},{"text":"in","s":31.548,"e":31.628},{"text":"our","s":31.708,"e":31.788},{"text":"tests,","s":31.988,"e":32.348},{"text":"repeated","s":32.428,"e":32.828},{"text":"runs","s":32.908,"e":33.228},{"text":"moved","s":33.308,"e":33.548},{"text":"by","s":33.668,"e":33.788},{"text":"four","s":33.868,"e":34.108},{"text":"hundredths","s":34.188,"e":34.668},{"text":"at","s":34.748,"e":34.908},{"text":"most.","s":35.348,"e":35.868},{"text":"So","s":35.948,"e":36.028},{"text":"whatever","s":36.108,"e":36.428},{"text":"errors","s":36.588,"e":36.748},{"text":"they","s":36.828,"e":36.908},{"text":"have","s":36.988,"e":37.308},{"text":"are","s":37.388,"e":37.548},{"text":"systematic.","s":37.708,"e":38.588},{"text":"The","s":38.708,"e":38.828},{"text":"same","s":38.908,"e":38.988},{"text":"mistake,","s":39.068,"e":39.708},{"text":"for","s":39.748,"e":39.788},{"text":"the","s":39.868,"e":40.028},{"text":"same","s":40.148,"e":40.268},{"text":"kind","s":40.348,"e":40.428},{"text":"of","s":40.508,"e":40.588},{"text":"case,","s":40.668,"e":41.228},{"text":"every","s":41.308,"e":41.548},{"text":"time.","s":41.628,"e":42.188}]},{"template":"quote","chapter":"Decisions about people","narration":"That matters most when the decision is about a person. Simon Willison wrote on launch day: I really hope nobody uses Jev to rank job applicants. That floating point number could conceal all manner of unseen bias baked into the models.","params":{"text":"I really hope nobody uses Jev to rank job applicants: that floating point number could conceal *all manner of unseen bias*.","who":"Simon Willison","role":"simonwillison.net","src":"Jev introduces a new shape of LLM"},"start":42.8,"dur":16.258,"words":[{"text":"That","s":43.1,"e":43.34},{"text":"matters","s":43.38,"e":43.74},{"text":"most","s":43.82,"e":44.14},{"text":"when","s":44.22,"e":44.3},{"text":"the","s":44.34,"e":44.38},{"text":"decision","s":44.62,"e":44.86},{"text":"is","s":44.94,"e":45.02},{"text":"about","s":45.14,"e":45.26},{"text":"a","s":45.34,"e":45.42},{"text":"person.","s":45.7,"e":46.06},{"text":"Simon","s":46.62,"e":46.94},{"text":"Willison","s":47.02,"e":47.42},{"text":"wrote","s":47.46,"e":47.66},{"text":"on","s":47.74,"e":47.82},{"text":"launch","s":47.98,"e":48.14},{"text":"day:","s":48.22,"e":48.7},{"text":"I","s":48.94,"e":49.1},{"text":"really","s":49.26,"e":49.58},{"text":"hope","s":49.66,"e":49.9},{"text":"nobody","s":50.06,"e":50.46},{"text":"uses","s":50.54,"e":50.86},{"text":"Jev","s":50.94,"e":51.18},{"text":"to","s":51.26,"e":51.34},{"text":"rank","s":51.42,"e":51.66},{"text":"job","s":51.74,"e":51.9},{"text":"applicants.","s":51.98,"e":52.82},{"text":"That","s":53.02,"e":53.18},{"text":"floating","s":53.26,"e":53.58},{"text":"point","s":53.66,"e":53.82},{"text":"number","s":53.9,"e":54.14},{"text":"could","s":54.22,"e":54.38},{"text":"conceal","s":54.5,"e":54.94},{"text":"all","s":55.06,"e":55.18},{"text":"manner","s":55.34,"e":55.66},{"text":"of","s":55.74,"e":55.82},{"text":"unseen","s":55.9,"e":56.38},{"text":"bias","s":56.46,"e":56.86},{"text":"baked","s":56.94,"e":57.18},{"text":"into","s":57.26,"e":57.42},{"text":"the","s":57.46,"e":57.5},{"text":"models.","s":57.58,"e":58.46}]},{"template":"stat","chapter":"Bias in the criteria","narration":"And bias doesn't only live inside models. It can live in your criteria. Remember the Apgar score from lesson twelve? One of its five items is skin color. A study of more than nine million American newborns found that three point three percent of white babies scored a perfect ten, and only one point four percent of Black babies, which can mean unneeded treatment for healthy infants.","params":{"title":"Apgar scores, 9 million US newborns (2016–2019)","items":[{"value":3.3,"decimals":1,"suffix":"%","label":"of white newborns scored a perfect 10","tone":"s2","at":"three point three"},{"value":1.4,"decimals":1,"suffix":"%","label":"of Black newborns","tone":"bad","at":"one point four","hero":true}],"src":"Grünebaum et al., via Wikipedia, Apgar score. The appearance item scores skin color.","punch":false},"start":59.058,"dur":24.669,"words":[{"text":"And","s":59.358,"e":59.518},{"text":"bias","s":59.598,"e":59.998},{"text":"doesn't","s":60.118,"e":60.398},{"text":"only","s":60.478,"e":60.718},{"text":"live","s":60.798,"e":60.958},{"text":"inside","s":61.118,"e":61.278},{"text":"models.","s":61.358,"e":62.158},{"text":"It","s":62.198,"e":62.238},{"text":"can","s":62.318,"e":62.398},{"text":"live","s":62.478,"e":62.558},{"text":"in","s":62.638,"e":62.678},{"text":"your","s":62.718,"e":62.798},{"text":"criteria.","s":62.878,"e":63.998},{"text":"Remember","s":64.158,"e":64.318},{"text":"the","s":64.398,"e":64.478},{"text":"Apgar","s":64.558,"e":64.958},{"text":"score","s":65.038,"e":65.278},{"text":"from","s":65.358,"e":65.438},{"text":"lesson","s":65.518,"e":65.758},{"text":"twelve?","s":65.838,"e":66.718},{"text":"One","s":66.758,"e":66.798},{"text":"of","s":66.878,"e":66.918},{"text":"its","s":66.958,"e":67.198},{"text":"five","s":67.358,"e":67.518},{"text":"items","s":67.598,"e":67.998},{"text":"is","s":68.158,"e":68.318},{"text":"skin","s":68.478,"e":68.638},{"text":"color.","s":68.718,"e":69.518},{"text":"A","s":69.598,"e":69.678},{"text":"study","s":69.838,"e":69.998},{"text":"of","s":70.038,"e":70.078},{"text":"more","s":70.158,"e":70.238},{"text":"than","s":70.318,"e":70.558},{"text":"nine","s":70.638,"e":70.878},{"text":"million","s":71.118,"e":71.438},{"text":"American","s":71.678,"e":71.918},{"text":"newborns","s":71.998,"e":72.558},{"text":"found","s":72.638,"e":72.878},{"text":"that","s":72.998,"e":73.118},{"text":"three","s":73.198,"e":73.358},{"text":"point","s":73.518,"e":73.758},{"text":"three","s":73.838,"e":73.918},{"text":"percent","s":74.078,"e":74.398},{"text":"of","s":74.478,"e":74.558},{"text":"white","s":74.638,"e":74.798},{"text":"babies","s":74.878,"e":75.278},{"text":"scored","s":75.398,"e":75.558},{"text":"a","s":75.598,"e":75.678},{"text":"perfect","s":75.758,"e":76.158},{"text":"ten,","s":76.318,"e":76.878},{"text":"and","s":76.958,"e":77.038},{"text":"only","s":77.118,"e":77.438},{"text":"one","s":77.518,"e":77.598},{"text":"point","s":77.758,"e":78.078},{"text":"four","s":78.158,"e":78.238},{"text":"percent","s":78.318,"e":78.638},{"text":"of","s":78.718,"e":78.798},{"text":"Black","s":78.918,"e":79.038},{"text":"babies,","s":79.118,"e":79.998},{"text":"which","s":80.078,"e":80.158},{"text":"can","s":80.238,"e":80.318},{"text":"mean","s":80.398,"e":80.638},{"text":"unneeded","s":80.718,"e":81.198},{"text":"treatment","s":81.238,"e":81.598},{"text":"for","s":81.678,"e":81.758},{"text":"healthy","s":81.838,"e":82.158},{"text":"infants.","s":82.238,"e":83.118}]},{"template":"chart","chapter":"Test it","narration":"So test it, with counterfactuals. Change only what must not matter, and see if the answer moves. We wrote twelve candidate summaries for a backend job and sent each one five times: with no name, and as Emily Walsh, Greg Baker, Lakisha Washington and Jamal Jones. The average chance of an interview barely moved. No single answer moved by more than point oh six, and no decision flipped.","params":{"title":"Should we interview? Same 12 candidates, only the name changes","kind":"bar","ymax":0.7,"data":[{"x":"no name","y":0.55,"label":"0.55","tone":"s2","at":"with no name"},{"x":"Emily Walsh","y":0.536,"label":"0.54","at":"Emily Walsh"},{"x":"Greg Baker","y":0.533,"label":"0.53"},{"x":"Lakisha Washington","y":0.535,"label":"0.54"},{"x":"Jamal Jones","y":0.534,"label":"0.53","at":"Jamal Jones"}],"src":"jev-1.13.0, 60 requests, 2026-10-15. Largest change for any one candidate: 0.06. Decisions flipped: 0 of 12."},"start":83.727,"dur":27.464,"words":[{"text":"So","s":84.027,"e":84.187},{"text":"test","s":84.267,"e":84.587},{"text":"it,","s":84.707,"e":85.067},{"text":"with","s":85.147,"e":85.227},{"text":"counterfactuals.","s":85.387,"e":86.427},{"text":"Change","s":86.667,"e":87.067},{"text":"only","s":87.147,"e":87.467},{"text":"what","s":87.547,"e":87.707},{"text":"must","s":87.867,"e":88.107},{"text":"not","s":88.187,"e":88.347},{"text":"matter,","s":88.427,"e":89.067},{"text":"and","s":89.147,"e":89.227},{"text":"see","s":89.307,"e":89.387},{"text":"if","s":89.467,"e":89.547},{"text":"the","s":89.587,"e":89.627},{"text":"answer","s":89.707,"e":89.947},{"text":"moves.","s":90.027,"e":90.787},{"text":"We","s":91.067,"e":91.227},{"text":"wrote","s":91.307,"e":91.467},{"text":"twelve","s":91.547,"e":91.867},{"text":"candidate","s":91.947,"e":92.347},{"text":"summaries","s":92.427,"e":92.827},{"text":"for","s":92.907,"e":92.947},{"text":"a","s":92.987,"e":93.067},{"text":"backend","s":93.147,"e":93.467},{"text":"job","s":93.547,"e":93.867},{"text":"and","s":93.947,"e":94.107},{"text":"sent","s":94.267,"e":94.427},{"text":"each","s":94.547,"e":94.667},{"text":"one","s":94.747,"e":95.067},{"text":"five","s":95.147,"e":95.387},{"text":"times:","s":95.467,"e":96.267},{"text":"with","s":96.347,"e":96.427},{"text":"no","s":96.507,"e":96.667},{"text":"name,","s":96.747,"e":97.307},{"text":"and","s":97.467,"e":97.627},{"text":"as","s":97.787,"e":98.027},{"text":"Emily","s":98.107,"e":98.427},{"text":"Walsh,","s":98.507,"e":99.067},{"text":"Greg","s":99.187,"e":99.307},{"text":"Baker,","s":99.387,"e":100.027},{"text":"Lakisha","s":100.107,"e":100.507},{"text":"Washington","s":100.587,"e":101.147},{"text":"and","s":101.227,"e":101.307},{"text":"Jamal","s":101.387,"e":101.707},{"text":"Jones.","s":101.787,"e":102.507},{"text":"The","s":102.747,"e":102.827},{"text":"average","s":102.907,"e":103.227},{"text":"chance","s":103.307,"e":103.467},{"text":"of","s":103.547,"e":103.627},{"text":"an","s":103.667,"e":103.707},{"text":"interview","s":103.787,"e":104.267},{"text":"barely","s":104.507,"e":104.907},{"text":"moved.","s":104.987,"e":105.787},{"text":"No","s":105.867,"e":106.027},{"text":"single","s":106.187,"e":106.427},{"text":"answer","s":106.507,"e":106.827},{"text":"moved","s":106.907,"e":107.147},{"text":"by","s":107.227,"e":107.307},{"text":"more","s":107.387,"e":107.467},{"text":"than","s":107.547,"e":107.707},{"text":"point","s":107.827,"e":107.947},{"text":"oh","s":108.107,"e":108.267},{"text":"six,","s":108.347,"e":108.907},{"text":"and","s":108.987,"e":109.067},{"text":"no","s":109.147,"e":109.307},{"text":"decision","s":109.387,"e":109.867},{"text":"flipped.","s":109.907,"e":110.587}]},{"template":"statement","chapter":"Test it","narration":"That's reassuring, and it proves very little. It's one test, of one attribute, on twelve made-up people. Bias can travel through proxies a name test never touches: schools, addresses, gaps in a career, the way someone writes. As Willison's advice goes, a model like this needs more evals, not fewer. Test by subgroup on real, labeled cases too.","params":{"eyebrow":"One passed test is not fairness","text":"A model like this needs *more evals, not fewer*.","sub":"Proxies (schools, addresses, career gaps, writing style) can carry bias a name swap never sees.","sub_at":"proxies","art":"scale","tone":"s2","punch":"proves very little"},"start":111.191,"dur":26.498,"words":[{"text":"That's","s":111.491,"e":111.771},{"text":"reassuring,","s":111.811,"e":112.851},{"text":"and","s":113.011,"e":113.171},{"text":"it","s":113.331,"e":113.491},{"text":"proves","s":113.571,"e":113.891},{"text":"very","s":114.051,"e":114.291},{"text":"little.","s":114.371,"e":115.011},{"text":"It's","s":115.251,"e":115.491},{"text":"one","s":115.571,"e":115.731},{"text":"test,","s":115.811,"e":116.451},{"text":"of","s":116.531,"e":116.611},{"text":"one","s":116.771,"e":116.931},{"text":"attribute,","s":117.011,"e":117.571},{"text":"on","s":117.651,"e":117.811},{"text":"twelve","s":117.891,"e":118.291},{"text":"made-up","s":118.531,"e":118.851},{"text":"people.","s":118.931,"e":119.411},{"text":"Bias","s":119.971,"e":120.291},{"text":"can","s":120.371,"e":120.451},{"text":"travel","s":120.531,"e":120.771},{"text":"through","s":120.891,"e":121.011},{"text":"proxies","s":121.171,"e":121.491},{"text":"a","s":121.571,"e":121.651},{"text":"name","s":121.731,"e":121.891},{"text":"test","s":121.971,"e":122.211},{"text":"never","s":122.451,"e":122.691},{"text":"touches:","s":122.771,"e":123.731},{"text":"schools,","s":123.811,"e":124.611},{"text":"addresses,","s":124.971,"e":125.491},{"text":"gaps","s":125.571,"e":125.811},{"text":"in","s":125.851,"e":125.891},{"text":"a","s":125.931,"e":125.971},{"text":"career,","s":126.051,"e":126.771},{"text":"the","s":126.851,"e":126.891},{"text":"way","s":126.931,"e":127.011},{"text":"someone","s":127.091,"e":127.331},{"text":"writes.","s":127.411,"e":127.971},{"text":"As","s":128.531,"e":128.611},{"text":"Willison's","s":128.691,"e":129.091},{"text":"advice","s":129.171,"e":129.491},{"text":"goes,","s":129.571,"e":130.131},{"text":"a","s":130.171,"e":130.211},{"text":"model","s":130.291,"e":130.531},{"text":"like","s":130.651,"e":130.771},{"text":"this","s":130.891,"e":131.011},{"text":"needs","s":131.091,"e":131.331},{"text":"more","s":131.491,"e":131.651},{"text":"evals,","s":131.731,"e":132.451},{"text":"not","s":132.531,"e":132.691},{"text":"fewer.","s":132.771,"e":133.251},{"text":"Test","s":133.731,"e":133.971},{"text":"by","s":134.051,"e":134.131},{"text":"subgroup","s":134.211,"e":134.691},{"text":"on","s":134.771,"e":134.851},{"text":"real,","s":135.011,"e":135.491},{"text":"labeled","s":135.571,"e":135.971},{"text":"cases","s":136.051,"e":136.451},{"text":"too.","s":136.531,"e":137.091}]},{"template":"quote","chapter":"Where the reasons come from","narration":"People also ask for reasons. Kahneman pointed out that algorithms clearly better than judges at predicting parole violations still meet resistance, because they can't explain their reasoning. A decision model gives you numbers, not reasons. But your design has them. Unsiloed put it simply: the explanations come from our application. Jev returns the typed answers.","params":{"text":"The explanations and routing labels come from our application; *Jev returns the typed answers*, not those explanations.","who":"Unsiloed","role":"document review with Jev","src":"unsiloed.ai"},"start":137.689,"dur":25.479,"words":[{"text":"People","s":137.989,"e":138.309},{"text":"also","s":138.389,"e":138.709},{"text":"ask","s":138.789,"e":139.029},{"text":"for","s":139.109,"e":139.269},{"text":"reasons.","s":139.349,"e":140.069},{"text":"Kahneman","s":140.469,"e":140.869},{"text":"pointed","s":140.949,"e":141.189},{"text":"out","s":141.269,"e":141.429},{"text":"that","s":141.549,"e":141.669},{"text":"algorithms","s":141.749,"e":142.309},{"text":"clearly","s":142.469,"e":142.949},{"text":"better","s":143.109,"e":143.349},{"text":"than","s":143.429,"e":143.509},{"text":"judges","s":143.589,"e":143.989},{"text":"at","s":144.029,"e":144.069},{"text":"predicting","s":144.149,"e":144.549},{"text":"parole","s":144.629,"e":144.949},{"text":"violations","s":145.029,"e":145.749},{"text":"still","s":145.989,"e":146.309},{"text":"meet","s":146.389,"e":146.549},{"text":"resistance,","s":146.629,"e":147.589},{"text":"because","s":147.669,"e":147.829},{"text":"they","s":147.909,"e":147.989},{"text":"can't","s":148.069,"e":148.229},{"text":"explain","s":148.309,"e":148.789},{"text":"their","s":148.869,"e":148.949},{"text":"reasoning.","s":149.029,"e":149.669},{"text":"A","s":150.149,"e":150.229},{"text":"decision","s":150.429,"e":150.629},{"text":"model","s":150.709,"e":150.949},{"text":"gives","s":151.029,"e":151.189},{"text":"you","s":151.229,"e":151.269},{"text":"numbers,","s":151.429,"e":152.149},{"text":"not","s":152.229,"e":152.389},{"text":"reasons.","s":152.469,"e":153.309},{"text":"But","s":153.509,"e":153.589},{"text":"your","s":153.669,"e":153.749},{"text":"design","s":153.829,"e":154.309},{"text":"has","s":154.469,"e":154.709},{"text":"them.","s":154.789,"e":155.109},{"text":"Unsiloed","s":155.509,"e":156.149},{"text":"put","s":156.229,"e":156.309},{"text":"it","s":156.389,"e":156.469},{"text":"simply:","s":156.589,"e":157.349},{"text":"the","s":157.429,"e":157.509},{"text":"explanations","s":157.589,"e":158.389},{"text":"come","s":158.469,"e":158.629},{"text":"from","s":158.709,"e":158.789},{"text":"our","s":158.869,"e":158.949},{"text":"application.","s":159.029,"e":159.829},{"text":"Jev","s":160.389,"e":160.629},{"text":"returns","s":160.709,"e":161.189},{"text":"the","s":161.229,"e":161.269},{"text":"typed","s":161.349,"e":161.589},{"text":"answers.","s":161.669,"e":162.549}]},{"template":"code","chapter":"Where the reasons come from","narration":"So log the reasons you designed. Every decision record holds the model version, every narrow answer, the thresholds and the action. That's an audit trail. And when someone asks why, turn that log into an explanation, with a template or an LLM. The decision stays in your code; only the wording is generated.","params":{"file":"decision_log.py","lang":"py","lines":["log.write({","  \"case\": ticket.id, \"model\": \"jev-1.13.0\", \"at\": now(),","  \"answers\": {\"duplicate_charge\": 0.98, \"policy_allows\": 0.99},","  \"rule\": \"refund if both > 0.80\", \"action\": \"auto_refund\",","  \"review_sample\": random() < 0.02,             # a person checks 2%","})","explain = explain_from(log)   # a template or an LLM: wording only"],"hl":[{"from":2,"to":4,"label":"the audit trail","tone":"agent","at":"audit trail"},{"from":7,"to":7,"label":"explanation from the log","tone":"good","at":"turn that log"}]},"start":163.168,"dur":21.143,"words":[{"text":"So","s":163.468,"e":163.708},{"text":"log","s":163.868,"e":164.188},{"text":"the","s":164.228,"e":164.268},{"text":"reasons","s":164.348,"e":164.788},{"text":"you","s":164.828,"e":164.908},{"text":"designed.","s":165.028,"e":165.548},{"text":"Every","s":165.948,"e":166.188},{"text":"decision","s":166.388,"e":166.588},{"text":"record","s":166.668,"e":166.988},{"text":"holds","s":167.068,"e":167.308},{"text":"the","s":167.388,"e":167.468},{"text":"model","s":167.548,"e":167.788},{"text":"version,","s":167.868,"e":168.508},{"text":"every","s":168.588,"e":168.828},{"text":"narrow","s":168.908,"e":169.228},{"text":"answer,","s":169.428,"e":169.868},{"text":"the","s":169.908,"e":169.948},{"text":"thresholds","s":170.028,"e":170.668},{"text":"and","s":170.828,"e":170.988},{"text":"the","s":171.028,"e":171.068},{"text":"action.","s":171.148,"e":171.708},{"text":"That's","s":172.108,"e":172.428},{"text":"an","s":172.508,"e":172.588},{"text":"audit","s":172.668,"e":172.908},{"text":"trail.","s":172.988,"e":173.548},{"text":"And","s":173.788,"e":173.868},{"text":"when","s":173.908,"e":173.948},{"text":"someone","s":174.028,"e":174.268},{"text":"asks","s":174.348,"e":174.668},{"text":"why,","s":174.748,"e":175.468},{"text":"turn","s":175.588,"e":175.708},{"text":"that","s":175.788,"e":175.868},{"text":"log","s":175.948,"e":176.188},{"text":"into","s":176.268,"e":176.428},{"text":"an","s":176.508,"e":176.588},{"text":"explanation,","s":176.708,"e":177.628},{"text":"with","s":177.668,"e":177.708},{"text":"a","s":177.788,"e":177.828},{"text":"template","s":177.868,"e":178.348},{"text":"or","s":178.428,"e":178.588},{"text":"an","s":178.668,"e":178.748},{"text":"LLM.","s":178.828,"e":179.388},{"text":"The","s":179.868,"e":179.908},{"text":"decision","s":179.948,"e":180.348},{"text":"stays","s":180.508,"e":180.828},{"text":"in","s":180.868,"e":180.908},{"text":"your","s":180.988,"e":181.068},{"text":"code;","s":181.148,"e":181.788},{"text":"only","s":181.908,"e":182.028},{"text":"the","s":182.068,"e":182.108},{"text":"wording","s":182.268,"e":182.588},{"text":"is","s":182.668,"e":182.748},{"text":"generated.","s":182.828,"e":183.708}]},{"template":"list","chapter":"Keep a person accountable","narration":"And keep a person accountable. Sample automated decisions for review, every week, because a threshold that worked at launch can drift. Give people a way to appeal to a human. Audit outcomes by subgroup. And for decisions about jobs, money, housing or health, let the model assist a person rather than decide alone, and check the rules where you operate.","params":{"title":"A responsible deployment","items":[{"text":"**Sample** automated decisions for human review, weekly","at":"Sample automated"},{"text":"Offer an **appeal** to a person","at":"appeal"},{"text":"**Audit** outcomes by subgroup, and with counterfactuals","at":"Audit outcomes"},{"text":"Jobs, money, housing, health: **assist**, don't decide alone","at":"assist a person"}]},"start":184.311,"dur":24.434,"words":[{"text":"And","s":184.611,"e":184.851},{"text":"keep","s":184.931,"e":185.091},{"text":"a","s":185.171,"e":185.251},{"text":"person","s":185.411,"e":185.571},{"text":"accountable.","s":185.651,"e":186.771},{"text":"Sample","s":186.851,"e":187.251},{"text":"automated","s":187.331,"e":187.731},{"text":"decisions","s":187.811,"e":188.291},{"text":"for","s":188.371,"e":188.451},{"text":"review,","s":188.531,"e":189.331},{"text":"every","s":189.491,"e":189.651},{"text":"week,","s":189.811,"e":190.451},{"text":"because","s":190.611,"e":190.771},{"text":"a","s":190.851,"e":190.931},{"text":"threshold","s":191.131,"e":191.331},{"text":"that","s":191.411,"e":191.491},{"text":"worked","s":191.571,"e":191.731},{"text":"at","s":191.811,"e":191.891},{"text":"launch","s":191.971,"e":192.451},{"text":"can","s":192.531,"e":192.691},{"text":"drift.","s":192.771,"e":193.651},{"text":"Give","s":193.731,"e":193.891},{"text":"people","s":193.971,"e":194.211},{"text":"a","s":194.251,"e":194.291},{"text":"way","s":194.371,"e":194.531},{"text":"to","s":194.611,"e":194.691},{"text":"appeal","s":194.851,"e":195.251},{"text":"to","s":195.291,"e":195.331},{"text":"a","s":195.411,"e":195.571},{"text":"human.","s":195.811,"e":196.451},{"text":"Audit","s":196.611,"e":196.771},{"text":"outcomes","s":196.851,"e":197.331},{"text":"by","s":197.411,"e":197.491},{"text":"subgroup.","s":197.651,"e":198.611},{"text":"And","s":198.691,"e":198.771},{"text":"for","s":198.811,"e":198.851},{"text":"decisions","s":198.931,"e":199.331},{"text":"about","s":199.411,"e":199.731},{"text":"jobs,","s":199.971,"e":200.371},{"text":"money,","s":200.451,"e":200.931},{"text":"housing","s":201.171,"e":201.411},{"text":"or","s":201.491,"e":201.571},{"text":"health,","s":201.771,"e":202.291},{"text":"let","s":202.371,"e":202.531},{"text":"the","s":202.611,"e":202.691},{"text":"model","s":202.851,"e":203.011},{"text":"assist","s":203.251,"e":203.491},{"text":"a","s":203.531,"e":203.571},{"text":"person","s":203.651,"e":204.051},{"text":"rather","s":204.211,"e":204.451},{"text":"than","s":204.531,"e":204.611},{"text":"decide","s":204.731,"e":204.931},{"text":"alone,","s":205.011,"e":205.811},{"text":"and","s":205.971,"e":206.131},{"text":"check","s":206.371,"e":206.531},{"text":"the","s":206.571,"e":206.611},{"text":"rules","s":206.691,"e":207.011},{"text":"where","s":207.091,"e":207.171},{"text":"you","s":207.251,"e":207.331},{"text":"operate.","s":207.571,"e":208.131}]},{"template":"quiz","chapter":"Quick check","narration":"Quick check. Your name-swap test shows no change in any decision. What can you conclude?","params":{"q":"A name-swap test changes **no decisions**. What can you conclude?","options":["The model is fair; ship it","Names alone didn't move these cases; proxies and real subgroups still need testing","The test was broken","Bias only matters for LLMs"],"answer":1,"explain":"A counterfactual test checks one path for bias. Proxies like schools or addresses, and outcomes on real labeled cases by subgroup, still need checking."},"start":208.745,"dur":11.456,"words":[{"text":"Quick","s":209.045,"e":209.285},{"text":"check.","s":209.365,"e":209.885},{"text":"Your","s":210.085,"e":210.245},{"text":"name-swap","s":210.325,"e":210.805},{"text":"test","s":210.885,"e":211.205},{"text":"shows","s":211.285,"e":211.605},{"text":"no","s":211.685,"e":211.845},{"text":"change","s":212.005,"e":212.325},{"text":"in","s":212.405,"e":212.565},{"text":"any","s":212.645,"e":212.805},{"text":"decision.","s":212.885,"e":213.765},{"text":"What","s":213.845,"e":213.925},{"text":"can","s":214.005,"e":214.085},{"text":"you","s":214.125,"e":214.165},{"text":"conclude?","s":214.245,"e":215.045}]},{"template":"exercise","chapter":"Try it","narration":"Try it. Have your agent add a decision log and a counterfactual test to one decision that touches people.","params":{"title":"Log it and test it","intro":"Paste this into your coding agent:","prompt":"For one decision in our product that affects a person (approvals,\nrefunds, moderation, screening):\n1. Log every decision: input reference, model version, every answer,\n   thresholds, action, and a flag that sends 2% to human review.\n2. Generate a plain-language explanation from that log.\n3. Write a counterfactual test: for 20 real cases, swap attributes that\n   must not matter (names, pronouns, locations) and report the largest\n   change in each answer and any decision that flips."},"start":220.201,"dur":7.063,"words":[{"text":"Try","s":220.501,"e":220.741},{"text":"it.","s":220.821,"e":221.061},{"text":"Have","s":221.301,"e":221.461},{"text":"your","s":221.541,"e":221.621},{"text":"agent","s":221.701,"e":222.021},{"text":"add","s":222.101,"e":222.181},{"text":"a","s":222.221,"e":222.261},{"text":"decision","s":222.501,"e":222.741},{"text":"log","s":222.821,"e":223.061},{"text":"and","s":223.141,"e":223.221},{"text":"a","s":223.274,"e":223.328},{"text":"counterfactual","s":223.381,"e":224.101},{"text":"test","s":224.181,"e":224.501},{"text":"to","s":224.621,"e":224.741},{"text":"one","s":224.821,"e":224.981},{"text":"decision","s":225.221,"e":225.461},{"text":"that","s":225.541,"e":225.621},{"text":"touches","s":225.781,"e":225.941},{"text":"people.","s":226.021,"e":226.661}]},{"template":"recap","chapter":"Recap","narration":"So: decision models trade noise for systematic error. Bias can sit in models and in criteria, so test with counterfactuals and by subgroup, and don't treat one pass as proof. Your questions are the reasons: log them. And keep a person accountable. Next, the capstone: putting the whole course together.","params":{"items":["Little noise, so errors are systematic.","Bias hides in criteria too (Apgar).","Counterfactuals + subgroups; one pass isn't proof.","Log the reasons; keep a person accountable."],"next":"Capstone: a support triage pipeline","next_at":"Next"},"start":227.264,"dur":24.339,"words":[{"text":"So:","s":227.564,"e":228.284},{"text":"decision","s":228.364,"e":228.764},{"text":"models","s":228.844,"e":229.244},{"text":"trade","s":229.324,"e":229.564},{"text":"noise","s":229.644,"e":230.124},{"text":"for","s":230.204,"e":230.284},{"text":"systematic","s":230.444,"e":231.004},{"text":"error.","s":231.164,"e":231.564},{"text":"Bias","s":232.044,"e":232.444},{"text":"can","s":232.524,"e":232.604},{"text":"sit","s":232.684,"e":232.844},{"text":"in","s":232.884,"e":232.924},{"text":"models","s":233.084,"e":233.484},{"text":"and","s":233.644,"e":233.884},{"text":"in","s":233.964,"e":234.044},{"text":"criteria,","s":234.124,"e":235.084},{"text":"so","s":235.164,"e":235.324},{"text":"test","s":235.484,"e":235.804},{"text":"with","s":235.884,"e":235.964},{"text":"counterfactuals","s":236.124,"e":236.924},{"text":"and","s":237.044,"e":237.164},{"text":"by","s":237.244,"e":237.324},{"text":"subgroup,","s":237.484,"e":238.364},{"text":"and","s":238.484,"e":238.604},{"text":"don't","s":238.684,"e":238.924},{"text":"treat","s":238.964,"e":239.164},{"text":"one","s":239.244,"e":239.404},{"text":"pass","s":239.484,"e":239.724},{"text":"as","s":239.804,"e":239.884},{"text":"proof.","s":240.044,"e":240.924},{"text":"Your","s":241.004,"e":241.084},{"text":"questions","s":241.164,"e":241.724},{"text":"are","s":241.804,"e":241.884},{"text":"the","s":241.924,"e":241.964},{"text":"reasons:","s":242.044,"e":243.004},{"text":"log","s":243.084,"e":243.324},{"text":"them.","s":243.404,"e":244.044},{"text":"And","s":244.164,"e":244.284},{"text":"keep","s":244.404,"e":244.524},{"text":"a","s":244.604,"e":244.684},{"text":"person","s":244.884,"e":245.084},{"text":"accountable.","s":245.164,"e":246.844},{"text":"Next,","s":247.084,"e":247.164},{"text":"the","s":247.244,"e":247.484},{"text":"capstone:","s":247.804,"e":248.364},{"text":"putting","s":248.444,"e":248.684},{"text":"the","s":248.764,"e":249.004},{"text":"whole","s":249.164,"e":249.324},{"text":"course","s":249.404,"e":249.484},{"text":"together.","s":249.804,"e":250.044}]}],"total":252.003,"audio":"/audio/21-responsible.mp3"}