{"id":"07-score","number":7,"slug":"score-questions","title":"Score: a position on a scale you describe","short":"Score","module":"Asking good questions","summary":"A Score places the state on ordered levels you describe and returns a position that can fall between them. Here is how the number is computed, why levels must describe situations rather than degrees, why one Score should measure one thing, and how an extra top level separates the dangerous cases.","voice":"Jessica","scenes":[{"template":"title","chapter":"Intro","narration":"Some answers aren't yes or no, or one of a few categories. They're a position on a scale: how severe, how frustrated, how much experience. That's a Score.","params":{"kicker":"Lesson 7","module":"Module 2 · Asking good questions","title":"Score: a position on a scale *you describe*","subtitle":"Ordered levels in, a position between them out.","cast":["snap",{"who":"mull","mood":"think"}]},"start":0.5,"dur":12.746,"words":[{"text":"Some","s":0.8,"e":1.04},{"text":"answers","s":1.12,"e":1.52},{"text":"aren't","s":1.68,"e":1.92},{"text":"yes","s":2,"e":2.24},{"text":"or","s":2.28,"e":2.32},{"text":"no,","s":2.48,"e":3.04},{"text":"or","s":3.16,"e":3.28},{"text":"one","s":3.44,"e":3.52},{"text":"of","s":3.56,"e":3.6},{"text":"a","s":3.68,"e":3.72},{"text":"few","s":3.76,"e":3.92},{"text":"categories.","s":4,"e":4.72},{"text":"They're","s":5.12,"e":5.28},{"text":"a","s":5.307,"e":5.333},{"text":"position","s":5.36,"e":5.92},{"text":"on","s":6,"e":6.08},{"text":"a","s":6.16,"e":6.24},{"text":"scale:","s":6.32,"e":7.28},{"text":"how","s":7.36,"e":7.44},{"text":"severe,","s":7.52,"e":8.24},{"text":"how","s":8.32,"e":8.4},{"text":"frustrated,","s":8.56,"e":9.44},{"text":"how","s":9.52,"e":9.6},{"text":"much","s":9.68,"e":9.76},{"text":"experience.","s":9.84,"e":10.64},{"text":"That's","s":11.04,"e":11.44},{"text":"a","s":11.48,"e":11.52},{"text":"Score.","s":11.6,"e":12.4}]},{"template":"levels","chapter":"Reading a Score","narration":"Three levels for bug severity: cosmetic, broken but there's a workaround, and blocking with no workaround. This bug crashes the page in Safari, but works in Chrome, and some customers only use Safari. The model puts point five six on workaround and point four four on blocking, so the position lands at one point four four.","params":{"question":"How severe is the reported issue?","state":"The export button crashes the settings page in Safari. It works in Chrome, but a few of our customers only use Safari.","levels":[{"text":"Cosmetic; no impact to functionality","p":0},{"text":"Broken or degraded feature, but workaround exists","p":0.56},{"text":"Blocking issue; no workaround exists","p":0.44}],"score":1.44,"at":"point five six","score_at":"lands at"},"start":13.246,"dur":23.232,"words":[{"text":"Three","s":13.546,"e":13.786},{"text":"levels","s":13.866,"e":14.266},{"text":"for","s":14.346,"e":14.426},{"text":"bug","s":14.506,"e":14.666},{"text":"severity:","s":14.746,"e":15.706},{"text":"cosmetic,","s":15.826,"e":16.666},{"text":"broken","s":16.746,"e":17.226},{"text":"but","s":17.306,"e":17.466},{"text":"there's","s":17.546,"e":17.706},{"text":"a","s":17.746,"e":17.786},{"text":"workaround,","s":17.906,"e":18.746},{"text":"and","s":18.826,"e":19.066},{"text":"blocking","s":19.306,"e":19.866},{"text":"with","s":19.946,"e":20.106},{"text":"no","s":20.266,"e":20.426},{"text":"workaround.","s":20.506,"e":21.146},{"text":"This","s":21.786,"e":21.946},{"text":"bug","s":22.026,"e":22.266},{"text":"crashes","s":22.346,"e":22.746},{"text":"the","s":22.826,"e":22.866},{"text":"page","s":22.906,"e":23.146},{"text":"in","s":23.186,"e":23.226},{"text":"Safari,","s":23.306,"e":24.026},{"text":"but","s":24.146,"e":24.266},{"text":"works","s":24.386,"e":24.506},{"text":"in","s":24.586,"e":24.666},{"text":"Chrome,","s":24.746,"e":25.466},{"text":"and","s":25.586,"e":25.706},{"text":"some","s":25.786,"e":25.946},{"text":"customers","s":26.026,"e":26.506},{"text":"only","s":26.586,"e":26.826},{"text":"use","s":26.906,"e":27.066},{"text":"Safari.","s":27.106,"e":28.106},{"text":"The","s":28.146,"e":28.186},{"text":"model","s":28.266,"e":28.506},{"text":"puts","s":28.586,"e":28.826},{"text":"point","s":28.906,"e":29.146},{"text":"five","s":29.226,"e":29.466},{"text":"six","s":29.586,"e":29.706},{"text":"on","s":29.786,"e":29.866},{"text":"workaround","s":29.986,"e":30.506},{"text":"and","s":30.666,"e":30.906},{"text":"point","s":31.066,"e":31.226},{"text":"four","s":31.386,"e":31.546},{"text":"four","s":31.706,"e":31.866},{"text":"on","s":31.946,"e":32.026},{"text":"blocking,","s":32.306,"e":32.906},{"text":"so","s":32.986,"e":33.146},{"text":"the","s":33.226,"e":33.306},{"text":"position","s":33.386,"e":33.786},{"text":"lands","s":33.986,"e":34.186},{"text":"at","s":34.306,"e":34.426},{"text":"one","s":34.506,"e":34.586},{"text":"point","s":34.746,"e":34.986},{"text":"four","s":35.066,"e":35.306},{"text":"four.","s":35.466,"e":35.866}]},{"template":"formula","chapter":"Reading a Score","narration":"The math is simple. Each level number, times its probability, added up. Zero times zero, plus one times point five six, plus two times point four four. One point four four. And confidence is low, point three four, because the model is torn between two levels.","params":{"title":"score = Σ level × probability","parts":[{"text":"0 × 0.00","tone":"paper"},{"op":"+"},{"text":"1 × 0.56","tone":"paper"},{"op":"+"},{"text":"2 × 0.44","tone":"paper"},{"op":"="},{"text":"1.44","sub":"confidence 0.34","tone":"agent"}],"caption":"Low confidence on a Score means the levels overlap for this case, the question measures two things, or the state doesn't say enough.","caption_at":"torn between"},"start":36.478,"dur":19.392,"words":[{"text":"The","s":36.778,"e":36.938},{"text":"math","s":37.018,"e":37.178},{"text":"is","s":37.258,"e":37.338},{"text":"simple.","s":37.418,"e":38.058},{"text":"Each","s":38.298,"e":38.458},{"text":"level","s":38.538,"e":38.778},{"text":"number,","s":38.858,"e":39.418},{"text":"times","s":39.578,"e":39.738},{"text":"its","s":39.818,"e":39.898},{"text":"probability,","s":39.978,"e":40.858},{"text":"added","s":41.018,"e":41.178},{"text":"up.","s":41.258,"e":41.898},{"text":"Zero","s":41.978,"e":42.218},{"text":"times","s":42.298,"e":42.538},{"text":"zero,","s":42.618,"e":43.258},{"text":"plus","s":43.378,"e":43.498},{"text":"one","s":43.658,"e":43.818},{"text":"times","s":43.898,"e":44.218},{"text":"point","s":44.298,"e":44.458},{"text":"five","s":44.538,"e":44.698},{"text":"six,","s":44.858,"e":45.498},{"text":"plus","s":45.658,"e":45.818},{"text":"two","s":45.978,"e":46.138},{"text":"times","s":46.218,"e":46.538},{"text":"point","s":46.658,"e":46.778},{"text":"four","s":46.898,"e":47.018},{"text":"four.","s":47.178,"e":47.818},{"text":"One","s":47.898,"e":47.978},{"text":"point","s":48.138,"e":48.458},{"text":"four","s":48.538,"e":48.618},{"text":"four.","s":48.698,"e":49.498},{"text":"And","s":49.618,"e":49.738},{"text":"confidence","s":49.818,"e":50.298},{"text":"is","s":50.378,"e":50.538},{"text":"low,","s":50.698,"e":51.258},{"text":"point","s":51.378,"e":51.498},{"text":"three","s":51.578,"e":51.818},{"text":"four,","s":51.978,"e":52.378},{"text":"because","s":52.458,"e":52.698},{"text":"the","s":52.738,"e":52.778},{"text":"model","s":52.858,"e":53.098},{"text":"is","s":53.178,"e":53.338},{"text":"torn","s":53.418,"e":53.738},{"text":"between","s":53.938,"e":54.138},{"text":"two","s":54.218,"e":54.298},{"text":"levels.","s":54.378,"e":55.258}]},{"template":"compare","chapter":"Describe situations","narration":"Rule one: describe situations, not degrees. Each level is judged on its own, so a level called just two, or moderately severe, gives the model nothing to match. For a pixel-misaligned button, levels written as bare numbers scored point seven one, unsure. With described levels: zero point zero, at full confidence.","params":{"title":"“The export button is misaligned by a few pixels.”","left":{"title":"Levels: “0”, “1”, “2”","tone":"bad","items":[{"text":"score **0.71** · confidence 0.52","at":"point seven one"},{"text":"Nothing to match against"}],"at":"bare numbers"},"right":{"title":"Levels describe situations","tone":"good","items":[{"text":"score **0.00** · confidence 1.00","at":"zero point zero"},{"text":"“Cosmetic; no impact to functionality”"}],"at":"described levels"},"vs":"vs"},"start":55.87,"dur":24.826,"words":[{"text":"Rule","s":56.17,"e":56.45},{"text":"one:","s":56.49,"e":56.89},{"text":"describe","s":57.61,"e":57.85},{"text":"situations,","s":57.93,"e":59.05},{"text":"not","s":59.13,"e":59.37},{"text":"degrees.","s":59.45,"e":60.17},{"text":"Each","s":60.65,"e":60.81},{"text":"level","s":60.97,"e":61.21},{"text":"is","s":61.29,"e":61.45},{"text":"judged","s":61.53,"e":61.77},{"text":"on","s":61.85,"e":61.93},{"text":"its","s":62.01,"e":62.09},{"text":"own,","s":62.17,"e":62.73},{"text":"so","s":62.81,"e":63.05},{"text":"a","s":63.13,"e":63.21},{"text":"level","s":63.33,"e":63.45},{"text":"called","s":63.53,"e":63.77},{"text":"just","s":63.93,"e":64.17},{"text":"two,","s":64.25,"e":64.73},{"text":"or","s":64.81,"e":64.97},{"text":"moderately","s":65.21,"e":65.69},{"text":"severe,","s":65.77,"e":66.49},{"text":"gives","s":66.57,"e":66.73},{"text":"the","s":66.77,"e":66.81},{"text":"model","s":66.89,"e":67.13},{"text":"nothing","s":67.29,"e":67.61},{"text":"to","s":67.65,"e":67.69},{"text":"match.","s":67.77,"e":68.49},{"text":"For","s":68.73,"e":68.81},{"text":"a","s":68.89,"e":68.93},{"text":"pixel-misaligned","s":68.97,"e":69.85},{"text":"button,","s":69.93,"e":70.57},{"text":"levels","s":70.65,"e":70.97},{"text":"written","s":71.09,"e":71.21},{"text":"as","s":71.29,"e":71.45},{"text":"bare","s":71.53,"e":71.77},{"text":"numbers","s":71.85,"e":72.33},{"text":"scored","s":72.41,"e":72.81},{"text":"point","s":72.97,"e":73.21},{"text":"seven","s":73.37,"e":73.53},{"text":"one,","s":73.69,"e":74.33},{"text":"unsure.","s":74.49,"e":75.37},{"text":"With","s":75.45,"e":75.53},{"text":"described","s":75.61,"e":76.09},{"text":"levels:","s":76.17,"e":76.97},{"text":"zero","s":77.13,"e":77.29},{"text":"point","s":77.45,"e":77.69},{"text":"zero,","s":77.77,"e":78.49},{"text":"at","s":78.57,"e":78.65},{"text":"full","s":78.81,"e":78.97},{"text":"confidence.","s":79.13,"e":80.09}]},{"template":"compare","chapter":"One dimension","narration":"Rule two: one dimension per Score. Rate a contractor on a scale from late, unskilled and rude to punctual, skilled and friendly. One who arrived on time, couldn't answer basic questions, and was very polite gets a confident Mixed. Which tells you nothing. Split it into three Scores, and you get on time, no skill, very polite.","params":{"title":"“Arrived on time, couldn't answer basic questions about our stack. Very polite.”","left":{"title":"One scale, three things","tone":"bad","items":[{"text":"“Mixed” at confidence **1.0**","at":"confident Mixed"},{"text":"Which tells you nothing","at":"tells you nothing"}]},"right":{"title":"Three Scores","tone":"good","items":[{"text":"punctual → **on time**","at":"on time"},{"text":"skill → **couldn't answer basics**","at":"no skill"},{"text":"manner → **very polite**","at":"very polite"}],"at":"Split it"},"vs":"vs"},"start":80.696,"dur":24.434,"words":[{"text":"Rule","s":80.996,"e":81.206},{"text":"two:","s":81.236,"e":81.636},{"text":"one","s":82.196,"e":82.383},{"text":"dimension","s":82.436,"e":82.943},{"text":"per","s":82.996,"e":83.236},{"text":"Score.","s":83.316,"e":83.796},{"text":"Rate","s":84.356,"e":84.516},{"text":"a","s":84.543,"e":84.569},{"text":"contractor","s":84.596,"e":85.156},{"text":"on","s":85.236,"e":85.316},{"text":"a","s":85.396,"e":85.436},{"text":"scale","s":85.476,"e":85.796},{"text":"from","s":85.876,"e":86.036},{"text":"late,","s":86.196,"e":86.676},{"text":"unskilled","s":86.756,"e":87.316},{"text":"and","s":87.396,"e":87.476},{"text":"rude","s":87.556,"e":87.876},{"text":"to","s":88.116,"e":88.276},{"text":"punctual,","s":88.436,"e":89.156},{"text":"skilled","s":89.236,"e":89.636},{"text":"and","s":89.716,"e":89.796},{"text":"friendly.","s":89.876,"e":90.356},{"text":"One","s":90.836,"e":90.996},{"text":"who","s":91.036,"e":91.076},{"text":"arrived","s":91.156,"e":91.476},{"text":"on","s":91.556,"e":91.636},{"text":"time,","s":91.716,"e":92.276},{"text":"couldn't","s":92.356,"e":92.596},{"text":"answer","s":92.676,"e":92.916},{"text":"basic","s":93.076,"e":93.396},{"text":"questions,","s":93.476,"e":94.036},{"text":"and","s":94.076,"e":94.116},{"text":"was","s":94.196,"e":94.356},{"text":"very","s":94.516,"e":94.756},{"text":"polite","s":94.836,"e":95.236},{"text":"gets","s":95.556,"e":95.716},{"text":"a","s":95.796,"e":95.876},{"text":"confident","s":95.956,"e":96.436},{"text":"Mixed.","s":96.596,"e":97.276},{"text":"Which","s":97.476,"e":97.716},{"text":"tells","s":97.756,"e":97.956},{"text":"you","s":98.036,"e":98.116},{"text":"nothing.","s":98.276,"e":98.836},{"text":"Split","s":99.316,"e":99.556},{"text":"it","s":99.596,"e":99.636},{"text":"into","s":99.716,"e":99.876},{"text":"three","s":100.036,"e":100.196},{"text":"Scores,","s":100.276,"e":100.836},{"text":"and","s":100.876,"e":100.916},{"text":"you","s":100.996,"e":101.076},{"text":"get","s":101.196,"e":101.316},{"text":"on","s":101.556,"e":101.716},{"text":"time,","s":101.796,"e":102.436},{"text":"no","s":102.516,"e":102.676},{"text":"skill,","s":102.756,"e":103.396},{"text":"very","s":103.476,"e":103.716},{"text":"polite.","s":103.796,"e":104.516}]},{"template":"statement","chapter":"One dimension","narration":"This isn't just a model quirk. Decision researcher Olivier Sibony gives the same advice to human hiring panels: evaluate each criterion separately from the others, so one strong impression doesn't color the rest. Then combine them, which, for us, happens in code.","params":{"eyebrow":"Olivier Sibony, on noise in human judgment","text":"Evaluate each criterion *separately* from the others.","art":"users","tone":"s2","punch":"evaluate each criterion separately"},"start":105.13,"dur":18.739,"words":[{"text":"This","s":105.43,"e":105.67},{"text":"isn't","s":105.75,"e":105.99},{"text":"just","s":106.07,"e":106.23},{"text":"a","s":106.31,"e":106.39},{"text":"model","s":106.47,"e":106.71},{"text":"quirk.","s":106.87,"e":107.27},{"text":"Decision","s":107.67,"e":108.07},{"text":"researcher","s":108.15,"e":108.63},{"text":"Olivier","s":108.71,"e":109.19},{"text":"Sibony","s":109.27,"e":109.83},{"text":"gives","s":109.87,"e":110.07},{"text":"the","s":110.19,"e":110.31},{"text":"same","s":110.43,"e":110.55},{"text":"advice","s":110.63,"e":110.95},{"text":"to","s":111.03,"e":111.11},{"text":"human","s":111.19,"e":111.43},{"text":"hiring","s":111.51,"e":111.83},{"text":"panels:","s":111.91,"e":112.87},{"text":"evaluate","s":112.95,"e":113.59},{"text":"each","s":113.67,"e":113.83},{"text":"criterion","s":113.91,"e":114.55},{"text":"separately","s":114.71,"e":115.11},{"text":"from","s":115.19,"e":115.35},{"text":"the","s":115.39,"e":115.43},{"text":"others,","s":115.51,"e":116.15},{"text":"so","s":116.23,"e":116.47},{"text":"one","s":116.63,"e":116.87},{"text":"strong","s":116.95,"e":117.27},{"text":"impression","s":117.35,"e":117.83},{"text":"doesn't","s":117.99,"e":118.31},{"text":"color","s":118.39,"e":118.55},{"text":"the","s":118.63,"e":118.71},{"text":"rest.","s":118.95,"e":119.59},{"text":"Then","s":119.75,"e":119.91},{"text":"combine","s":120.07,"e":120.47},{"text":"them,","s":120.59,"e":121.03},{"text":"which,","s":121.11,"e":121.43},{"text":"for","s":121.47,"e":121.51},{"text":"us,","s":121.67,"e":122.07},{"text":"happens","s":122.11,"e":122.39},{"text":"in","s":122.47,"e":122.55},{"text":"code.","s":122.63,"e":123.27}]},{"template":"code","chapter":"Combine in code","narration":"Before you combine Scores, normalize them. A four level scale runs from zero to three, a three level scale from zero to two. Divide each by its top level, so everything runs from zero to one. Now your weights mean what they say.","params":{"file":"priority.py","lang":"py","lines":["a = response.answers","severity    = a[\"severity\"].score / 2       # 3 levels: 0..2","frustration = a[\"frustration\"].score / 2    # 3 levels: 0..2","quality     = a[\"report_quality\"].score / 3 # 4 levels: 0..3","","priority = 0.6 * severity + 0.3 * frustration + 0.1 * quality"],"hl":[{"from":2,"to":4,"label":"divide by the top level","tone":"agent","at":"Divide each"},{"from":6,"to":6,"label":"weights mean what they say","tone":"good","at":"weights"}]},"start":123.869,"dur":16.441,"words":[{"text":"Before","s":124.169,"e":124.569},{"text":"you","s":124.609,"e":124.649},{"text":"combine","s":124.729,"e":125.209},{"text":"Scores,","s":125.289,"e":126.009},{"text":"normalize","s":126.089,"e":126.649},{"text":"them.","s":126.769,"e":126.969},{"text":"A","s":127.369,"e":127.449},{"text":"four","s":127.529,"e":127.689},{"text":"level","s":127.769,"e":127.929},{"text":"scale","s":128.009,"e":128.329},{"text":"runs","s":128.409,"e":128.649},{"text":"from","s":128.729,"e":128.889},{"text":"zero","s":128.969,"e":129.209},{"text":"to","s":129.369,"e":129.529},{"text":"three,","s":129.609,"e":130.249},{"text":"a","s":130.329,"e":130.409},{"text":"three","s":130.489,"e":130.649},{"text":"level","s":130.729,"e":130.889},{"text":"scale","s":130.969,"e":131.289},{"text":"from","s":131.369,"e":131.529},{"text":"zero","s":131.609,"e":131.769},{"text":"to","s":131.929,"e":132.089},{"text":"two.","s":132.209,"e":132.409},{"text":"Divide","s":132.889,"e":133.209},{"text":"each","s":133.289,"e":133.529},{"text":"by","s":133.609,"e":133.689},{"text":"its","s":133.809,"e":133.929},{"text":"top","s":134.049,"e":134.169},{"text":"level,","s":134.249,"e":134.729},{"text":"so","s":134.809,"e":135.049},{"text":"everything","s":135.209,"e":135.609},{"text":"runs","s":135.689,"e":135.929},{"text":"from","s":136.009,"e":136.169},{"text":"zero","s":136.249,"e":136.489},{"text":"to","s":136.649,"e":136.809},{"text":"one.","s":136.889,"e":137.209},{"text":"Now","s":137.689,"e":137.929},{"text":"your","s":138.009,"e":138.169},{"text":"weights","s":138.249,"e":138.489},{"text":"mean","s":138.569,"e":138.729},{"text":"what","s":138.809,"e":138.889},{"text":"they","s":138.969,"e":139.049},{"text":"say.","s":139.129,"e":139.689}]},{"template":"levels","chapter":"Room for the extreme","narration":"Rule three: give the rare extreme its own level. On a calm, annoyed, very angry scale, a customer threatening your support team scores one point eight eight, right next to someone who's merely furious. Add a fourth level, abusive or threatening, and the threat lands at two point eight six, while ordinary anger stays at two.","params":{"question":"How angry is the customer?","state":"If this isn't fixed by 5pm I will find out where your support team lives.","levels":[{"text":"Calm","p":0},{"text":"Annoyed","p":0.02},{"text":"Very angry","p":0.11},{"text":"Abusive or threatening","p":0.87}],"score":2.86,"at":"Add a fourth level","score_at":"two point eight six","caption":"Without the 4th level: 1.88. “I am done with your company” stays at 2.00."},"start":140.31,"dur":23.232,"words":[{"text":"Rule","s":140.61,"e":140.85},{"text":"three:","s":141.01,"e":141.81},{"text":"give","s":141.89,"e":142.05},{"text":"the","s":142.13,"e":142.21},{"text":"rare","s":142.37,"e":142.61},{"text":"extreme","s":142.69,"e":143.25},{"text":"its","s":143.33,"e":143.57},{"text":"own","s":143.65,"e":143.81},{"text":"level.","s":143.89,"e":144.85},{"text":"On","s":144.93,"e":145.01},{"text":"a","s":145.05,"e":145.09},{"text":"calm,","s":145.17,"e":145.81},{"text":"annoyed,","s":145.93,"e":146.69},{"text":"very","s":146.77,"e":147.09},{"text":"angry","s":147.25,"e":147.57},{"text":"scale,","s":147.65,"e":148.37},{"text":"a","s":148.45,"e":148.53},{"text":"customer","s":148.73,"e":148.93},{"text":"threatening","s":149.01,"e":149.33},{"text":"your","s":149.41,"e":149.49},{"text":"support","s":149.65,"e":149.81},{"text":"team","s":149.89,"e":150.21},{"text":"scores","s":150.29,"e":150.69},{"text":"one","s":150.77,"e":150.93},{"text":"point","s":151.01,"e":151.33},{"text":"eight","s":151.45,"e":151.57},{"text":"eight,","s":151.65,"e":152.21},{"text":"right","s":152.29,"e":152.37},{"text":"next","s":152.45,"e":152.61},{"text":"to","s":152.65,"e":152.69},{"text":"someone","s":152.77,"e":153.09},{"text":"who's","s":153.13,"e":153.33},{"text":"merely","s":153.41,"e":153.65},{"text":"furious.","s":153.81,"e":154.73},{"text":"Add","s":155.01,"e":155.09},{"text":"a","s":155.17,"e":155.25},{"text":"fourth","s":155.33,"e":155.57},{"text":"level,","s":155.65,"e":156.21},{"text":"abusive","s":156.33,"e":156.93},{"text":"or","s":156.97,"e":157.01},{"text":"threatening,","s":157.17,"e":158.05},{"text":"and","s":158.13,"e":158.21},{"text":"the","s":158.25,"e":158.29},{"text":"threat","s":158.37,"e":158.61},{"text":"lands","s":158.77,"e":158.93},{"text":"at","s":159.01,"e":159.17},{"text":"two","s":159.25,"e":159.33},{"text":"point","s":159.41,"e":159.65},{"text":"eight","s":159.77,"e":159.89},{"text":"six,","s":159.97,"e":160.61},{"text":"while","s":160.77,"e":160.93},{"text":"ordinary","s":161.17,"e":161.41},{"text":"anger","s":161.49,"e":161.81},{"text":"stays","s":161.89,"e":162.21},{"text":"at","s":162.29,"e":162.37},{"text":"two.","s":162.53,"e":162.93}]},{"template":"statement","chapter":"What a score is not","narration":"And one warning from the Jev docs. A score is a position for ranking and thresholds, not a measurement. Don't interpolate it into an exact quantity. One point four four doesn't mean forty four percent of customers lack a workaround.","params":{"eyebrow":"Jev 1.13 jaggedness notes","text":"Use a score to *rank and threshold*, not to measure an exact quantity.","art":"warn","tone":"bad","punch":"not a measurement"},"start":163.542,"dur":15.866,"words":[{"text":"And","s":163.842,"e":164.082},{"text":"one","s":164.162,"e":164.322},{"text":"warning","s":164.402,"e":164.802},{"text":"from","s":164.882,"e":164.962},{"text":"the","s":165.042,"e":165.082},{"text":"Jev","s":165.122,"e":165.362},{"text":"docs.","s":165.602,"e":165.922},{"text":"A","s":166.322,"e":166.402},{"text":"score","s":166.482,"e":166.882},{"text":"is","s":166.962,"e":167.042},{"text":"a","s":167.082,"e":167.122},{"text":"position","s":167.202,"e":167.602},{"text":"for","s":167.682,"e":167.762},{"text":"ranking","s":167.842,"e":168.242},{"text":"and","s":168.282,"e":168.322},{"text":"thresholds,","s":168.482,"e":169.362},{"text":"not","s":169.522,"e":169.842},{"text":"a","s":169.922,"e":169.962},{"text":"measurement.","s":170.002,"e":170.642},{"text":"Don't","s":171.042,"e":171.282},{"text":"interpolate","s":171.442,"e":171.922},{"text":"it","s":171.962,"e":172.002},{"text":"into","s":172.082,"e":172.242},{"text":"an","s":172.322,"e":172.362},{"text":"exact","s":172.402,"e":172.802},{"text":"quantity.","s":172.962,"e":173.682},{"text":"One","s":173.922,"e":174.002},{"text":"point","s":174.082,"e":174.242},{"text":"four","s":174.402,"e":174.562},{"text":"four","s":174.722,"e":175.042},{"text":"doesn't","s":175.282,"e":175.602},{"text":"mean","s":175.682,"e":175.842},{"text":"forty","s":176.082,"e":176.322},{"text":"four","s":176.402,"e":176.482},{"text":"percent","s":176.562,"e":176.882},{"text":"of","s":176.922,"e":176.962},{"text":"customers","s":177.042,"e":177.522},{"text":"lack","s":177.562,"e":177.762},{"text":"a","s":177.802,"e":177.842},{"text":"workaround.","s":177.962,"e":178.802}]},{"template":"quiz","chapter":"Quick check","narration":"Quick check. You rate support replies on one scale: one, poor, to five, excellent. Scores keep landing near three with low confidence. What's the most likely fix?","params":{"q":"A 1-to-5 “reply quality” Score keeps landing near **3 with low confidence**. Best fix?","options":["Add more levels, say 1 to 10","Describe each level as a concrete situation, and split quality into separate dimensions","Average three runs","Switch to a Noul: “Is this reply good?”"],"answer":1,"explain":"Bare numbers give the model nothing to match, and “quality” bundles accuracy, tone and completeness. Describe situations and score each dimension separately."},"start":179.408,"dur":17.307,"words":[{"text":"Quick","s":179.708,"e":179.948},{"text":"check.","s":180.028,"e":180.348},{"text":"You","s":180.748,"e":180.828},{"text":"rate","s":180.908,"e":181.068},{"text":"support","s":181.268,"e":181.468},{"text":"replies","s":181.548,"e":181.948},{"text":"on","s":182.028,"e":182.188},{"text":"one","s":182.268,"e":182.428},{"text":"scale:","s":182.508,"e":183.308},{"text":"one,","s":183.468,"e":183.868},{"text":"poor,","s":183.948,"e":184.588},{"text":"to","s":184.628,"e":184.668},{"text":"five,","s":184.828,"e":185.388},{"text":"excellent.","s":185.508,"e":186.348},{"text":"Scores","s":186.428,"e":186.748},{"text":"keep","s":186.828,"e":186.988},{"text":"landing","s":187.148,"e":187.308},{"text":"near","s":187.468,"e":187.628},{"text":"three","s":187.708,"e":188.108},{"text":"with","s":188.188,"e":188.268},{"text":"low","s":188.428,"e":188.588},{"text":"confidence.","s":188.668,"e":189.788},{"text":"What's","s":189.868,"e":190.028},{"text":"the","s":190.068,"e":190.108},{"text":"most","s":190.188,"e":190.348},{"text":"likely","s":190.508,"e":190.668},{"text":"fix?","s":190.748,"e":191.548}]},{"template":"exercise","chapter":"Try it","narration":"Try it. Have your agent rewrite one rating in your product as well-built Scores.","params":{"title":"Rebuild a rating","intro":"Paste this into your coding agent:","prompt":"Find one place where we rate something on a scale (priority, quality,\nseverity, risk). Rewrite it as Score questions:\n- one dimension per Score; split compound scales\n- every level a concrete, self-contained situation (no bare numbers)\n- a separate top level for rare extremes that need different handling\nNormalize each score by its top level, combine with explicit weights in\ncode, and show 5 real examples with each score and the combined result."},"start":196.715,"dur":6.436,"words":[{"text":"Try","s":197.015,"e":197.335},{"text":"it.","s":197.455,"e":197.895},{"text":"Have","s":197.975,"e":198.055},{"text":"your","s":198.135,"e":198.215},{"text":"agent","s":198.295,"e":198.535},{"text":"rewrite","s":198.615,"e":199.095},{"text":"one","s":199.175,"e":199.335},{"text":"rating","s":199.415,"e":199.735},{"text":"in","s":199.775,"e":199.815},{"text":"your","s":199.895,"e":199.975},{"text":"product","s":200.055,"e":200.455},{"text":"as","s":200.535,"e":200.775},{"text":"well-built","s":201.015,"e":201.575},{"text":"Scores.","s":201.655,"e":202.535}]},{"template":"recap","chapter":"Recap","narration":"So: a Score is a position, level times probability, added up. Describe situations, not degrees. One dimension per Score. Give the extreme its own level. And use the number to rank, not to measure. Next: the state, what you send the model to judge.","params":{"items":["score = Σ level × probability; it can land between levels.","Describe situations, never bare numbers or degrees.","One dimension per Score; normalize, then weight in code.","Give rare extremes their own level; rank, don't measure."],"next":"State: give it what it needs","next_at":"Next"},"start":203.151,"dur":22.171,"words":[{"text":"So:","s":203.451,"e":203.851},{"text":"a","s":204.251,"e":204.291},{"text":"Score","s":204.331,"e":204.731},{"text":"is","s":204.811,"e":204.931},{"text":"a","s":204.971,"e":205.051},{"text":"position,","s":205.091,"e":205.771},{"text":"level","s":206.091,"e":206.427},{"text":"times","s":206.491,"e":206.891},{"text":"probability,","s":206.971,"e":207.851},{"text":"added","s":207.931,"e":208.171},{"text":"up.","s":208.251,"e":208.691},{"text":"Describe","s":208.891,"e":209.371},{"text":"situations,","s":209.451,"e":210.571},{"text":"not","s":210.651,"e":210.811},{"text":"degrees.","s":210.891,"e":211.611},{"text":"One","s":212.011,"e":212.171},{"text":"dimension","s":212.251,"e":212.731},{"text":"per","s":212.891,"e":213.051},{"text":"Score.","s":213.211,"e":213.691},{"text":"Give","s":214.091,"e":214.251},{"text":"the","s":214.291,"e":214.331},{"text":"extreme","s":214.411,"e":215.051},{"text":"its","s":215.131,"e":215.291},{"text":"own","s":215.371,"e":215.611},{"text":"level.","s":215.811,"e":216.091},{"text":"And","s":216.411,"e":216.731},{"text":"use","s":216.891,"e":217.051},{"text":"the","s":217.131,"e":217.171},{"text":"number","s":217.211,"e":217.451},{"text":"to","s":217.531,"e":217.611},{"text":"rank,","s":217.771,"e":218.331},{"text":"not","s":218.411,"e":218.651},{"text":"to","s":218.731,"e":218.811},{"text":"measure.","s":218.891,"e":219.811},{"text":"Next:","s":220.331,"e":220.691},{"text":"the","s":220.731,"e":220.811},{"text":"state,","s":221.291,"e":221.771},{"text":"what","s":221.851,"e":221.931},{"text":"you","s":222.091,"e":222.211},{"text":"send","s":222.331,"e":222.371},{"text":"the","s":222.411,"e":222.491},{"text":"model","s":222.731,"e":222.811},{"text":"to","s":222.891,"e":222.971},{"text":"judge.","s":223.451,"e":223.771}]}],"total":225.722,"audio":"/audio/07-score.mp3"}