A decision model makes the same call the same way, millions of times. That removes noise, and it makes any bias systematic. Here is how to audit decisions about people (including the name-swap test we ran), where the reasons come from when the model gives none, and what a responsible deployment logs, samples and leaves to a person.
Key takeaways
- Decision models have little noise, so their mistakes are systematic: the same error, for the same kind of case, every time.
- Bias can hide in a criterion as well as in a model: the Apgar score's skin-color item gives Black newborns lower scores.
- Audit by subgroup and with counterfactual tests; in our name-swap test, answers moved at most 0.06 and no decision flipped, which is one test, not proof of fairness.
- The model gives numbers, not reasons; your decomposed questions and thresholds are the reasons, so log them and turn them into explanations.
- Keep a person accountable: sample automated decisions for review, give people a way to appeal, and be most careful with decisions about jobs, money, housing and health.
Try it with your agent
Paste this into your coding agent:
For one decision in our product that affects a person (approvals, refunds, moderation, screening): 1. Log every decision: input reference, model version, every answer, thresholds, action, and a flag that sends 2% to human review. 2. Generate a plain-language explanation from that log. 3. Write a counterfactual test: for 20 real cases, swap attributes that must not matter (names, pronouns, locations) and report the largest change in each answer and any decision that flips.
Transcript
Intro
A decision model will make the same call the same way, a million times a day. That's its strength. It's also why some of those calls deserve extra care.
Noise and bias
Olivier Sibony and Daniel Kahneman separate two kinds of error. Noise is scatter: in one audit at an investment firm, two analysts valuing the same security differed by forty four percent on average. Bias is a shared, predictable error. Decision models have very little noise: in our tests, repeated runs moved by four hundredths at most. So whatever errors they have are systematic. The same mistake, for the same kind of case, every time.
Decisions about people
That matters most when the decision is about a person. Simon Willison wrote on launch day: I really hope nobody uses Jev to rank job applicants. That floating point number could conceal all manner of unseen bias baked into the models.
Bias in the criteria
And bias doesn't only live inside models. It can live in your criteria. Remember the Apgar score from lesson twelve? One of its five items is skin color. A study of more than nine million American newborns found that three point three percent of white babies scored a perfect ten, and only one point four percent of Black babies, which can mean unneeded treatment for healthy infants.
Test it
So test it, with counterfactuals. Change only what must not matter, and see if the answer moves. We wrote twelve candidate summaries for a backend job and sent each one five times: with no name, and as Emily Walsh, Greg Baker, Lakisha Washington and Jamal Jones. The average chance of an interview barely moved. No single answer moved by more than point oh six, and no decision flipped.
That's reassuring, and it proves very little. It's one test, of one attribute, on twelve made-up people. Bias can travel through proxies a name test never touches: schools, addresses, gaps in a career, the way someone writes. As Willison's advice goes, a model like this needs more evals, not fewer. Test by subgroup on real, labeled cases too.
Where the reasons come from
People also ask for reasons. Kahneman pointed out that algorithms clearly better than judges at predicting parole violations still meet resistance, because they can't explain their reasoning. A decision model gives you numbers, not reasons. But your design has them. Unsiloed put it simply: the explanations come from our application. Jev returns the typed answers.
So log the reasons you designed. Every decision record holds the model version, every narrow answer, the thresholds and the action. That's an audit trail. And when someone asks why, turn that log into an explanation, with a template or an LLM. The decision stays in your code; only the wording is generated.
Keep a person accountable
And keep a person accountable. Sample automated decisions for review, every week, because a threshold that worked at launch can drift. Give people a way to appeal to a human. Audit outcomes by subgroup. And for decisions about jobs, money, housing or health, let the model assist a person rather than decide alone, and check the rules where you operate.
Quick check
Quick check. Your name-swap test shows no change in any decision. What can you conclude?
Try it
Try it. Have your agent add a decision log and a counterfactual test to one decision that touches people.
Recap
So: decision models trade noise for systematic error. Bias can sit in models and in criteria, so test with counterfactuals and by subgroup, and don't treat one pass as proof. Your questions are the reasons: log them. And keep a person accountable. Next, the capstone: putting the whole course together.