# System One

> Learn to build with System One decision models like TypeSafe's Jev, OpenAI's Decisions API, Cloudflare's Clef and AWS's Strands Decider. 22 short animated lessons on typed questions, calibrated probabilities, thresholds, composition patterns and evaluation, distilled from the docs, 57 talks and 70+ articles and papers, plus original measurements. Free and open source, and your agent can take it too.

Each lesson below is a Markdown file with a summary, key takeaways, practices for coding agents, a full transcript and cited sources (talks link to exact timestamps). Content is CC BY 4.0.

## Lessons

### Module 1: The idea

- [Lesson 1: Software needs decisions, not essays](https://learn-system-one.pages.dev/lessons/software-needs-decisions.md): Language models write beautifully, but most software needs a decision it can branch on. Here is the prompt-and-parse problem, the new shape of a decision model (state and typed questions in, typed answers and probabilities out), and how a whole category of these models appeared in three weeks.
- [Lesson 2: System One and System Two](https://learn-system-one.pages.dev/lessons/system-one-and-system-two.md): The name comes from Daniel Kahneman: fast, automatic judgment versus slow, effortful reasoning. Here is what the metaphor gets right about decision models, where it breaks, and the rule of thumb it gives you: if an expert could answer in a few seconds, it is a System One question.
- [Lesson 3: How a decision model works](https://learn-system-one.pages.dev/lessons/how-decision-models-work.md): A language model writes its answer one token at a time. A decision model reads your state once and scores every allowed answer in a single pass. Here is the core trick in a few lines of Python, what production models add on top (trained readout heads, shared state, calibration training), and why typed output is not the same as a correct answer.
- [Lesson 4: Jev, Decisions, Clef, Strands: the landscape](https://learn-system-one.pages.dev/lessons/decision-models-compared.md): Four decision models you can use today, one request shape they almost share, and what they cost. Then what we measured ourselves: accuracy on a public benchmark and on fresh news published after every model's release, latency from a real server, and why you should trust your own evals over anyone's leaderboard.

### Module 2: Asking good questions

- [Lesson 5: Noul: the probability of yes](https://learn-system-one.pages.dev/lessons/noul-yes-no-questions.md): The simplest decision is yes or no. A Noul returns the probability that the answer is yes, and that single number carries both the answer and the doubt. Here is how to read it, where to put thresholds, why 0.5 never means medium, and how to phrase yes/no questions the model can't misread.
- [Lesson 6: Choice: one answer from a set](https://learn-system-one.pages.dev/lessons/choice-questions.md): A Choice picks one option from a set you define and returns a probability for every option. Here is how to read the distribution and its confidence, why a none-of-the-above option matters, how contrastive descriptions sharpen close calls, and what reversing the option order did to 3,080 real answers.
- [Lesson 7: Score: a position on a scale you describe](https://learn-system-one.pages.dev/lessons/score-questions.md): A Score places the state on ordered levels you describe and returns a position that can fall between them. Here is how the number is computed, why levels must describe situations rather than degrees, why one Score should measure one thing, and how an extra top level separates the dangerous cases.
- [Lesson 8: State: give it what it needs](https://learn-system-one.pages.dev/lessons/designing-state.md): The state is everything the model sees besides your questions. Here is how to shape it (named fields, paths the questions point at, the record next to the policy), what we measured when we buried the one thing that mattered, and the limits on length, language and context you should design around.

### Module 3: Probabilities you can act on

- [Lesson 9: Calibration: does 0.9 mean 90%?](https://learn-system-one.pages.dev/lessons/calibration.md): A probability is only useful if it means what it says. Here is what calibration is, how to see it in a reliability diagram, why chat-trained models drift into overconfidence, what independent tests and our own measurements found for decision models, and how a few hundred labels let you measure and fix it.
- [Lesson 10: Confidence and thresholds](https://learn-system-one.pages.dev/lessons/confidence-thresholds.md): The answer tells you what; the probability tells you whether to act. Here is the three-path pattern (act, confirm, escalate), why thresholds should differ by action, a one-line formula that turns the cost of mistakes into a threshold, and the one thing thresholds cannot protect you from.
- [Lesson 11: Automate the sure ones](https://learn-system-one.pages.dev/lessons/selective-automation.md): You don't need a model that is right about everything. You need to know which answers you can trust. Here is the coverage curve that answers 'how much can we automate at 95%?', how to pick thresholds from a few hundred labels, what confidence-ordered review saves on our own data, and how to answer less precisely instead of wrongly.

### Module 4: Composing judgments

- [Lesson 12: Decompose: many small questions](https://learn-system-one.pages.dev/lessons/decompose-judgments.md): TypeSafe calls it the most important concept in its guide: break one broad judgment into narrow questions and combine the answers in code. Here is what decomposition buys you (reasons, control, per-signal thresholds), why decision science has recommended it for seventy years, and the case where it made things worse.
- [Lesson 13: Ask everything at once](https://learn-system-one.pages.dev/lessons/speculative-fan-out.md): Questions about the same state belong in one request: they are evaluated in parallel and independently, so extra questions add a few tokens and almost no time. Here is what we measured with 13 questions, why you should ask speculative questions you might not need, and the few cases where a second request is genuinely warranted.
- [Lesson 14: Route and cascade](https://learn-system-one.pages.dev/lessons/routing-and-cascades.md): One fast decision in front of expensive things. Here is intent routing (code, a specialist model, or a person), the cascade (cheap model first, escalate only the uncertain), how to tune it so it matches a strong model at a fraction of the cost, and the prompt-cache gotcha that turned one developer's model router into a money loser.
- [Lesson 15: Select, don't generate](https://learn-system-one.pages.dev/lessons/select-dont-generate.md): A decision model can't write a value, but most extraction is really selection: the value is already in the text. Let code find the candidates and let the model pick one, then copy it verbatim. Here is that pattern for phone numbers, dates, search reranking, line-level search and deep taxonomies.
- [Lesson 16: Verify and guard](https://learn-system-one.pages.dev/lessons/verification-and-guardrails.md): Decision models are cheap and fast enough to check everything: every citation an LLM writes, every message going in and out, every extracted field. Here is a citation check, a one-request guardrail screen, why two independent checks beat one, and why none of this is a security boundary.

### Module 5: Real systems

- [Lesson 17: Decision models inside agents](https://learn-system-one.pages.dev/lessons/decision-models-in-agents.md): Agents make dozens of small decisions per task: which tool, which skill, are these arguments grounded, is it time to ask the user, is the task done. Here is where a fast decision model fits in an agent loop, a gate that stops an agent from guessing, what skill selection gained in TypeSafe's test, and real-time control at ten decisions a second.
- [Lesson 18: Judgments as data](https://learn-system-one.pages.dev/lessons/judgments-as-data.md): At a fraction of a cent per answer, judgments stop being calls you make and become data you keep: columns you can sort, filter, weight and train on. Here is a study that coded half a million police narratives for $25, a loop that turned tasting notes into features for a classic model, and why cheaper judgment will mean much more of it.

### Module 6: Shipping it

- [Lesson 19: Know the jagged edges](https://learn-system-one.pages.dev/lessons/jagged-edges.md): Decision models are strong at semantic judgment and unreliable at things a calculator does. Here are the failure modes we reproduced (counting, arithmetic, dates), TypeSafe's own list of nine, the cases where the model is wrong without sounding unsure, and the rule that fixes most of them: reason in code, decide with the model.
- [Lesson 20: Evaluate on your own data](https://learn-system-one.pages.dev/lessons/evaluate-on-your-data.md): The only benchmark that matters is the one you ship. Here is the evaluation loop for decision models: real inputs, careful labels, per-question results, enough examples to trust a difference, and a regression test that pins the model version, with what we measured when Jev went from 1.13 to 1.14.
- [Lesson 21: Decide responsibly](https://learn-system-one.pages.dev/lessons/responsible-decisions.md): A decision model makes the same call the same way, millions of times. That removes noise, and it makes any bias systematic. Here is how to audit decisions about people (including the name-swap test we ran), where the reasons come from when the model gives none, and what a responsible deployment logs, samples and leaves to a person.
- [Lesson 22: Capstone: a support triage pipeline](https://learn-system-one.pages.dev/lessons/capstone-support-triage.md): The whole course in one pipeline you can run: code prepares the state, one request asks fifteen questions, and code routes every ticket to a refund, a reset link, an LLM, a log, a person or a block. Here is how we built it, the three bad questions our first run exposed, what held-out tickets showed, and why twelve out of twelve still proves less than it sounds.

## For agents

- [Install as agent skills](https://learn-system-one.pages.dev/agents/): `npx skills add Haple/system-one`
- [Full course in one file](https://learn-system-one.pages.dev/llms-full.txt)
- [Structured course data](https://learn-system-one.pages.dev/api/course.json)

## Optional

- [What we measured](https://learn-system-one.pages.dev/measured/): original accuracy, calibration, latency and cost results
- [All sources](https://learn-system-one.pages.dev/sources/): 199 docs, talks, papers and articles
- [Source code](https://github.com/Haple/system-one)