Free course · 22 animated lessons · open source

Build software that decides

Decision models like TypeSafe's Jev, OpenAI's Decisions API and Cloudflare's Clef answer typed questions with probabilities, in a quarter of a second, for a fraction of a cent. Learn what to ask them, which numbers to trust, and where they break, from 199 sources and 14,318 API calls we ran ourselves.

Start lesson 1 →Let your agent take it

82 min of lessons
199 sources
0 sign-ups

Not a video file: every lesson is animated live by code in your browser.

For developers

Replace fragile prompt-and-parse calls with typed answers your code can branch on, and know which probabilities to trust.

For tech leads

Where decision models fit next to code and LLMs, how to evaluate them on your own data, and where they fail.

For your agent

Install the course as a skill, and your coding agent designs decision-model integrations the way the course teaches.

The course

Your progress is saved in this browser.

Module 1

The idea

Why software needs decisions, and what a decision model is.

01Software needs decisions, not essaysLanguage models write beautifully, but most software needs a decision it can branch on. Here is the prompt-and-parse problem, the new shape of a decision model (state and typed questions in, typed answers and probabilities out), and how a whole category of these models appeared in three weeks.✓4 min 02System One and System TwoThe name comes from Daniel Kahneman: fast, automatic judgment versus slow, effortful reasoning. Here is what the metaphor gets right about decision models, where it breaks, and the rule of thumb it gives you: if an expert could answer in a few seconds, it is a System One question.✓3 min 03How a decision model worksA language model writes its answer one token at a time. A decision model reads your state once and scores every allowed answer in a single pass. Here is the core trick in a few lines of Python, what production models add on top (trained readout heads, shared state, calibration training), and why typed output is not the same as a correct answer.✓4 min 04Jev, Decisions, Clef, Strands: the landscapeFour decision models you can use today, one request shape they almost share, and what they cost. Then what we measured ourselves: accuracy on a public benchmark and on fresh news published after every model's release, latency from a real server, and why you should trust your own evals over anyone's leaderboard.✓4 min

Module 2

Asking good questions

The three primitives, and the state they read.

05Noul: the probability of yesThe simplest decision is yes or no. A Noul returns the probability that the answer is yes, and that single number carries both the answer and the doubt. Here is how to read it, where to put thresholds, why 0.5 never means medium, and how to phrase yes/no questions the model can't misread.✓4 min 06Choice: one answer from a setA Choice picks one option from a set you define and returns a probability for every option. Here is how to read the distribution and its confidence, why a none-of-the-above option matters, how contrastive descriptions sharpen close calls, and what reversing the option order did to 3,080 real answers.✓4 min 07Score: a position on a scale you describeA Score places the state on ordered levels you describe and returns a position that can fall between them. Here is how the number is computed, why levels must describe situations rather than degrees, why one Score should measure one thing, and how an extra top level separates the dangerous cases.✓4 min 08State: give it what it needsThe state is everything the model sees besides your questions. Here is how to shape it (named fields, paths the questions point at, the record next to the policy), what we measured when we buried the one thing that mattered, and the limits on length, language and context you should design around.✓4 min

Module 3

Probabilities you can act on

Calibration, confidence and thresholds.

09Calibration: does 0.9 mean 90%?A probability is only useful if it means what it says. Here is what calibration is, how to see it in a reliability diagram, why chat-trained models drift into overconfidence, what independent tests and our own measurements found for decision models, and how a few hundred labels let you measure and fix it.✓4 min 10Confidence and thresholdsThe answer tells you what; the probability tells you whether to act. Here is the three-path pattern (act, confirm, escalate), why thresholds should differ by action, a one-line formula that turns the cost of mistakes into a threshold, and the one thing thresholds cannot protect you from.✓4 min 11Automate the sure onesYou don't need a model that is right about everything. You need to know which answers you can trust. Here is the coverage curve that answers 'how much can we automate at 95%?', how to pick thresholds from a few hundred labels, what confidence-ordered review saves on our own data, and how to answer less precisely instead of wrongly.✓4 min

Module 4

Composing judgments

Small questions, combined in code.

12Decompose: many small questionsTypeSafe calls it the most important concept in its guide: break one broad judgment into narrow questions and combine the answers in code. Here is what decomposition buys you (reasons, control, per-signal thresholds), why decision science has recommended it for seventy years, and the case where it made things worse.✓4 min 13Ask everything at onceQuestions about the same state belong in one request: they are evaluated in parallel and independently, so extra questions add a few tokens and almost no time. Here is what we measured with 13 questions, why you should ask speculative questions you might not need, and the few cases where a second request is genuinely warranted.✓3 min 14Route and cascadeOne fast decision in front of expensive things. Here is intent routing (code, a specialist model, or a person), the cascade (cheap model first, escalate only the uncertain), how to tune it so it matches a strong model at a fraction of the cost, and the prompt-cache gotcha that turned one developer's model router into a money loser.✓4 min 15Select, don't generateA decision model can't write a value, but most extraction is really selection: the value is already in the text. Let code find the candidates and let the model pick one, then copy it verbatim. Here is that pattern for phone numbers, dates, search reranking, line-level search and deep taxonomies.✓3 min 16Verify and guardDecision models are cheap and fast enough to check everything: every citation an LLM writes, every message going in and out, every extracted field. Here is a citation check, a one-request guardrail screen, why two independent checks beat one, and why none of this is a security boundary.✓4 min

Module 5

Real systems

Decision models inside agents and data pipelines.

17Decision models inside agentsAgents make dozens of small decisions per task: which tool, which skill, are these arguments grounded, is it time to ask the user, is the task done. Here is where a fast decision model fits in an agent loop, a gate that stops an agent from guessing, what skill selection gained in TypeSafe's test, and real-time control at ten decisions a second.✓4 min 18Judgments as dataAt a fraction of a cent per answer, judgments stop being calls you make and become data you keep: columns you can sort, filter, weight and train on. Here is a study that coded half a million police narratives for $25, a loop that turned tasting notes into features for a classic model, and why cheaper judgment will mean much more of it.✓4 min

Module 6

Shipping it

Limits, evaluation, responsibility, and a full build.

19Know the jagged edgesDecision models are strong at semantic judgment and unreliable at things a calculator does. Here are the failure modes we reproduced (counting, arithmetic, dates), TypeSafe's own list of nine, the cases where the model is wrong without sounding unsure, and the rule that fixes most of them: reason in code, decide with the model.✓4 min 20Evaluate on your own dataThe only benchmark that matters is the one you ship. Here is the evaluation loop for decision models: real inputs, careful labels, per-question results, enough examples to trust a difference, and a regression test that pins the model version, with what we measured when Jev went from 1.13 to 1.14.✓4 min 21Decide responsiblyA decision model makes the same call the same way, millions of times. That removes noise, and it makes any bias systematic. Here is how to audit decisions about people (including the name-swap test we ran), where the reasons come from when the model gives none, and what a responsible deployment logs, samples and leaves to a person.✓4 min 22Capstone: a support triage pipelineThe whole course in one pipeline you can run: code prepares the state, one request asks fifteen questions, and code routes every ticket to a refund, a reset link, an LLM, a log, a person or a block. Here is how we built it, the three bad questions our first run exposed, what held-out tickets showed, and why twelve out of twelve still proves less than it sounds.✓5 min

We didn't just read about it

Measured for this course

Vendor numbers are upper bounds, so we ran our own tests from a server in Brazil. A few results:

96.3%right on the half of 3,080 banking messages where Jev said ≥ 0.99 (80.1% overall)

0.27 sfor 13 questions in one request, vs 3.3 s as 13 separate calls

0 of 2,116answers above 0.9 changed when we reversed the 77 options; 36% of those under 0.5 did

Everything we measured →

Your agent can take this course too

One course, two students

The lessons double as agent skills. Install them and Claude Code, Codex, Cursor or any agent that reads skills will design decision-model calls the way the course teaches: typed questions, one request, thresholds from costs, code for the arithmetic.

Agent setup →

$ npx skills add Haple/system-one

# or point any agent at the course
$ curl https://learn-system-one.pages.dev/llms.txt

Built from the people building it

The TypeSafe, OpenAI, Cloudflare and AWS docs, 58 talks and podcasts, 18 papers, and essays and benchmarks from independent testers. See all 199 sources →

Daniel SamDaniel KahnemanStrands Decider READMETypeSafe AIEmil LindforsDesi IvanovaAmir RafeSubasish DasOlivier SibonyBarry ZhangDiogo AlmeidaHamel HusainSimon WillisonShreya ShankarMike RamosIndyDevDanRyan Porter

Not affiliated with or endorsed by any of the people or companies cited. Quotes link to the original source, at the exact moment for talks.

Questions

Is it really free?

Yes. No sign-up, no paywall, no tracking cookies. The content is CC BY 4.0 and the code is MIT on GitHub.

Which decision models does it cover?

TypeSafe's Jev, OpenAI's Decisions API, Cloudflare's Clef and Clef-flash, and AWS's open Strands Decider. They share one request shape, so the lessons apply to all of them. Most examples use Jev, because it's the one we measured most.

Do I need to know machine learning?

No. If you can call an API and write an if statement, you can follow the course. The statistics you need (calibration, thresholds, margins of error) are explained from scratch.

Are these videos?

Sort of. Each lesson is a scene graph rendered live by JavaScript and synced to narration, so it's tiny, crisp at any size, scrubbable and searchable. The narration is an AI voice. Every lesson also has a full transcript and a Markdown version.

Who made this, and how?

It was researched, measured and produced with AI agents. Sources were distilled into quote-checked insights, every number shown on screen comes from a source or from our own runs, and the lessons are animated by code. Read how it was made.

I found a mistake.

Please open an issue. Every claim links to its source, so corrections are easy to check.