At a fraction of a cent per answer, judgments stop being calls you make and become data you keep: columns you can sort, filter, weight and train on. Here is a study that coded half a million police narratives for $25, a loop that turned tasting notes into features for a classic model, and why cheaper judgment will mean much more of it.
Key takeaways
- Ask once, store the answers, and reuse them: changing a weight or a filter doesn't need another model call.
- Whole corpora are now affordable: a study screened 499,500 police narratives for $25 and coded 195,857 with 27 questions for $30.
- Judgments make great features: questions turned wine notes into 67 numeric columns, and a classic model beat every baseline.
- With labels, let the data choose the questions: an autoresearch loop proposes questions and keeps the ones that cut error.
- Cheaper judgment means more judgment, which is why TypeSafe named Jev after Jevons.
Try it with your agent
Paste this into your coding agent:
Pick one judgment we make about records (tickets, leads, reviews, documents). Create a table that stores the raw answer for each question (probabilities), the model ID and the date, keyed by record. Backfill 1,000 records in batches and report the measured cost per 1,000. Move the weighting and thresholds into a query or function over that table, and show that changing a weight needs no new model calls.
Transcript
Intro
When a judgment costs a fraction of a cent, you stop thinking of it as a call. It becomes data: a column you compute once, store, and use in a hundred ways.
Ask once, reuse
The habit is simple. Store the raw answers: every probability, with the model version and the date. Then your weights, thresholds and filters are just code over a table. Change a weight and the ranking updates instantly, with no new model calls. One tester put it neatly: the judgments are cached, so rewriting the composition rules costs nothing.
Whole corpora
That changes what's affordable. Texas has five million police crash narratives that analysts rarely read, because coding them with a frontier model is too expensive. A research team screened half a million of them with ten questions, for twenty five dollars. Then they coded almost two hundred thousand with twenty seven questions, for thirty dollars, with zero failed calls. The whole five million would cost about seven hundred seventy four.
Notice the two stages. A lean screen with ten presence questions runs over everything. The detailed twenty seven question schema only runs where the screen found something to code. The raw probabilities overstated how common things were, but once recalibrated on human labels, the review budget became a number: how many records a person would still need to read.
Judgments as features
Judgments also make great features for classic machine learning. A TypeSafe cookbook predicts a wine critic's score from the tasting note. Predicting the average gives an error of three point oh nine points. Word counts get two point four seven. Asking the decision model for the score directly, two point one five. But thirty eight narrow questions, fed as features to a CatBoost model, got it down to one point seven seven.
Let the data pick the questions
And nobody wrote those thirty eight questions by hand. An LLM proposed them, the decision model answered them for every row, and CatBoost reported which questions helped and which rows it still got wrong. The next round read that report and proposed better questions. Five rounds took the error from one point eight seven to one point seven seven. Trying a change costs no API calls, because the answers are already stored.
Jevons
Which brings us to the name. TypeSafe named Jev after William Stanley Jevons, who noticed in 1865 that more efficient steam engines made Britain burn more coal, not less. Their bet: every order of magnitude drop in the cost of intelligence unlocks orders of magnitude more uses. Whether that holds depends on demand, but it's the right way to think about judgments as data.
Quick check
Quick check. You scored ten thousand tickets with six questions last week. Today the team wants to change the priority weights. What do you need to do?
Try it
Try it. Have your agent turn one judgment in your product into a stored, reusable table.
Recap
So: judgments are data. Store the raw answers with the model and date, and compute composites in code. Whole corpora are affordable now, so screen first, then code. Use judgments as features, and let the data pick the questions. That's module five. Next, module six: shipping it, starting with the jagged edges.