Four decision models you can use today, one request shape they almost share, and what they cost. Then what we measured ourselves: accuracy on a public benchmark and on fresh news published after every model's release, latency from a real server, and why you should trust your own evals over anyone's leaderboard.
Key takeaways
- Jev (TypeSafe, hosted), OpenAI's Decisions API (hosted), Clef (Cloudflare, open weights) and Strands Decider (AWS, open, runs locally) share one shape: state plus typed questions in, probabilities out.
- Jev, Clef and Strands speak the same System One request format; OpenAI's is the same idea with different field names.
- Pricing is per input token with free output: a thousand decisions typically cost cents.
- Public benchmarks can leak into training data and leaderboards move; vendor latency is model time, not your round trip.
- Choose on your constraints (data residency, images, latency from your region, price, fine-tuning) and then measure on your own data.
Try it with your agent
Paste this into your coding agent:
Write decide.py with one function: decide(state, questions, provider). questions use the System One shape (noul / choice / score). Support: - jev: POST https://api.typesafe.ai/v1/systemone (TYPESAFE_API_KEY) - clef: Cloudflare Workers AI @cf/cloudflare/clef-flash (same body) - openai: POST /v1/decisions (map state->input, noul->predicate, criteria->choices/levels; model gpt-6-luna) Return the same shape for all. Log provider, model id, latency and probabilities to decisions.jsonl. Add a test that runs one question through every provider that has a key set.
Transcript
Intro
You can use a decision model today from at least four places. They share one shape, but they differ in where they run, what they cost and how fast they answer from where you are. Let's lay them out, and then look at what we measured ourselves.
The four
Jev, from TypeSafe, is a hosted API at four point two cents per million input tokens. OpenAI's Decisions API runs on GPT-6 Luna at ten cents. Cloudflare's Clef comes in two sizes on Workers AI, and you can download the weights. And AWS's Strands Decider is a two billion parameter model you run yourself, even on a laptop. All four charge nothing for output.
One shape
And they speak nearly the same language. Jev's request format became a de facto standard: Clef and Strands accept it as is, so switching is a change of endpoint. OpenAI uses the same idea with different names: input instead of state, predicate instead of noul.
What we measured
We ran all 3,080 messages of BANKING77, a public dataset of bank support questions with seventy seven intents, through each model we could reach. Jev 1.13 and 1.14 both got about eighty percent right. A fast LLM asked for its answer as JSON got seventy six. And Clef flash got ninety six percent.
Before you crown a winner, notice something. BANKING77 has been public since 2020, and it's one of the benchmarks Cloudflare reports for Clef. Public test sets can end up in training data. So we built a fresh test: 279 news stories published after every one of these models was released. There, Jev 1.14 got eighty two percent, and the LLM eighty.
Latency is local
Latency tells a similar story. Vendors quote model time, often under a tenth of a second. From our server in Brazil, the full round trip told the truth: Jev answered in a quarter of a second, Clef flash in eight tenths, and the LLM in one point two.
Cents per thousand
And the bill? With seventy seven options in every question, about seventeen hundred tokens each, a thousand decisions cost about seven cents on Jev or on Clef flash. Even the LLM was only about eleven cents. At these prices, the question is never can we afford to ask. It's what else should we ask.
How to choose
So how do you choose? Start from your constraints. If data can't leave your servers, run open weights: Strands Decider, or Clef yourself. If you need images, OpenAI and Clef read them today. Pick the region closest to your code. Compare the price per thousand of your decisions, not per token. And if you'll want to fine tune, Cloudflare offers that for Clef.
And whatever you pick, take TypeSafe's own advice from their essay on benchmarks: run your own private evals, treat public ones with a grain of salt, and don't amplify every number or plot you see. That includes ours.
Quick check
Quick check. Your customer data is not allowed to leave your own servers. Which option fits?
Try it
Try it. Have your agent write one small adapter that can call any of these models with the same questions, and log the model, latency and probabilities, so every later lesson can compare them on your data.
Recap
So: four decision models, one shape, cents per thousand decisions. Public benchmarks are upper bounds, latency is local, and the only score that counts is the one on your data. That's module one. Next, how to ask good questions, starting with the simplest: the Noul.