The state is everything the model sees besides your questions. Here is how to shape it (named fields, paths the questions point at, the record next to the policy), what we measured when we buried the one thing that mattered, and the limits on length, language and context you should design around.
Key takeaways
- State is the material you would hand an expert panel before asking: the message, the record, the policy.
- Use a JSON object with named fields, and point each question at the field it is about with a backticked path.
- Unrelated text in its own named field barely hurt in our test; asking about one item buried by index among 40 similar ones dropped accuracy from 98% to 72%.
- Name the thing you are asking about: one item per request, or put the record in the question itself.
- Filter in code before you send; long context exists but is slower and less accurate, and English is still the strongest language.
Try it with your agent
Paste this into your coding agent:
Pick one decision we make with a model. Design its state as a JSON object with named fields: only what an expert would need (the message, the record, the relevant policy). Rewrite each question to point at its field with a backticked path. Then run 20 labeled examples three ways: the clean state, the state plus a large unrelated document in its own field, and the item hidden in a list of 20 similar items. Report the accuracy of each.
Transcript
Intro
Every request has two halves: your questions, and the state they're asked about. Most failures we'll see in this course start on the state side. So let's shape it well.
What state is
The TypeSafe docs have a good image for it. Think of the state as the material you'd present to a panel of experts before asking them to make a judgment. The customer's message. Their order. The policy that applies. Everything an expert would need, and nothing they wouldn't.
Shapes of state
State can be a plain string, which is fine for one message. An array works for a sequence, like a conversation. But for most real decisions, use an object, so every part has a name and the relationships stay clear.
Point at the field
Then point each question at the field it's about, with a path in backticks. Does the policy support the refund requested in the ticket, given the order's charges? The model knows exactly which parts to compare.
Bury it, lose it
How much does extra material hurt? We tested sixty banking messages, half of them about refunds, with one Noul: does the ticket ask about a refund? Alone, ninety eight percent right. With six thousand tokens of unrelated handbook in its own field: still ninety eight. But bury the same ticket among forty similar tickets and ask about ticket number seventeen, and it drops to seventy two, missing seventeen of the thirty refund requests.
So the lesson isn't just keep the state short. It's name the thing you're asking about. Don't make the model hunt for item seventeen in a list of lookalikes. Send that item on its own, or put it right in the question.
Data in the question
Questions can carry data too. To check a new resume against candidate records, put each record inside its own question's instructions, next to the same question text. Twenty records, twenty Nouls, one request, and each answer is about exactly one record.
Limits to design around
A few limits to design around. Question IDs aren't sent to the model, so write the full question in the instructions. Jev reads up to sixty four thousand tokens per request, and thirty two thousand for the state plus any one question. Long context can be turned on, but TypeSafe warns it's slower and less accurate. And English is still the strongest language, so test others before you rely on them.
Facts vs judgments
One last habit, from TypeSafe's own guide for coding agents. When you store answers and feed them into later requests, keep what you observed separate from what the model inferred. A field that says refund requested, point nine nine, is a judgment, not a fact. Label it, and check it's still fresh before you act on it.
Quick check
Quick check. You need to know whether ticket seventeen, in a batch of forty, asks for a refund. What's the best state?
Try it
Try it. Have your agent design the state for one decision in your code, and test what happens when you bury it.
Recap
So: state is what you'd hand an expert panel. Use named fields and point questions at them. Name the thing you're asking about, and never make the model hunt for it. Design around the limits, and keep judgments separate from facts. That's module two. Next: probabilities you can act on, starting with calibration.