Every entry links to its published source. The code is quoted from that source and trimmed, never invented, and performance claims stay attributed to whoever made them.
One option out of up to 255 labelled criteria. Returns the winning key plus a probability per option.
score
A position on an ordered 2–10 level scale. Returns a possibly fractional score plus the distribution.
noul
A calibrated yes/no returned as a probability between 0 and 1.
Filter by use case, tool or pattern
Every case carries the source link, the code it published, a prompt template with its required inputs and outputs, and one-click copy on all of it. To run a decision instead of reading one, open the harness demo.
10 of 10 cases
harnessagent control2026-09-17LangChain — Sydney Runkle and Hunter Lovell
Building a Harness with Jev
The reference harness write-up: where a System One decision replaces a chat call inside the agent loop, with model routing and tool gating shipped as LangChain middleware.
model routingtool gatinglangchainmiddlewareagent looppython
—Decisions inside the loop (route, gate, grade) do not need a text-generating model.
—ModelRouterMiddleware picks the cheapest model that can do the task, from the last user message.
—AutoModeMiddleware checks a proposed tool call and blocks it before the runtime executes it.
—Probabilities and confidence stay in agent state, so thresholds are yours to set.
model-router.py
from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
ModelChoice,
ModelRouterMiddleware,
)
router = ModelRouterMiddleware(
choices={
"fast": ModelChoice(
model="openai:luna",
criteria="Direct lookups, extraction, and localized changes.",
),
"powerful": ModelChoice(
model="openai:sol",
criteria="Architecture and high-stakes decisions.",
),
},
instructions="Choose the least costly model that can complete the task.",
)
agent = create_agent("openai:gpt-5.6-luna", middleware=[router])
Routing middleware from the LangChain post: the decision selects the model, the model does the writing.
tool-gate.py
from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import AutoModeMiddleware
guardrail = AutoModeMiddleware(tools=["bash"])
agent = create_agent("openai:gpt-5.6-luna", middleware=[guardrail])
Auto Mode inspects the proposed tool call for risky effects before the tool runs.
prompt template
prompt · langchain-harness
Add a control layer to this agent.
1. List every place the loop asks a language model for a decision: which model to use, whether a tool call is safe, whether an answer is good enough.
2. For each one, write the decision as a typed question: choice over named options, score over an ordered scale, or noul for yes/no.
3. Keep the state you already have in the loop as the input. Do not build a new prompt.
4. Act on the probability: below the threshold you pick, hand the case to a human or to the safer branch.
5. Leave generation exactly where text is written for a person.
Return the middleware or wrapper, the threshold per decision, and what happens when the decision is not confident.
Why this works: The loop's decisions are closed sets the code already knows, so the only thing a chat call adds is prose you have to parse back. Moving them to typed answers keeps the branch in code and makes the threshold explicit instead of implicit in a prompt.
required inputs
· The agent state or last message
· The named options (models, tools, verdicts)
· A confidence threshold per decision
expected outputs
· The selected option plus a probability per option
TypeSafe integration — TypeSafeClassifier as a Runnable
The supported Python surface: a classifier exposed as a LangChain Runnable, so decisions invoke, batch and compose like any other step, with traces and token usage in LangSmith.
langchainpythonbatchinglangsmithclassification
—State and named questions are passed together on each invocation.
—Questions sharing one state are evaluated in parallel in a single request.
—Accepts strings, structured JSON and LangChain message objects.
install.sh
pip install langchain-typesafe
export TYPESAFE_API_KEY=...
# optional: point at a compatible gateway
# export TYPESAFE_BASE_URL=https://gateway.example.com
classifier.py
from langchain_typesafe import Choice, Noul, Score, TypeSafeClassifier
classifier = TypeSafeClassifier()
result = classifier.invoke(
{
"state": "Stripe has failed to connect for three days. Help ASAP.",
"questions": {
"department": Choice(
instructions="Which team should handle this?",
criteria={
"billing": "charges, invoices and refunds",
"technical": "bugs, outages and integration failures",
"account": "login, permissions and profile changes",
},
),
"severity": Score(
instructions="How severe is this for the customer?",
criteria=[
"cosmetic or informational",
"degraded, workaround exists",
"blocking, no workaround",
"blocking with financial or data loss",
],
),
"escalate": Noul(instructions="Escalate to a human now?"),
},
}
)
Three typed answers in one round trip instead of three prompt-and-parse calls.
prompt template
prompt · langchain-classifier
Replace this prompt-and-parse classification step with one typed call.
Current code: <paste the call-site>
Do this:
1. Group every value the step extracts into one question map under a single state.
2. Give each question a type: choice with named criteria, score with an ordered scale, noul for yes/no.
3. Delete the output parsing, the retry on malformed JSON and the "respond only with" instructions.
4. Keep the probabilities in the return value so the caller can threshold.
Return the new call and the fields the caller now reads.
Why this works: Several extractions from the same text are several questions about one state, not several prompts. One request answers them in parallel and returns typed values, which removes the parser and the malformed-output retry entirely.
integrationframework integrationVercel knowledge base
Classify, route and score with AI SDK
TypeScript path through AI Gateway: the experimental evaluate API returns typed answers, and the app branches on confidence instead of parsing prose.
ai sdktypescriptvercelconfidencegateway
—Confidence and the selected option's probability are separate metrics — threshold on the one you mean.
—Low confidence routes to review or to a generative fallback, it does not silently pass.
—Zero data retention is a per-request gateway option.
route-ticket.ts
import { experimental_evaluate as evaluate } from 'ai';
const result = await evaluate({
model: 'typesafe-ai/jev',
state: ticket,
questions: {
department: {
type: 'choice',
instructions: 'Which team should handle this ticket?',
criteria: {
billing: 'Charges, invoices, and refunds',
technical: 'Bugs, outages, and integration failures',
account: 'Login, permissions, and profile changes',
other: 'Anything that does not fit the other teams',
},
},
severity: {
type: 'score',
instructions: 'How severe is the issue for the customer?',
criteria: [
'Cosmetic or informational',
'Degraded, but a workaround exists',
'Blocking with no workaround',
'Blocking and causing financial or data loss',
],
},
requestsRefund: {
type: 'boolean',
instructions: 'Is the customer asking for money back?',
},
},
providerOptions: { gateway: { zeroDataRetention: true } },
});
const { department, severity, requestsRefund } = result.answers;
const confidence = result.providerMetadata?.typesafe?.confidence;
Quoted from the Vercel guide; the same shape works against the provider API directly.
prompt template
prompt · ai-sdk-evaluate
Wire a typed decision into this TypeScript path and branch on confidence.
1. Call the evaluation API with the state the request already carries.
2. Read the answer and the confidence separately. Do not treat the winning probability as confidence.
3. Accept the answer above the threshold. Below it, fall back to a generative model or to review, and record which one decided.
4. Bound the call with a timeout and at most one retry on a transient failure.
5. Return the decision, the deciding path and the timings.
Return the handler and the test that mocks the provider instead of calling it.
Why this works: A typed answer is only trustworthy above a threshold, and the threshold is a product decision, not a model setting. Recording which path decided is what makes the behaviour reviewable later.
templateclassification and routingVercel templates
Jev x AI SDK form router (deployable template)
A running application, not a snippet: three forms routed by context, with a generative model as the fallback when the typed decision is not confident enough.
routingformsfallbackzodnext.jstypescript
—Zod validates the submission before any model sees it.
—The typed choice is accepted only at 95% confidence or above.
—On low confidence, invalid output or provider failure, a generative model decides instead — and its choice is final.
—Bounded timeouts, one transient retry, and tests that mock providers rather than calling them.
decision-flow.txt
1. Zod validates the submission against the declared fields.
2. Jev evaluates the whole submission and selects an allowed team/specialty pair.
3. Accept the choice when confidence >= 0.95 (unrounded).
4. Otherwise, or on failure, a generative model re-evaluates the same criteria and decides.
5. The response carries destination, deciding model, timings and the routing stats.
The template's documented control flow — the part worth copying into your own harness.
prompt template
prompt · vercel-form-router
Route this submission to one of my known destinations.
Destinations and what belongs in each: <list them>
Rules:
1. Validate the submission with a schema before any model call.
2. Ask one typed choice question over the destinations, with the submission as state.
3. Accept the answer only at or above <threshold> confidence, unrounded.
4. Below the threshold, or on provider failure, let a generative model decide with the same criteria, and mark that decision as the fallback.
5. Never let the model invent a destination that is not on the list.
Return the route handler, the schema and the response shape including which path decided.
Why this works: Routing is a closed set, so the only failure that matters is an unknown destination. Validating first and constraining the answer to the declared list means a wrong decision is still a routable one.
Harness tutorial focused on the control layer: gating dangerous tool calls, selecting from a large tool catalogue in one request, and detecting loops from agent state.
—The strongest role is the control layer, not the task itself.
—Bounded semantic judgment goes to the decision model; state, arithmetic, policy and side effects stay in code.
—Humans take the ambiguous and the high-risk cases.
—A harness must stay honest about what it cannot see.
division-of-labour.txt
A generative model drafts, plans, or explains.
Jev supplies bounded semantic judgments.
Code handles state, arithmetic, policy, permissions, and side effects.
Humans take the ambiguous or high-risk cases.
— Anthony Maio, quoted by Learn Jev
prompt template
prompt · learnjev-harness
Gate the dangerous tool calls in this agent.
1. List the tools whose effects are hard to undo: writes, deletes, shell, payments, outbound messages.
2. Before the runtime executes one, ask a typed question about the proposed call: is this reversible, is it in scope, does it touch production.
3. Block or ask a human above the risk threshold. Allow below it. Log the answer either way.
4. Keep permissions and policy checks in code. The decision is about meaning, not about authorisation.
Return the gate, the questions and what the agent sees when a call is blocked.
Why this works: The risky part of an agent is the moment before a side effect, and that moment needs a fast verdict on a bounded question. Keeping authorisation in code means a confident wrong answer still cannot exceed the agent's permissions.
Each field of an output type becomes one question, so a typed model extracts several values in one request — and swapping the model name runs the same agent on an LLM for comparison.
pydanticpythonschemaextractioncomparison
—Output type first: the schema is the question set.
—Same agent, two models, so the comparison is a one-line change.
Turn this output schema into a decision call.
Schema: <paste the model or type>
1. Map each field to a question: enum field to choice, ordered rating to score, boolean to noul.
2. Free-text fields stay with a generative model. Say which fields those are.
3. Run the same agent twice, once on each model, over the same inputs, and show the disagreements.
4. Do not report savings. Report the disagreement count and where the two differ.
Return the agent definition and the disagreement table.
Why this works: A schema already declares which fields are closed sets, so it is a complete question set for everything except free text. Comparing both models over the same inputs shows disagreement, which is evidence you can inspect, unlike a savings estimate.
One endpoint, one shape: a state plus a map of typed questions, answered in parallel with probabilities and confidence attached.
apihttpcurlquestionsprobabilities
—Pin a model version when your thresholds are tuned.
—State can be a string, an object or an array of text.
—Every answer carries a distribution, so escalation is a threshold, not a guess.
first-call.sh
curl -X POST https://api.typesafe.ai/v1/systemone \
-H "Authorization: Bearer $TYPESAFE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "jev-latest",
"state": "Customer: I was charged twice and I am furious.",
"questions": {
"topic": {
"type": "choice",
"instructions": "What is the issue about?",
"criteria": { "billing": "money problems", "bug": "broken product" }
},
"urgent": {
"type": "noul",
"instructions": "Escalate to a human now?"
}
}
}'
The request body every integration on this page wraps.
prompt template
prompt · typesafe-api
Write the smallest possible decision call for my case.
Context I already have in code: <describe the state>
The decision I need: <describe it in one sentence>
1. Put the context in state, unchanged. Do not write a prompt around it.
2. Express the decision as exactly one question of the right type.
3. Show how to read the answer and its distribution.
4. Tell me which threshold to start from and how to revisit it with real traffic.
Return the request body and the code that reads the response.
Why this works: The request has no prompt to tune: the quality comes from the state you pass and the criteria you name. Starting with one question makes the threshold observable before the question set grows.
Reported as an evaluation model with zero output tokens and input-only pricing, reachable through a gateway without a direct provider account.
gatewaypricingobservabilitytypescript
—Model id on the gateway: typesafe-ai/jev.
—Evaluation calls appear in gateway logs and count toward budgets like any other call.
minimal.ts
import { experimental_evaluate as evaluate } from 'ai';
const result = await evaluate({
model: 'typesafe-ai/jev',
state: 'The support agent issued a full refund to the customer.',
questions: {
refunded: { type: 'boolean', instructions: 'Was a refund issued?' },
},
});
prompt template
prompt · gateway-availability
Route my decision calls through the gateway I already use.
1. Use the gateway model id instead of a direct provider account.
2. Keep the key server-side. Never call the decision endpoint from the browser.
3. Confirm the call appears in the gateway logs with its input token count.
4. Handle the documented statuses: rate limit with backoff, credit and policy errors as a pause, denials as terminal.
Return the server-side client and the error handling.
Why this works: Running decisions through the same gateway as generation puts both in one log and one budget, so the volume is visible from day one instead of arriving as a surprise line item.
The rule of thumb that turned into a term: an LLM writes, a System One model decides, code acts — with the reminder that the call is the easy part.
methodcall-site testdivision of labourreading
—If it creates text, an LLM does it. If it picks, scores or answers yes/no, the decision model does it. If it follows an exact rule, code does it.
—The work is the state you send and the confidence threshold you act on.
—Keep it out of arithmetic, writing and irreversible execution.
test-for-each-call-site.txt
Creates text -> generative model
Picks / scores / yes-no -> System One decision
Follows an exact rule -> code
Ambiguous or high risk -> human
prompt template
prompt · jev-engineering
Apply the call-site test to this file.
For each AI call you find, answer in one line each:
- does it create text a person reads?
- does it pick, score or answer yes/no?
- does it follow a rule that code could enforce exactly?
- is it ambiguous or high risk?
Then say which of the four it is, and what in the code told you.
If the output shape is not visible, say unknown. Do not guess and do not estimate savings.
Why this works: The four-way test is decidable from the code itself, so the answer is checkable by a second reader. Forcing an explicit unknown stops a guess from entering the report as a finding.
required inputs
· The file or call-site
· The surrounding code that consumes the output
expected outputs
· One of four verdicts per call-site
· The evidence behind it
· An explicit unknown where the shape is invisible