Jev, the model that answers in probabilities: what a 150 ms judgment changes in an agent harness

By Rogier Muller09.16.26

Jev is a new model from TypeSafe AI, the company founded by Diogo Almeida, one of the authors of the InstructGPT work behind ChatGPT. It launched into early access on 15 September 2026. Jev does not write text. You give it a piece of state and a list of typed questions, and it returns a choice, a score, or a yes/no probability for each question, with a confidence value, in roughly 70 to 500 milliseconds.

The problem it addresses is one every agent harness has: most of the calls in a workflow are not "write this", they are "which one", "how bad", and "is this true". Teams run those through a full language model and pay for tokens, latency, and a parsed string they then have to trust. Jev's claim is that a model built only for the decision is two orders of magnitude cheaper and faster, and that it tells you how sure it is.

Our takeaway after reading the launch material, the docs, and the only independent test published so far: the architecture is worth understanding now, the vendor numbers are plausible but self-reported, and accuracy sits below frontier models on the vendor's own benchmark. It belongs in the harness as a gate, a router, and a continuous checker, not as a replacement for the model doing the work.

What Jev does

The API takes one state (a ticket, a diff, a log line, a JSON blob) and a map of questions. Each question has one of three types. choice picks one option from a list you define and returns a probability for every option. score places the state on a rubric you define and returns a weighted position on that scale. noul answers a statement with a probability between 0 and 1. All questions are evaluated in parallel against the same state, so adding a question barely changes the response time.

A shortened version of the quickstart request and response from TypeSafe's docs:

{
  "state": "I've been trying to connect my Stripe account for 3 days. I'm losing sales.",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this",
      "criteria": {
        "billing": "Payments",
        "technical": "Integration bugs",
        "sales": "Pricing"
      }
    },
    "is_urgent": {
      "type": "noul",
      "instructions": "The message conveys urgency"
    }
  }
}

{
  "answers": {
    "department": {
      "choice": "technical",
      "probabilities": { "billing": 0.159, "technical": 0.84, "sales": 0.001 },
      "confidence": 0.596
    },
    "is_urgent": { "noul": 0.999 }
  }
}

The output shape is fixed in advance, so the model cannot return an option that is not in the list or a value of the wrong type. TypeSafe calls this "zero hallucination". Read that narrowly: it means the answer is always schema-valid, not that the answer is always right.

The example that circulated on X puts it well. Ask an LLM "you are a model router, use fast mode for simple UI edits and the full agent for complex work" and you get a string, "route": "fast_mode". Ask Jev the same question as a choice and you get fast_mode: 0.97, full_agent: 0.03. The second answer is something code can branch on, and something you can threshold.

The numbers, and who measured them

The speed and cost figures come from TypeSafe, as does the accuracy-versus-cost chart at the top of this article. End-to-end response time is 70 to 500 ms. Input is priced at $0.042 per million tokens, which the company writes as $42 per billion, and output tokens are free because nothing is decoded. On its own four production-style workflows it reports gains of up to 193.6x in speed and 444.6x in cost against frontier models, and it describes those as the higher end.

The accuracy figures also come from TypeSafe, and they are the more interesting table. Across 711 cases Jev scored 67.8% against 74.1% for the best comparator. On security incident triage it was 61.7% against 66.2% for Opus 5. On invoice processing it was 61.8% against 79.1%. On customer service it was 76.0% against 78.3%. Reference labels were produced by averaging other models rather than by human ground truth, and the workflows were written by TypeSafe's own team, which the company notes as a possible source of bias.

One independent test exists at the time of writing. Mike Taylor, Head of Evals at Every, ran 21 writing-quality questions across 37 documents and got 777 judgments back in under 0.7 seconds for about a quarter of a cent. In a second experiment against 12 passages with planted defects, Jev caught six of seven; Claude Fable 5.1 caught all seven, took 8.83 seconds per passage against Jev's 0.35, and cost roughly 580 times more. Across 11 experiments and 1,709 judgments his total spend was under one cent. His conclusion was that it is fast and cheap enough to check work continuously instead of after the fact, provided the accuracy is enough for the task, which you have to verify yourself.

What has not been published: the architecture, the parameter count, calibration curves, or any third-party evaluation of the calibration claim on messy real-world input. Training uses a method TypeSafe calls Reinforcement Learning for Calibrated Decisions (RLCD), which optimises for the stated probability matching the observed hit rate. That is the part that matters most for a harness, and it is the part with the least outside evidence.

Why this landed with harness builders

Three things happened at once. The founder's credibility, two years in stealth, and a launch thread that asked why superhuman chat models have not led to AGI gave the release a story. The router example gave developers a picture they recognised from their own code. And the pricing made the "judge every step" idea, which everyone agrees with in principle, affordable in practice.

There is a quieter reason. Coding agents already spend most of their calls on judgments: is this diff in scope, did the test actually run, is this file relevant, should a human look at this. Those calls are currently made by the same expensive model that writes the code, or by nothing at all. A cheap model that only makes decisions, and reports its uncertainty, is a good fit for the control layer of an agent, which is the layer we spend most of our training time on.

Where it fits in a harness

TypeSafe's own docs describe the pattern we would start with, confidence-gated routing. The answer tells you what; the confidence tells you whether to act on it. Their recommended shape:

Below a floor, around 0.6, route to a person.
Low-stakes actions proceed at the floor.
High-stakes actions need a higher bar, around 0.85, or a confirmation step in between.

Applied to a coding-agent harness, the candidate decisions look like this:

Decision in the harness	Question type	What the probability lets you do
Is this change inside the agreed scope?	`noul`	Block the PR below a threshold instead of asking a reviewer to notice
Which reviewer or model handles this task?	`choice`	Route on the distribution, escalate when it is flat
How risky is this diff?	`score`	Require evidence proportional to the score
Did the agent's claim match the log?	`noul`	Reject "tests pass" when the transcript disagrees
Is this retrieved file relevant?	`noul`	Filter context at 100 ms per file instead of per token

The last two are the ones we find most useful. A check that costs a fraction of a cent and returns in under half a second can run on every tool call, not just at the end. That changes review from a single gate to a continuous signal, which is what our own checklist for reviewable AI-generated changes asks for.

What it does not change

Jev cannot write. Anything that needs prose, code, or an explanation still needs a language model or a template. The company is explicit that the intended architecture is Jev for the decision, ordinary code for the policy, and a larger model for the words.

Accuracy is mid-tier on the vendor's own numbers. A 17-point gap on invoice processing is not a rounding error; on a task where a wrong decision is expensive, the cheap decision is not cheap.

The context budget is about 32,000 tokens and a choice question is capped at 255 options. Image input is not supported yet.

Calibration is a claim until someone outside the company measures it. A probability that drifts when your policy changes, or when input becomes adversarial, is worse than no probability, because code will act on it. Anthony Maio's analysis makes this point well: constraining the output shape does not prevent a confident wrong answer.

Access is waitlisted. Reports put TypeSafe's funding at around $40 million led by DCVC; we have not verified that figure with the company.

How we would pilot it

Follow the same method we use for measuring an AI workflow before scaling it. Pick one judgment your harness makes repeatedly, ideally one you currently make with a frontier model or with a regex. Log three things for every call over a week: what the current method decided, what Jev decided, and Jev's confidence.

Then answer three questions from the log. How often do the two agree, and who was right when they disagree, checked by a person on a sample. Do the confidence buckets mean anything: of the calls Jev marked above 0.9, how many were right, and of those below 0.6, how many. And what threshold per action would you have needed to never auto-act on a wrong answer.