Routing Multi-Agent Requests with Jev

In a planner-based multi-agent system, every single user turn starts with the same question: which agent should handle this? In my multi-agent reference architecture, that one decision shapes the whole plan: a lookup against the Acme or Stark agent, a cart operation, or a lookup followed by a cart step when the price isn’t known yet.

Most teams answer it the obvious way: hand it to a frontier LLM with a structured-output prompt. It works. But it also means every request waits on a multi-second, token-billed generation call just to find out where it’s going — and what comes back is still a string you have to parse and hope stays on schema.

I pulled that step out and replaced it with TypeSafe AI’s Jev. Here’s what Jev actually is, how it sits in the planner, what it bought me, and where OpenAI’s answer to it fits if you’d rather wait.

A model that answers questions instead of writing text

Jev is the first thing TypeSafe calls a “System One Model,” released in early access on September 15, 2026. The name is the whole pitch: it doesn’t generate text at all — it answers typed questions about a state you hand it.

You give it two things: a state (a string, or something structured) and a set of typed questions about that state. It answers all of them in one parallel pass. Three question types cover almost everything you’d want to ask:

• Choice — pick one option from a set you define, like which agent should take this request
• Score — place the state somewhere on a scale you define
• Noul — a yes/no judgment, returned as a probability rather than a flat boolean

Every answer comes back with a full probability distribution and a confidence value — TypeSafe trains for this specifically with what they call Reinforcement Learning for Calibrated Decisions. The mental model that stuck for me: Jev is a function call, not a chatbot. The LLM still does your open-ended reasoning and writing. Jev handles the fast, bounded decisions wedged in between.

Where it sits in the planner

The reference architecture is an e-commerce assistant built on LangGraph. A planner node sits in front of a handful of specialized agents — an Acme agent and a Stark agent, each owning one company’s catalog and orders, plus a Cart agent that pulls items together from both. A summarization node runs alongside to keep the context window from bloating.

The planner’s job isn’t just picking one next node — some requests need a short chain of steps. “Add this to my cart” is one step if we already know the price, but it’s a lookup followed by a cart step if we don’t. So the route Jev returns doesn’t map to a single agent, it maps to an ordered plan. Here’s the actual planner node from the repo:

import logging

from typesafe_sdk import Choice, Noul, TypeSafeClient

from agent.models.graph_models import B2BGraphState
from agent.models.llm_models import Plan, PlanType

ROUTES = {
    "cart_known_price": [PlanType.CART_MANAGER],
    "lookup_then_cart": [PlanType.ACME_AGENT, PlanType.STARK_AGENT, PlanType.CART_MANAGER],
    "item_or_order_lookup": [PlanType.ACME_AGENT, PlanType.STARK_AGENT],
}


def planner(state: B2BGraphState):
    with TypeSafeClient() as client:
        result = client.system_one(
            state={
                "conversation": [m.content for m in state["messages"]],
                "new_input": state["input"],
                "cart": state.get("cart", []),
                "items": state.get("items", []),
            },
            questions={
                "route": Choice(
                    instructions=(
                        "Which plan should handle the user's new input? Pick "
                        "cart_known_price only if the user wants a cart operation "
                        "and the item's price is already known from items already "
                        "looked up. Pick lookup_then_cart if the user wants a cart "
                        "operation but the item's price is not yet known. Pick "
                        "item_or_order_lookup for item lookups or order operations."
                    ),
                    criteria={
                        "cart_known_price": "Cart operation; item price already known.",
                        "lookup_then_cart": "Cart operation; item price not yet known.",
                        "item_or_order_lookup": "Item lookup or order operation.",
                    },
                ),
                "place_order": Noul(
                    instructions="Should an order be placed as part of this request?"
                ),
            },
        )

    route = result.choices["route"].choice
    place_order = result.nouls["place_order"].noul > 0.5

    plans = [
        Plan(plan=state["input"], plan_type=plan_type, place_order=place_order)
        for plan_type in ROUTES[route]
    ]

    logging.info(f"Plans determined by planner is: {plans}")
    return {"plans": plans, "items": [], "order_numbers": []}

One Jev call, two questions: route picks which of the three plan shapes applies, and place_orderdecides — independently of which agents get involved — whether this request should actually result in placing an order. ROUTES[route] then expands into the ordered list of plan steps, each one a typed Plan object the rest of the graph already knows how to execute.

The guarantee that matters is in that dict lookup: route can only ever come back as one of the three keys declared in ROUTES, because those are exactly the options Jev was given. There’s no string to sanitize and no KeyError to guard against — the shape of the answer is enforced before it ever reaches application code. This planner keeps it simple and always commits to the top choice; Jev does return a per-choice confidence score if you want to add a clarify-or-escalate branch on top of this later.

Why I think it belongs in the planner

It’s fast

TypeSafe quotes 70–500ms end-to-end, and that tracks with what people are reporting in the wild. For a planner that fires on every single turn, shaving seconds off routing is the difference between an assistant that feels instant and one that feels like it’s thinking it over.

It’s cheap

Jev bills $0.042 per million input tokens, and output is free — TypeSafe claims 40–400x lower cost than a frontier LLM on comparable tasks. Even on the conservative end of that range, routing stops being a line item anyone needs to think twice about.

You get a confidence score you can actually branch on

Every decision comes back with a calibrated probability, which turns routing from a guess into a policy: route automatically above a threshold, ask a clarifying question in the uncertain middle band, and escalate to a full LLM or a human below it. That’s the pattern TypeSafe recommends. The planner above doesn’t need it yet — it always commits to the top choice and lets a downstream agent push back if the plan turns out to be wrong — but the confidence score is sitting right there on result.choices the day I decide I want that extra layer.

No schema drift — even when someone’s trying to break it

An LLM router hands you back a string, and a sufficiently crafted user message can push that string off-schema: a node name that doesn’t exist, stray fields, free text your parser trips over. Jev only ever returns one of the options you declared, so the planner physically cannot receive a route outside the graph.

Worth being precise about what this does and doesn’t guarantee: it guarantees the shape of the answer, not its correctness. TypeSafe’s own jaggedness notes for jev-1.13 are upfront that adversarial content in the state can still steer which valid option it picks. Keep your authorization checks in each node regardless — Jev narrows the attack surface, it doesn’t remove it.

One call, several questions, no extra cost

A single Jev call can ask multiple questions at once, all answered in one parallel pass, and you only pay for the input. The planner above asks “which plan shape?” and “should this place an order?” for the price of reading the message once — and there’s nothing stopping a third or fourth question from riding along in the same call. Do that with an LLM instead, and every extra question means more output tokens or another round trip.

If you’re all-in on OpenAI, the Decisions API might be worth waiting for

Two weeks after Jev shipped, on September 29, 2026, OpenAI announced a Decisions API at DevDay — same underlying pattern, applying GPT-6 Luna to bounded answers instead of open generation. Their DevDay numbers claimed 150ms responses versus 1.6 seconds for Luna doing the same classification through a regular prompt.

It’s limited preview only right now. As of the end of September, OpenAI hadn’t published a request schema, an endpoint, an SDK method, or pricing, and they haven’t said what their confidence scores mean or how they’re calibrated. That’s a meaningfully different place to be than Jev, which you can get a key for today.

My honest take: if your stack is already all-in on OpenAI — one vendor, one bill, one data-processing agreement — it’s a reasonable call to wait for the Decisions API rather than onboard a new provider for this. If you need fast, cheap routing today, Jev is shippable right now. Either way, design your planner against a typed decision interface rather than hardcoding to one vendor’s SDK, so swapping one for the other later is a config change, not a rewrite.

Use the LLM to think, and something else to decide

Routing is a decision, not a conversation, so it shouldn’t cost a generation call. Putting Jev in the planner made every turn in the reference architecture faster and cheaper, and gave the graph a hard guarantee it didn’t have before: it only ever receives a node it actually knows about.

The code is in the multi-agent reference architecture repo. Clone it, swap in your own agents, and measure the routing latency before and after for yourself.