jev·agent

← All use cases

Use case · Choice

LLM model routing with Jev

Put a Choice question in front of your model cascade so a cheap fast model handles the easy majority and only genuinely hard requests reach a frontier model.

The problem

Model cascades only pay off if the router is cheaper and faster than the models it routes to. Using an LLM as the router eats most of the savings — you have added a full round-trip to save a full round-trip.

How Jev handles it

Jev classifies difficulty or task type in a fraction of the time, so the router cost rounds to zero against the model call it selects. This is arguably the single most natural fit for a System One model.

python
response = client.system_one(
    model="jev-latest",
    state=user_prompt,
    questions={
        "tier": Choice(
            instructions="How much capability does answering this require?",
            criteria={
                "trivial":         "Lookup, greeting, or one-line factual answer",
                "standard":        "Ordinary request, no multi-step reasoning",
                "complex":         "Multi-step or domain-specific work",
                "needs_reasoning": "Requires planning, math, or careful analysis",
            },
        ),
    },
)

MODEL = {
    "trivial":         "small-fast-model",
    "standard":        "mid-tier-model",
    "complex":         "frontier-model",
    "needs_reasoning": "frontier-reasoning-model",
}[response.answers["tier"].choice]

Does it actually work?

Independent evidence

An independent test found swapping an LLM classifier for Jev's Choice primitive was faster than either original configuration and significantly cheaper — though it used one sample per tier, so treat it as a directional result, not an accuracy study.

DevelopersIO — replacing model routing with Jev

Notes from the field