Jev Choice: pick one option, with a probability for each

Independent and unofficial. Jev is a model by TypeSafe AI; Jagent is not affiliated with or endorsed by them. Reference: TypeSafe's Choice docs. Our figures: measured 2026-11-12.

Measured: one task, fourteen calls

To see how a Choice behaves rather than how it is described, we gave it a single job — filing expense lines into five categories — and changed one thing at a time. Every call asked for jev-latest on TypeSafe's own endpoint and was answered by jev-1.14.0. Each cost under $0.00003 at $0.042 per 1M input tokens. Latency is the full round trip from our machine in Asia, so a server close to the API will see less. The raw requests and responses are in data/measured-choice-2026-11-12.json.

The easy case is boringly certain. A GitHub seat renewal went to software at 1.00, confidence 1.00, and did so again on a second call. This is the case you automate without a second look.

Our “ambiguous” case was not. We expected an O'Reilly team subscription to split between software and training. It went to training at 0.99, because the training description says “courses, books” and the line says the same. The descriptions decided it, not the brand. A hotel minibar bill on a sales trip did split: meals 0.73, travel 0.27, confidence 0.66. That is a real disagreement a finance policy has to settle, and the number tells you so.

Option descriptions do most of the work

Same line — a coworking day pass in Lisbon while visiting a client — and the same five option names. With one-word descriptions (“Travel”, “Office”) the answer was a coin toss: office 0.55, travel 0.45, confidence 0.44. Then we rewrote two descriptions to draw the line: travel is getting to and staying somewhere, not where you work once there; office is a place to work, at home base or on the road. The answer moved to office at 1.00, confidence 1.00.

The extra words cost 74 tokens, about a ten-millionth of a dollar. Option names are labels for your code; the descriptions are the prompt. When two options keep trading places, say what each one is not for before you reach for a bigger model.

When nothing fits: add a companion Noul

A Choice has no way to say “none of these”. A company donation to a food bank fits none of the five categories, and Jev still filed it under office at 0.67 (and 0.65 on a repeat). Confidence 0.59 looks middling, not alarming — a gate at 0.5 would have let it through.

Adding one yes-or-no question to the same request fixed it. “This expense clearly belongs to one of these categories” came back at 0.06 for the donation and 0.95 for the GitHub line. The Choice answers were unchanged in kind, the call stayed one call, and the cost rose by 53 tokens. Read the Choice only when the Noul is high; how the Noul itself behaves is on the Jev Noul page.

fits = answers["fits"].noul
category = answers["category"]
if fits < 0.3:
    send_to_review(line, reason="no category fits")
elif category.confidence < 0.5:
    send_to_review(line, reason=f"split: {category.probabilities}")
else:
    book(line, category.choice)

Gotchas we hit

Confidence is not the top probability. For n options with a top probability p, Jev reports (n·p − 1)/(n − 1). The minibar line had p = 0.73 over five options: (5 × 0.73 − 1)/4 = 0.66, which is what came back. The same p over two options gives 0.46. So a threshold tuned on a five-way question means something else on a two-way one; set it per question.

Close A/B calls lean toward the second slot. We asked which of two expense notes was clearer, then swapped the order. The same note won both times, but at 0.75 when listed first and 0.89 when listed second: the slot alone was worth 0.14. On a pair closer than ours that is enough to flip the winner. When comparing two versions, ask twice with the order swapped and average.

Ask everything in one request. Category, payment method and a receipt check on a team dinner went in one request of 542 tokens and came back in 355 ms, no slower than a single question. The first attempt at the same request took 732 ms; one slow call in fourteen is network, not the model, so measure p95 rather than one run.

255 options is the ceiling. Long lists work, but past a few dozen it is usually better to ask a coarse Choice and then a fine one inside the winner. The exact error for option 256 is on the rate limits page; picking tools from a long list is on Jev agents.

Small drift between identical calls. The donation line read 0.67, 0.65 and 0.59 in office across three calls (the last with a Noul alongside). Clear cases stayed at 1.0. Thresholds sitting right at a borderline value will flicker.

When a Choice is the wrong question

• Several labels can be true. Probabilities compete and sum to 1, so a ticket that is both a bug and a refund will be split between them. Ask one Noul per label.
• The options are an ordered scale. “Low, medium, high” as a Choice throws away the order. Use a Score, which knows that medium sits between.
• The answer is open-ended. Extracting a merchant name or an amount is generation, not selection; that needs an LLM.
• The answer is somewhere in an image. Finding where a total sits on a scanned receipt is a Box question.

Calling a Choice

The body is the same on both hosts; only the key differs. Field details are in our API reference, and you can write and run your own options in the playground first. With a key from this site:

curl -X POST https://jev-agent.com/api/v1/systemone \
  -H "Authorization: Bearer $JEV_AGENT_KEY" \
  -H "content-type: application/json" \
  -d '{
    "state": "Hotel minibar and room-service breakfast during the Denver sales trip: $64.",
    "questions": {
      "category": {
        "type": "choice",
        "instructions": "Which expense category does this line belong to?",
        "criteria": { "travel": "Flights, trains, taxis and hotels", "meals": "Food and drink" }
      }
    }
  }'

With a TypeSafe key, on their endpoint, naming the model:

curl -X POST https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "content-type: application/json" \
  -d '{ "model": "jev-1.14.0", "state": "…", "questions": { … } }'

Pin the versioned ID once you have tuned thresholds; jev-latest moves when a release ships.

Jev Choice: common questions