Decisions API vs Jev: thirteen decision models on the same items

What the Decisions API is

OpenAI's Decisions API is POST /v1/decisions: you send an input (text, or user messages with images) and an array of questions, and get typed answers back instead of text. There are three question types — predicate (the probability a statement is true), choice (one of up to 255 options, with a probability for each) and score (a level on an ordered rubric) — and a question the model declines comes back as a refusal. It runs one model, GPT-6 Luna, at $0.10 per million input tokens with output free.

Jev, launched by TypeSafe on September 15, 2026, does the same job on its System One contract: a state and named choice, score and noul questions (noul is the predicate). Eight other makers serve models on that contract, and OpenRouter carries them all, GPT-6 Luna Decisions included, behind one endpoint.

Every decision model on the Decisions API

Prices are OpenRouter's list prices; none of these models bills output. Clef, Clef Flash and Kev publish their weights. Respan's Span-01 is a different kind of model — it scores whether behaviours you describe occur in a conversation — so it appears only in the yes/no results below.

Accuracy on the same items

Six sets of 100 items come from BTZSC, the zero-shot classification benchmark, sampled with a fixed seed: news topics (4 options), tweet emotions (6), voice-assistant intents (59), bank support intents (77), toxic comments and product-review sentiment as yes/no questions. The seventh is our own 27 support tickets, written with one decisive signal each before any model ran. Every item was one question, sent to every model in the same words.

“News + tickets” averages the two choice sets every model accepted. Solar Decide and Tev1 refused the two large sets: Solar takes at most 26 options and Tev1 at most 20. With 100 items a set, models a few points apart are not reliably different: read tiers, not ranks. Tweet emotions are left out of the average because Mercury Decide scored 92.0% there against 56.0%–64.0% for everyone else, which suggests it saw that dataset in training.

When a confident answer is wrong

Confidence is what gates an action: act above a threshold, send the rest to a person. So the useful number is how often an answer at 90% confidence or more was wrong. It ranged from 2.3% for Clef to 13.7% for Solar Decide; Jev was at 11.5% and GPT-6 Luna at 9.2%.

Calibration error is the average gap between stated confidence and accuracy; lower is better. The last column compares our 27 clear tickets with six written to pull two ways: confidence that drops on those is usable.

Speed and cost per decision

Latency was measured from Washington, D.C. through OpenRouter, 20 calls each, Jev included. The two free-tier models are rate-limited upstream and were not timed. Cost is what the gateway billed. Price per token misleads: the same item cost 393 tokens on Jev and 218 on GPT-6 Luna, so at under half the list price Jev billed 76% of Luna's cost per decision. The cheapest paid model for full choice, score and yes/no questions was Decider V1.1 27B.

Three things that break quietly

Images on OpenRouter. OpenAI's format sends an image as an input_image part. OpenRouter's Decisions API does not read that shape — it answers anyway, from the base64 read as text. A plain red square came back “blue” from all three image models we tried; sent as a chat-style image_url part, all three said red.

Option caps. Solar Decide rejects a choice with more than 26 options and Tev1 with more than 20. Neither cap is in the catalogue. Respan answers yes/no questions only and wants text or a conversation, not a JSON object.

Call any of them with the OpenAI SDK

POST https://jev-agent.com/api/v1/decisions takes OpenAI's request and returns OpenAI's response, so the official SDK works with the base URL and key changed — and model can be any of the thirteen. Images are rewritten to the shape that is read, and a model that cannot take a request returns a 400 instead of guessing.

import OpenAI from "openai"; // 7.30 or later

const client = new OpenAI({ baseURL: "https://jev-agent.com/api/v1", apiKey: process.env.JAGENT_KEY });

const d = await client.decisions.create({
  model: "jev-latest", // or "gpt-6-luna", "cloudflare/clef", "liquid/d1" …
  input: "I was charged twice for order 5512. Please refund the duplicate.",
  questions: [
    { type: "predicate", name: "refund", instructions: "Does the customer ask for money back?" },
    { type: "choice", name: "queue", instructions: "Which queue?", choices: [{ value: "billing" }, { value: "technical" }, { value: "sales" }] },
  ],
});

Billing is in credits: one per 1,000 input tokens for Jev and every model priced at or below it, scaled by price above that (GPT-6 Luna 2.4, Clef 5.8). The full list with limits is GET /api/v1/models, the same models answer Jev-shaped requests at /api/v1/systemone, and the playground has a model picker. Keys are on API access.

Which decision model to use

For yes/no gates the spread was small, so pick on latency and price. For dozens of intents, pick a model that takes them (Clef Flash led at 77 options). For thresholds that trigger actions, weigh the confident-and-wrong column above accuracy. Then run your own items: these are public datasets.

Questions