Jev vs a fine-tuned classifier

The trade, line by line

What Jev actually costs per classification

Worth doing this properly, because the two sides of the comparison are usually quoted in different units. Jev charges $0.042 per million input tokens and nothing for output. So a classification whose state is a 500-token support ticket costs:

500 tokens / 1,000,000 × $0.042 = $0.000021 per call
                              × 1,000,000 calls = $21

TypeSafe's own evaluation quotes $0.0004 per decision, which implies a state around twenty times larger — a realistic reminder that your cost is set by how much context you send, not by the number of questions you ask. Asking five questions about one state costs the same as asking one, because the state is ingested once.

Where the trained classifier wins

• One task, lots of labels, stable taxonomy. This is what fine-tuning is for. A small model trained on your own distribution beats a general one on your own distribution.
• Data that cannot leave. Jev is a hosted API with closed weights. There is no VPC option and no air gap, so a residency rule ends the discussion.
• Numeric and structural judgements. TypeSafe documents that jev-1.13 does not count reliably, cannot compare dates as ordered quantities, and is weak on numeric representations such as hex colours. A classifier over engineered features has no such problem.
• Very high volume with a large state. Cost scales with tokens sent. At tens of millions of long documents, self-hosted inference wins on pure economics.
• Latency floors. A local encoder answers in single-digit milliseconds. Jev quotes 70–500ms end to end, and from a machine far from the host the network alone can eat 200ms before the model starts.

Where Jev wins

• You have no labelled data. This is the big one. The cold start that costs a classifier project its first month costs Jev an afternoon of writing option descriptions.
• The taxonomy is still moving. Adding a class is an edit to a string, not a relabelling exercise. Products in their first year change categories constantly.
• You need many judgements, not one. A classifier answers the question it was trained on. One Jev call can carry a dozen unrelated questions about the same state for the price of sending the state once.
• The long tail. Classes with forty examples are where fine-tuning struggles most and where a written description works fine.
• Calibration out of the box. You get a probability per option from a model trained to be calibrated, rather than a softmax you have to post-hoc calibrate yourself.

And where embeddings fit

“Jev vs embeddings” is a category error worth untangling. An embedding model turns text into a vector so you can measure similarity. It does not answer a question. The comparable approach is embeddings plus a k-nearest-neighbour lookup over labelled examples, which is a legitimate and very cheap classifier — but it still needs labelled neighbours, and it can only answer questions that similarity happens to encode.

The practical split: embeddings for retrieval and dedup, Jev for the judgement you make about what you retrieved. They compose rather than compete — see reranking retrieved passages for the pattern.

The hybrid that beats both

The most interesting published result on Jev is not Jev winning anything. In an independent study of 2,000 phishing emails, Jev's raw verdict was statistically worse than Claude Haiku 4.5 (McNemar p < 0.0001). Then the author asked five cheap signal questions in the same call and fitted a plain logistic regression over the resulting probabilities:

That is Jev used as a feature extractor feeding a classical model — the cheapest classifier in the world sitting on top of signals that needed no labelling to produce. You still need labels to fit the regression, but far fewer than to fine-tune an encoder, and the features transfer when the task shifts.

TypeSafe endorses the same shape in its own documentation, pointing at an AutoResearch cookbook for training a downstream classical model on Jev's probabilities. If you are choosing between a decision model and a classifier, the honest third option is to stop choosing.

# Five cheap signals in one call, then a model you own.
answers = client.system_one(
    model="jev-latest",
    state=email_text,
    questions={
        "sender_mismatch": Noul(instructions="Does the display name disagree with the sending domain?"),
        "urgency":         Noul(instructions="Does the message pressure the reader to act immediately?"),
        "credential_ask":  Noul(instructions="Does it ask for a password, code or payment detail?"),
        "link_mismatch":   Noul(instructions="Does any link text disagree with its destination?"),
        "brand_spoof":     Noul(instructions="Does it impersonate a known brand?"),
    },
).answers

features = [answers[k].noul for k in FEATURE_ORDER]
label = logistic_regression.predict_proba([features])[0][1]

How to choose, honestly

There is also a sequence, which is usually the right answer for a new product: run Jev in production, log every state alongside the decision and whatever ground truth arrives later, and you have built a labelled dataset as a side effect. Train the classifier when the volume justifies it, using the data your Jev deployment generated.

Common questions