Jev Score: place text on your rubric, with a probability per level

Independent and unofficial. Jev is a model by TypeSafe AI; Jagent is not affiliated with or endorsed by them. Field definitions are in TypeSafe's Score reference; numbers here are ours, measured 2026-11-12.

How to read a fractional score

The levels are numbered by their place in the array, starting at 0. With our four risk levels the score runs from 0 (Safe) to 3 (Critical). On the pull request above, Jev put 0.32 on level 2 and 0.68 on level 3, so the score is 0.32 × 2 + 0.68 × 3 = 2.68. That is not “68% risky”; it is “mostly Critical, partly High”.

A fractional score has two possible meanings, and the number alone cannot tell them apart. 1.5 can be a clean split between levels 1 and 2, or half the weight on level 0 and half on level 3. Read probabilities before you act on the score: if your decision turns on one level (“block the merge if Critical”), threshold that level's probability, not the mean. To put Scores of different lengths on one scale, divide by the number of levels minus one: 2.68 / 3 = 0.89.

Measured: what moves a Jev Score

Nine calls on 2026-11-12 to TypeSafe's API with jev-latest, which answered as jev-1.14.0 every time. The state is the pull request in the box above. Where a run was repeated, both results are shown; cost is input tokens at $0.042 per million. Raw responses are in data/measured-score-2026-11-12.json.

Repeats differed by up to 0.06 in score, so treat the second decimal as noise. The 851 ms is the first call of the session, connection setup included; warm calls took 361 to 470 ms from our side of the network.

Level wording moved the answer a full level

With the bare labels Low, Medium, High, Critical, Jev settled on High (0.73) and scored 1.79. With levels that say what each one means, the same text scored 2.68 and the peak moved to Critical. Nothing else changed. The descriptive top level names the actual failure, “could lock users out”, and a session-cookie refactor can do exactly that. One-word levels leave the model to guess your threshold; descriptions let you check its reading against your own. They cost 43 more tokens.

Four levels or ten: same position, higher confidence

On ten levels the score was 7.88, which normalises to 0.88 — within 0.02 of the four-level 0.89. The finer rubric did not change where the pull request sits. It did change two other things. Confidence rose to 0.88 with a similar share of weight off the peak, because Score confidence divides by the spread an even distribution would have, and that divisor is larger on ten levels. And the winning level, “Very high: sensitive area with weak testing”, mixes risk with test coverage: long rubrics invite levels that measure two things at once. Ten levels read 66 more tokens.

Reversed levels do not just mirror

We sent the same four descriptions highest-first. If order were only numbering, the score would flip to about 3 − 2.68 = 0.32 with Critical still on top. Instead it came back 0.54 and 0.60, and the peak moved to High both times (0.54, 0.60). The answer itself changed, not just the labels. List levels from the low end to the high end, as TypeSafe specifies, and keep that order when you edit a rubric.

Composite scoring: several Scores, combined in your code

One Score should measure one thing. When the decision depends on several, ask a Score for each in the same request and do the arithmetic yourself, where it can be read and tuned. This is what this site's essay grader does with rubric rows; the rubric generator and single-point rubric tool write the levels. For the pull request, the three questions in the box gave risk 2.71, tests 1.42 and rollback 1.94, in one 546-token call:

LEVELS = {"risk": 4, "tests": 4, "rollback": 3}

def norm(answers, qid):
    return answers[qid].score / (LEVELS[qid] - 1)   # 0..n-1  ->  0..1

def review_need(answers):
    risk = norm(answers, "risk")
    gaps = 0.25 * (1 - norm(answers, "tests")) + 0.15 * (1 - norm(answers, "rollback"))
    return risk * (0.6 + gaps)   # no risk, no review, whatever the tests say

That gives 0.90 × (0.6 + 0.13 + 0.005) ≈ 0.66 for the refactor. The README typo fix scored risk 0.00, rollback 2.00 — and tests 0.03, because a docs change has no tests, and a Score always picks a level even when the question does not apply. A plain weighted sum with the same weights would have handed the typo fix 0.25 for missing tests. Multiplying by risk is the fix here; elsewhere, gate the Score with a Noul such as “this change touches code”.

Gotchas

• Ten levels is the ceiling. On our rate-limits page an eleventh level came back as a 400. Validate the length before sending, especially when a person edits the rubric.
• Lowest first. Index 0 is the first entry. The reversed run shows order is read, not just counted.
• Use legend to map back. It returns your level text under the string keys "0" to "n-1", the same keys as probabilities. Log it beside the answer, so a later rubric edit cannot silently re-label old scores.
• Confidence counts distance. Per TypeSafe's confidence page, weight on a neighbouring level lowers it less than weight three levels away. In our four-level risk runs confidence equalled the peak's probability: the leftover weight sat on one neighbour, and on four levels the divisor is exactly 1. It is not a probability of being right, and it shrinks less on long rubrics.
• Billing is on input. usage also lists 17 output tokens per Score question; the price is $0.042 per 1M input tokens.

When not to use a Score

When the options have no order — department, language, intent — use a Choice; a Score would invent a distance between them. When the question is yes or no, a Noul returns one probability and is easier to threshold. When the levels need two sentences each to separate, the rubric is usually measuring two things: split it. And when someone needs to read why, pair the Score with an LLM; Jev returns no explanation. How Score fits beside the other types is on the Jev AI page; worked pipelines are in use cases.

Calling a Score question

On this site's endpoint, with a key from API access; the full request and response are in the API reference:

curl -X POST https://jev-agent.com/api/v1/systemone \
  -H "Authorization: Bearer $JEV_AGENT_KEY" \
  -H "content-type: application/json" \
  -d '{"state": "Touches auth/session.ts and 14 call sites; unit tests only.",
       "questions": {"risk": {"type": "score",
         "instructions": "How risky is merging this pull request?",
         "criteria": ["Safe: no runtime effect", "Low: easy to verify",
                      "High: shared or sensitive code", "Critical: could lock users out"]}}}'

On TypeSafe's API, the same body plus a model ID, with your TypeSafe key:

curl https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "jev-latest", "state": "...", "questions": {"risk": {"type": "score", ...}}}'

To try wording changes without writing a request, the playground runs Score questions you write yourself.

Jev Score FAQ

Sources

• TypeSafe: Score
• TypeSafe: Confidence
• TypeSafe: Composite scoring
• Our runs: data/measured-score-2026-11-12.json, nine calls on 2026-11-12