Jev guardrails: check every LLM draft before it reaches a user

The problem

Verifying LLM output usually means a second LLM pass (doubling cost and latency) or a pile of brittle regexes that miss anything semantic.

How Jev handles it

Because questions run in parallel against one state, a battery of checks costs about what one costs. Put it on the way out, between generation and the user.

response = client.system_one(
    model="jev-latest",
    state=draft_response,
    questions={
        "answers": Noul(
            instructions="Does this answer the user's question?",
            criteria=NoulCriteria(true="Directly addresses the ask",
                                  false="Evasive or off-topic"),
        ),
        "no_pii": Noul(
            instructions="Is this free of personal data?",
            criteria=NoulCriteria(true="No names, emails, addresses or IDs",
                                  false="Contains personal data"),
        ),
        "on_brand": Noul(
            instructions="Is the tone professional?",
            criteria=NoulCriteria(true="Courteous and appropriate",
                                  false="Rude, flippant or off-brand"),
        ),
        "no_promise": Noul(
            instructions="Is this free of commitments we cannot keep?",
            criteria=NoulCriteria(true="Makes no guarantee about dates or outcomes",
                                  false="Promises a refund, date or result"),
        ),
    },
)

a = response.answers
if a["no_pii"].noul < 0.9 or a["answers"].noul < 0.7:
    return regenerate_or_escalate()

Phrasing the checks for Jev guardrails

Point every check the same way. In the sample each Noul's true side is the safe outcome — answered, no personal data, on-brand, no promise — so one rule, "block if any check falls below its threshold", reads them all. Mixing directions invites a quietly inverted threshold, and you cannot patch one by subtracting from 1: TypeSafe's own example has a question scoring 0.72 while its negation scores 0.47, and the two need not sum to one.

Make each check atomic and name what it looks for. "Is this free of commitments we cannot keep?" works because the criteria spell out what a commitment is — a refund, a date, a result. Jev guardrails worded that concretely caught both problems in the measured draft, no_pii at 0.01 and no_promise at 0.06, while the tone check passed at 0.89.

What to put in the state

A Jev guardrails call can only check against what the state contains, so send the draft and the user message it replies to. The measured state held only the draft, and the "answers the question" check returned 0.75 — a hedge, since nothing in the state said what the question was. Label the two parts and name them in the instruction ("Does draft answer user_message?") so the model knows which text is which.

For policy checks, add the few facts the draft must respect as a short labelled block — refund window, delivery times, what agents may promise. Only the lines the checks need; a whole policy handbook in every call buries them.

What one decision costs

Jev guardrails bill for reading the draft and nothing for the verdicts, so cost follows the length of what you check. First line: the measured draft with four checks. Second: an assumed 1,200 tokens for a longer reply plus the user's message and a policy block. Third: 200,000 replies a month at that larger size.

# the measured draft, four checks
474 tokens × $0.042 / 1M = $0.0000199

# a longer reply with its question and a policy block
1,200 tokens × $0.042 / 1M = $0.0000504

# 200,000 replies a month at that size
200,000 × 1,200 = 240,000,000 tokens × $0.042 / 1M = $10.08

Regenerations add calls. If one draft in ten fails and is rewritten once, the check runs 1.1 times per reply, so budget for that rather than for replies sent.

When not to use Jev for this

• The check needs the right answer, not a plausible one. Jev can only judge against what is in the state; verifying facts means retrieving the source and including it.
• The rule is countable or exact — a word limit, a maximum number of links, a disclaimer that must appear verbatim. Code checks those precisely; Jev does not count reliably.
• You need the draft fixed, not flagged. Jev cannot rewrite a sentence; the regeneration still comes from your LLM.
• One check is the only barrier before an irreversible action, such as issuing a refund. Injected text in a draft cannot make Jev write anything, but it can argue for a pass and move a probability, so put a deterministic control in front of anything that cannot be undone.

Where it fits in your stack

Jev guardrails sit between generation and delivery. The LLM drafts, a Jev guardrails call runs every check, and a failure either triggers a regeneration with the failed check named in the prompt ("remove the customer's email address") or escalates to a person once a retry limit is reached. Cap the retries so a stubborn failure escalates instead of looping.

Pair them with deterministic filters for what code does perfectly — email addresses, card numbers and phone formats by regular expression — and leave Jev the judgements a pattern cannot make, such as whether a sentence amounts to a promise.

Notes from the field

• Guardrails are where input-only pricing pays off most: a battery of checks costs the draft's tokens once, however many verdicts come back.
• Because Jev cannot generate text, a prompt-injected draft cannot make the checker emit anything harmful.
• Log every check result. The distribution over time tells you which failure mode your prompts actually have.

Common questions