How to Test the Jev Model on Vercel AI Gateway
Testing the Jev decision model on Vercel is straightforward once you know which tools to reach for: use jev directly in the AI Gateway model playground with zero code, or call it from your application through the AI SDK’s experimental_evaluate function with the model id typesafe-ai/jev. The Jev model page on AI Gateway shows the full spec: input tokens cost $0.042 per million, the context window is 32,000 tokens, and routing goes through Vercel AI Gateway. Both paths require a Vercel team with an active AI Gateway API key.
Jev is TypeSafe AI’s System One decision model. Instead of generating prose, it takes your shared state and typed questions — Noul, Choice, or Score — and returns typed answers plus a probability distribution over possible outcomes. That means testing Jev means reading decisions and confidence values, not parsing text.
This guide covers the complete test flow: playground trial, key setup, your first evaluation in code, all three question types, reading probabilities, mocking Jev in unit tests offline, and current pricing and free tier details so you know exactly what a test run costs.
What Jev is and why you test it differently
Jev is TypeSafe AI’s System One decision model available on Vercel AI Gateway, and it works differently from every language model in the catalogue: rather than generating text, it evaluates your shared state against a set of typed questions and returns typed answers with probability distributions attached to each answer. That core design is what makes testing it a distinct workflow.
Jev, TypeSafe AI and Vercel AI Gateway
TypeSafe AI built Jev as a purpose-specific evaluation primitive — the intent is to sit between your application logic and a final decision, giving you a machine-readable probability rather than a sentence. Vercel hosts it in the AI Gateway model catalogue under the canonical model id typesafe-ai/jev, giving every Vercel team with AI Gateway access a single stable endpoint.
Jev launched on AI Gateway on September 15, 2026, and became the fastest-adopted model in AI Gateway history, reaching approximately 13% of paid teams within 24 hours. That adoption speed reflects both the novelty of a typed-decision model in the catalogue and the low barrier to entry — the playground lets teams evaluate it in minutes without writing a line of code.
The model processes your state and questions in parallel, returning choices, scores, and boolean probabilities in one response object. There are no output tokens in the traditional sense: the response carries structured answers, not free-form prose.
Why testing a decision model is not like testing a chat model
When you test a chat model you read text. You check whether the prose is coherent, relevant, or safe. The evaluation surface is open-ended and subjective. Jev collapses that surface: the answer is always one of a small, typed set, and the model attaches a probability to each possible value.
A practical test for Jev therefore checks two things: whether the top answer matches the expected decision, and whether the probability clears a confidence threshold you set for your use case. The questions Jev accepts include:
- Boolean — estimates truth of a statement, returns a probability between 0 and 1
- Choice — selects one of up to 255 named options plus per-option probabilities
- Score — grades against a 2–10 level rubric plus per-level probabilities
Because there are no output tokens to inspect, standard text-quality metrics do not apply. Your test suite validates decisions, not words.
Test Jev in the playground with no code
The model playground lets you try Jev directly from the AI Gateway model page before you add it to any project — no API key, no SDK, no deployment required. Opening the playground and running a question takes about two minutes and shows you exactly what the response structure looks like before you write a single line of code.
Open the Jev model page
To reach the playground, open your Vercel Dashboard, navigate to the AI Gateway tab in the left sidebar, and select Models. Find Jev in the catalogue — its canonical model id is typesafe-ai/jev — and open the model detail page. The playground panel appears on the right side of that page.
The Jev playground differs visually from the chat model playgrounds in AI Gateway. Chat models expose a prompt field plus sliders for temperature, max output tokens, and top-p. The Jev playground surfaces two inputs instead: a state field and a questions field. That layout reflects the model’s actual interface and prevents you from sending a chat-style prompt that Jev would reject.
Enter state and a question
In the state field, paste any text that represents the context you want Jev to evaluate — a support ticket, a JSON object describing a transaction, or a plain sentence. For a first test, something like "The support agent issued a full refund to the customer." works well. In the questions field, add a Boolean question such as "Was a refund issued?" and run the evaluation.
The response panel shows the typed answer — true or false for a Boolean — alongside the probability Jev assigns. A probability near 0.99 signals high confidence; near 0.5 signals uncertainty. Settings that shape chat model outputs, such as temperature and top-p, do not control Jev’s decision output in the same way, so the playground correctly hides or disables them for this model type.
Set up: API key and AI SDK version
Testing Jev in code requires a Vercel team, an AI Gateway API key exported as AI_GATEWAY_API_KEY, and AI SDK 7.0.105 or later — the experimental_evaluate function that invokes Jev ships in that release and does not exist in older SDK versions. The official Jev documentation covers every parameter in detail.
Create and export an AI Gateway API key
Open your Vercel Dashboard, go to AI Gateway, and select the API Keys tab. Click Create API Key, give it a name that identifies the project or environment (for example jev-test-dev), and click Create. Copy the key immediately — Vercel shows it only once. Export it in your shell before running any evaluation:
export AI_GATEWAY_API_KEY="your_key_here"If you are deploying to Vercel rather than running locally, you can skip the long-lived key entirely. Vercel deployments support an OIDC token for AI Gateway authentication, which rotates automatically and avoids storing a static secret in environment variables. For local development and CI pipelines, the exported key is the straightforward path.
Install the right AI SDK
The experimental_evaluate function requires AI SDK 7.0.105 or later, and that version is what introduces support for Choice (up to 255 options), Score (2–10 ordered levels), and Boolean question types. Install or upgrade with:
pnpm add ai@latestNode.js 22.18 or later is required; earlier Node versions are not supported by this SDK release. If your test suite runs in Python, the AI SDK for Python exposes the same evaluation interface, so you are not limited to JavaScript or TypeScript projects.
Run your first Jev evaluation in code
A Jev evaluation runs by importing experimental_evaluate from the AI SDK, passing the model id typesafe-ai/jev, your state, and a questions object — the SDK routes the call through Vercel AI Gateway and returns a typed result object. The model id typesafe-ai/jev is the only model identifier that routes to Jev; aliases do not exist.
Minimal Boolean example
Import experimental_evaluate as evaluate (the rename is optional but common), then call it with the three required fields:
import { experimental_evaluate as evaluate } from 'ai';
const result = await evaluate({
model: 'typesafe-ai/jev',
state: 'The support agent issued a full refund to the customer.',
questions: {
refundIssued: { type: 'boolean', question: 'Was a refund issued?' },
},
});
console.log(result.answers.refundIssued);A Boolean evaluation returns a probability value, not a raw boolean. For the state above, Jev returns a probability close to 0.99, where 0.98 means a strong yes, ~0.02 means a strong no, and 0.5 indicates uncertainty. Your application logic converts the probability to an action using a threshold you define.
Passing state and the gateway provider
State can be a plain string, a JSON object, or an array of objects — Jev serialises all three formats. For richer context, pass a JSON object describing the full event: user tier, agent id, refund amount, and outcome. Larger state objects consume more input tokens, so the 32,000-token context window sets the practical ceiling for what you can pass in one call.
An alternative form uses the provider constructor explicitly, which is useful when you need to configure gateway options per call:
import { gateway } from '@ai-sdk/gateway';
const result = await evaluate({
model: gateway.evaluationModel('typesafe-ai/jev'),
state: myStateObject,
questions: { ... },
});Log result.answers to inspect each typed answer and its associated probability. The gateway.evaluationModel provider form and the string shorthand produce identical responses.
Boolean, Choice and Score questions
Jev answers three typed question kinds — Boolean, Choice, and Score — and every kind returns both a top answer and a probability distribution across all possible values. The type you choose determines what Jev returns and how you read confidence from the result.
When to use each type
Choosing the wrong question type is the most common setup mistake. Boolean suits binary truth claims where you need a 0-to-1 probability. Choice suits classification tasks where the answer is one of a known set of labels. Score suits quality grading where the answer is a ranked level from a rubric you define.
| Question type | Use case example | Answer returned | Options |
|---|---|---|---|
| Boolean | “Was a refund issued?” | Probability 0–1 | 2 (true / false) |
| Choice | “Which department should handle this?” | Named option + probabilities | Up to 255 options |
| Score | “How clear is this support reply?” | Level + probabilities | 2–10 ordered levels |
Boolean probability near 0.98 signals a confident yes; near 0.02 a confident no; near 0.5 uncertainty. Choice returns the name of the selected option plus a probability for each option in your list. Score returns the winning level (for example "level_3") plus a probability per level.
Reading probabilities and confidence
Every answer in result.answers carries the top value and the full probability distribution. For a Choice question the distribution object lists every option you defined, each with its probability, summing to 1.0. For a Score question the distribution covers every level in your rubric.
When automating decisions, apply explicit thresholds rather than always trusting the top answer. A practical pattern is: if the department confidence is 0.6 or higher and the selected option’s probability is 0.7 or higher, process the decision automatically; otherwise route to human review. These thresholds are business decisions — Jev provides the probability, and your code decides what to do with it. Reading providerMetadata.confidence alongside result.answers gives you two independent signals to combine before committing to an automated action.
Test your Jev logic without calling the API
Experimental_EvaluationMockModelV4 from ai/test lets you test branches offline by returning fixed answers without making any network call, which makes it the right tool for unit tests where you need speed and determinism. Import it alongside your evaluation function, pass a mock answers object including providerMetadata.confidence, and the SDK treats it exactly like a live Jev response.
Here is the step-by-step setup for a Vitest test suite:
- Install
aiandvitestas dev dependencies. - Import
Experimental_EvaluationMockModelV4fromai/testin your test file. - Build a mock model that returns the
answersshape your function expects, including aconfidencevalue inproviderMetadata. - Pass the mock as the
modelargument toexperimental_evaluatein your test call. - Assert that your application logic routes correctly for each fixed answer value.
- Add a second test case with confidence below your threshold and assert the human-review branch is taken.
- Add a third case with missing
providerMetadatato confirm your code handles absent confidence gracefully.
What you cover with these cases:
- Clear assignment: top answer matches expected value and confidence is high
- Confidence floor: top answer is correct but confidence is below threshold, triggering review
- Low probability: top answer probability below 0.7 despite high raw confidence
- Missing metadata:
providerMetadatais undefined or empty
Using the mock means your unit tests run in milliseconds, consume zero tokens, and never depend on network availability. Integration tests against the live typesafe-ai/jev endpoint remain useful for verifying that the state you pass produces the probabilities you expect under real conditions, but those belong in a separate slower suite.
Pricing, free tier and what a test costs
Jev costs $0.042 per million input tokens on Vercel AI Gateway, bills only input tokens with no output-token charge, and was free for all AI Gateway users through September 25, 2026. That pricing model makes Jev evaluation dramatically cheaper to run at scale than a comparable language-model call.
Cost per test run
The context window is 32,000 tokens, and the entire evaluation cost comes from the input side — state text plus question definitions. A typical Boolean evaluation with a short support-ticket state string uses roughly 50–150 tokens, putting the cost at a fraction of a cent per call. Larger state objects, such as a full JSON transaction record, may use 500–2,000 tokens but remain under $0.0001 each at the $0.042 per million rate.
| Item | Value |
|---|---|
| Input token price | $0.042 per million tokens |
| Output token price | $0.00 (none billed) |
| Context window | 32,000 tokens |
| Typical short evaluation | ~100 input tokens |
| Cost per batch of short evaluations | a fraction of a cent |
AI Gateway Credits cover Jev calls, but Credits require a payment method to be on file for your Vercel team. The System One API pricing page documents the full rate card including any volume tiers TypeSafe AI publishes.
Free access window
Jev was available at no cost on AI Gateway through September 25, 2026, giving teams roughly ten days from its September 15 launch to run evaluations without spending Credits. That window was intentional — TypeSafe AI and Vercel wanted teams to build and test integrations before billing began.
TypeSafe AI’s own benchmarks report Jev runs up to 194 times faster and up to 445 times cheaper than language models on equivalent workflow evaluations. Both figures reflect the absence of output token generation and the narrower inference task: Jev does not produce prose, so it skips the autoregressive token-by-token generation that dominates latency and cost in standard language models. For high-volume test pipelines that evaluate thousands of decisions per run, those multipliers translate directly to lower CI costs and faster feedback loops.
cURL, HTTP API and monitoring test requests
You can also test Jev through the HTTP Evaluation API using cURL and inspect every run in AI Gateway Logs — useful when your test environment is neither JavaScript nor Python, or when you need a raw request to debug a failing evaluation. Jev is available on Vercel AI Gateway through a standard HTTP endpoint, and the authorization header is the same AI_GATEWAY_API_KEY used by the SDK.
A cURL call to the Evaluation API uses Authorization: Bearer $AI_GATEWAY_API_KEY and a JSON body containing model, state, and questions. The endpoint follows the TypeSafe-compatible evaluation schema, so any HTTP client that can POST JSON can run a Jev evaluation. The response is the same typed answers object the AI SDK returns, serialised as JSON.
Fields to include in a minimal cURL request:
model:"typesafe-ai/jev"state: your state string or serialised JSON objectquestions: object with one or more typed question definitionsAuthorizationheader:Bearer $AI_GATEWAY_API_KEY
AI Gateway Logs show every Jev request within roughly 90 seconds of it completing, including status code, model id, input token count, cost, latency, and routing metadata. Open the Logs tab in the AI Gateway section of your Vercel Dashboard and filter by model typesafe-ai/jev to isolate your test traffic. The token count and cost columns confirm what each call consumed, which is useful for estimating production spend before you move off the free tier.
Common failure patterns visible in the logs include 401 errors (key not exported or expired), 422 errors (malformed question type or out-of-range Score levels), and 413 errors (state exceeds the 32,000-token context window). Matching the status code in the log to the request body in your test is the fastest way to diagnose a failed evaluation without adding SDK-level debug logging.