Routing on a number: one CrewAI support router on three decision APIs

Most agent routers I've built ask a language model to pick a route, then parse whatever it wrote back. That works. But a router doesn't need a paragraph. It needs a number it can compare against a threshold.

In the last couple of months three providers shipped APIs built for exactly that: TypeSafe's Jev, OpenAI's Decisions API, and Google's Gemini Decision. I already had a support router running on Jev inside a CrewAI flow, so I ported it to the other two and ran all three side by side. Two of them handled every case. The third missed the one case the demo exists to teach.

What a decision API is

You send some context and a set of typed questions. Each question is a yes/no, a choice from options you define, or a score on a scale. The API sends back a probability for every possible answer and writes no text at all. Your code reads the numbers and decides what to do.

Two lanes. A generative call takes history and an order, the LLM writes a JSON route, code parses and validates it, then branches on a label in about 1.2 seconds. A decision call takes the same input with three typed questions, returns probabilities such as returns 0.98, clarify 0.02 and order 0.00 with zero output tokens, and code branches on a probability cutoff in 0.14 to 0.30 seconds. — Text out vs numbers out. With a decision API the parsing step disappears, and you get a probability you can set a threshold on. Times are medians from this test.

The three APIs are close cousins. Gemini's request format is nearly identical to Jev's. OpenAI renamed a few things and returns answers as a list. The practical differences are elsewhere:

	Jev	OpenAI Decisions	Gemini Decision
Model	`jev-1.13.0` / `1.14.0`	`gpt-6-luna`	`gemini-decision-1`
Yes/no type	`noul`	`predicate`	`noul`
Input	Text, PNG/JPEG	Text, images, JSON, chat and tool-call history	Text, image, audio, video, PDF
Chat history	Pass it as JSON	Native message items	Not supported; send it as text
Input price / 1M tokens	$0.042	$0.10	$0.10 (unconfirmed)
Confidence for a choice	(p − 1/n) / (1 − 1/n)	Formula not published	Top probability

All of this comes from each provider's docs as of November 2026. Output tokens are free or zero on all three.

The use case: a support router in CrewAI

The router comes from module 04 of crewai-lab. A customer is asking about order #4021, which has three blue items: Studio Headphones, Sport Earbuds and a Travel Speaker. A CrewAI conversational Flow keeps the session history. On every turn it makes one request to the decision API with three questions, runs the answers through a few rules in Python, and dispatches to an @listen handler. The handlers only reply. They never start a return or touch the order.

Flow diagram. handle_turn adds the message to history. route_turn sends one request with three questions: route (order, returns or clarify), item_action (is this about a particular item) and item (one of three items or unresolved). A Python gate sends the turn to clarify if route confidence is below 0.50, or if the route is returns, item_action is at least 0.50, and the item is unresolved or below 0.60 confidence. CrewAI then dispatches to the order, returns or clarify handler. — Where each part lives. The model answers the questions. Python decides whether those answers are good enough to act on. CrewAI runs the handler. Returning a route from `route_turn()` skips CrewAI's built-in LLM router but keeps its session history and dispatch.

The demo conversation is four turns long. The third one is the reason the module exists.

Four turns. Turn 1, Where are my Studio headphones, routes to order. Turn 2, They arrived damaged, can I return them, routes to returns. Turn 3, Can I return one of the other blue items too, is highlighted and routes to clarify because two items still fit. Turn 4, I mean the Sport Earbuds, routes to returns. — The four-turn demo and its expected routes. On turn 3 the model correctly says returns, but a correct intent label is not enough to act. The application asks which item.

How I tested it

31 authored conversations from the module's eval set: 10 with an ambiguous item, 5 with a clear item, plus policy questions, corrections, follow-ups, order questions, unclear intent and one prompt-injection attempt.
Three repeats of every case on all three decision APIs, randomly interleaved, from one laptop. That's 279 decision calls with zero request errors.
The same input everywhere. Gemini doesn't accept chat history, so every provider got the order and messages as one JSON block.
The same rules everywhere. Each provider defines confidence differently, so I recomputed it from each provider's probabilities with Jev's formula. A 0.60 cutoff then means the same thing for all three.
Two generative baselines for comparison: GPT-6 Luna and GPT-6 Astra through the Responses API with structured output. These answer the same questions as JSON, without probabilities.

What I found

Model	Correct route	Median	p95	$ / 1k calls
Jev 1.13	93/93	142 ms	219 ms	$0.031
OpenAI Decisions	93/93	235 ms	488 ms	$0.067
Gemini Decision	87/93	302 ms	412 ms	$0.048*
GPT-6 Luna, generative	92/93	1,205 ms	1,868 ms	$0.055
GPT-6 Astra, generative	93/93	2,102 ms	3,340 ms	$5.68

*Gemini cost uses a list price I haven't confirmed yet. Every decision API gave the same answer on all three repeats of every case.

Gemini missed turn 3

On turn 3, all three decision APIs picked returns as the route and unresolved as the item. They split on the yes/no question, “Is this about a particular item?” Jev said 0.94, OpenAI said 0.61, and Gemini said 0.474.

The gate only checks for an unresolved item when that answer is at least 0.5. At 0.474 the check never ran, so the flow went ahead with a return without knowing which item. The same thing happened on a second case, “What about sending one back?”, at 0.481. Those two cases account for all six of Gemini's misses.

Dot plot of each provider's yes/no probability for 31 cases from 0 to 1, with a dashed cutoff at 0.5. Jev's should-be-yes answers all sit at 0.87 or above and its should-be-no answers near 0. OpenAI's should-be-yes answers sit at 0.61 or above. Gemini has five should-be-yes answers below the cutoff, between 0.28 and 0.49, marked in red. — “Is this about a particular item?” for all 31 cases. Jev and OpenAI separate the yes and no cases cleanly. Gemini's answers on ambiguous cases drift toward the middle, and five fall under the cutoff. Three of those still ended on the right route because another rule caught them.

I don't read this as Gemini being bad at the task. Its answers on clear-item cases were 0.96 to 0.98. On ambiguous ones they ranged from 0.28 to 0.87. It seems to read “a particular item” as “an item that's already been identified.” That's a defensible reading. It just isn't what my rule assumed, and my cutoff was tuned on Jev.

A threshold is part of the model it was tuned on. Moving it to another provider is a new experiment.

Confidence means three different things

Take one choice with four options and a top probability of 0.80. Jev reports a confidence of 0.73, because it measures how far the answer is from an even split. Gemini reports 0.80, the top probability itself. OpenAI returns a confidence but doesn't say how it's calculated. If I'd thresholded on each provider's own number, the same 0.60 cutoff would have meant three different things.

OpenAI is very sure of itself

OpenAI Decisions got every case right. It also reported an item confidence of exactly 1.0 on 57 of 93 answers. Gemini never reached 1.0, and Jev's lowest item confidence was 0.40. Being right with a 1.0 is fine. But you can't set a threshold below a value that's always 1.0, and this set is too easy to tell whether those numbers would hold up on harder traffic.

The endpoint matters as much as the model

Horizontal bar chart of median latency: Jev 142 ms, OpenAI Decisions 235 ms, Gemini Decision 302 ms, GPT-6 Luna generative 1,205 ms, GPT-6 Astra generative 2,102 ms. — Median round trip, drawn to scale. OpenAI Decisions and the generative Luna call use the same model. The decision endpoint was about 5× faster, short of the 10× OpenAI advertises.

Jev was the fastest and cheapest of the three. Its one weak spot showed up as expected: on “the smaller one,” where the order has no size information, it picked the Sport Earbuds three times. Its confidence was 0.40 to 0.49 each time, so the 0.60 rule caught it and the flow asked the customer instead.

What I'd take from this

If your code branches on the answer, ask for a number. All three decision APIs answered in 142 to 302 ms at the median. The generative calls took 1.2 to 2.1 seconds.
Give the model a way to abstain. The unresolved option and the confidence rules did more for correctness than any single provider did.
Keep the decision in code and log the raw answers. Gemini's miss is only visible because the raw 0.474 was recorded next to the final route.
Retune thresholds per provider on conversations you didn't use to pick them.
Choose by fit. Going by the docs, Jev is the cheapest for text, OpenAI is the one that takes chat and tool-call history natively, and Gemini is the one that reads audio, video and PDFs.

Limits of this test

These are 31 conversations I wrote, not real support traffic, and the cutoffs were tuned on Jev before the other two existed. That setup favors Jev. Latency came from one laptop on one day. Moving Gemini's cutoff to 0.45 would likely fix both misses, but picking it after seeing these results would be tuning on the test. The next step is a set of harder conversations, held out from any threshold tuning.

Run it yourself

The OpenAI and Gemini backends live on the decision-api-backends branch of crewai-lab, not on main yet. The full results are in that branch's docs/support-evaluation.md.

git clone -b decision-api-backends https://github.com/crewAIInc/crewai-lab.git
cd crewai-lab/modules/04-agentic-routing-jev/example
python3 eval_router.py --check   # no API keys needed
uv run --env-file .env --no-project python3 eval_router.py \
  --models jev openai-decisions gemini-decision --repeats 3

You'll need TYPESAFE_API_KEY, OPENAI_API_KEY and GEMINI_API_KEY in .env. Raw responses, timings and scores are saved under eval_results/.