We put a decision model in production the day it was released
TypeSafe released a new kind of AI model today. It does not write text. It answers typed questions with a choice and a probability, in about half a second, for a few cents per thousand decisions. It is already making decisions inside Recruitly, and this is how we wired it in.
Ask AI about this
Today TypeSafe released a model called Jev, and it is already running inside Recruitly. I want to write down what it is, why it changes how we build, where it is deciding things for our customers, and what the production setup around it looks like, because the shape of the system it lets you build is the part other teams can copy.
A model that decides and never writes
Every model we have run in production until now has been a language model. You send it text and it sends text back, and if what you wanted was a yes or a no, you ask for the word "yes" and parse it out of a sentence. With Jev you give it a situation and a typed question, and it answers with a typed value and a probability for how sure it is. The question can be a choice, such as which of these five things an email reply is. It can be a yes or no, such as whether a request is destructive. It can be a score, such as a candidate's experience from one to five.
It replies in about half a second and costs a few cents per thousand decisions. Those two numbers matter more than they look, because they change which questions are worth asking. A full language model call is slow and costs real money, so we asked one only where the answer justified it, and we skipped a long list of small questions that would have made the product better. At half a second and a fraction of a cent, those questions can be asked every single time.
Why a confidence changes the design
Until now every yes or no and every classification in Recruitly went to a large language model. That is slow, it costs money, and the answer arrives with no sense of its own confidence. A language model that is guessing sounds exactly like one that is sure, so the code around it has to treat every answer the same, and we used it sparingly as a result.
A probability changes what the surrounding code can do. When Jev is confident, the decision is made and the system moves on. When it is unsure, the case goes to the language model exactly as it did before. So the new path only ever takes the cases it is good at, the old path is still there for everything else, and nothing gets worse. Most things get faster and cheaper.
The split also tidies the architecture. Jev makes the decision and the language model does the writing. A scorecard verdict is a decision and the comment underneath it is writing, and until today both came from the same slow call, and now each goes to the model built for it.
Where it is deciding things
These are the places inside Recruitly where Jev now answers the question. Each one is a decision that used to be a language model call, or a decision we did not make at all because the call was too expensive to justify.
| Area | What Jev decides |
|---|---|
| Sequences | When a reply comes in to an outreach email, Jev sorts it into out of office, interested, not interested, bounced or neutral. |
| Agents | When a recruiter asks an agent to change something, Jev checks whether it is risky or destructive so the agent asks before acting. |
| Agents | When an agent task starts, Jev picks which of the agency's skills apply to it. |
| Support inbox | When a customer message arrives, Jev decides whether it needs a human reply and how urgent it is. |
| Job adverts | When an advert is published to a job board, Jev picks the board category, job type and salary period. |
| Calls and meetings | When notes are written up, Jev judges whether the meeting went well, badly or neither. |
| Candidates | When a candidate is reviewed against a job, Jev scores skills, experience, location, visa and deal breakers, each with a confidence. |
| Knowledge base | When Simi answers a question, Jev checks that the retrieved content actually answers it before anything is written. |
| Scorecards | When an application or interview is scored, Jev gives the verdict per attribute and the language model writes the comment. |
Every one of these keeps its old path. When Jev is not confident, or unavailable, the language model answers as before.
From model release to production in one day
The reason this took a day is that we did not build anything special for it. Jev went into the same two internal frameworks as every other model we run, our Eval framework and our Chain framework, and those two are the part I would point other engineering teams at.
The Eval framework is how a model earns its place. Before it is allowed near a customer it runs an evaluation battery, a fixed set of real cases with known right answers lifted from the exact places it will be used, and the results go on a leaderboard next to every other model we have tested. It is the same discipline we used when we picked the OCR that reads CVs. For Jev that meant a battery of labelled decisions written from the questions the product already asks, and a model that had to get them right on both hosts before anything else happened.
The Chain framework is how a model gets used. Every model call in Recruitly goes through one AI gateway, and the route a question takes, which model first and which host second, is a published chain that we change without a deploy. Behind the gateway Jev runs on two hosts, so when one is down or slow the other answers and the code asking the question never knows which one did. Every decision is logged with its latency and the host that answered, so usage is visible live. Adding Jev was a new chain for decisions, published the same day.
That is the argument for building the boring infrastructure first. When a new kind of model appears, the work is a row in the Eval registry, a set of test cases, a chain, and a typed question next to an old prompt. TypeSafe call this family System One models, and we are one of the first to have Jev in production, and the credit for that belongs to the gateway, the battery and the chain we built long before today.
What this means if you build with models
- Separate deciding from writing. Most of what software asks a language model is a choice, a yes or no, or a score, and a model built for that is faster, cheaper and tells you how sure it is.
- Use the confidence, because a probability lets the fast path take only the cases it is good at and hand the rest to the path you already trust.
- Keep the old path, as every decision we moved still falls back to the language model, so the worst outcome of the change is the behaviour we had last week.
- Build the two frameworks before the model arrives. An eval framework with a battery of real cases and a leaderboard, and a chain framework with one gateway, a route you can change without a deploy and usage you can see. With those in place a new model is a day of work.
For recruiters none of this is visible, which is the intent. Replies get sorted the moment they land, an agent stops to ask before it does something destructive, a candidate's scores arrive with a confidence next to each one, and Simi checks that what it found answers the question before it writes a word. The product asks more questions than it used to, and it asks them every time.
Written for engineers who are sending every yes and no to a language model and wondering what it is costing them.
