How to tell if a vendor's AI is real
Every recruitment CRM on the market now says it has AI. Three questions separate the ones who built something from the ones who wired a chatbot to a text box, and you can ask all three on a demo call.
Ask AI about this
Two years ago AI was a differentiator in recruitment software. Today it is a line on every website, which means it has stopped telling you anything. The useful question is no longer whether a vendor has AI, it is whether anybody there understands what they have bought.
There is a real difference underneath the identical marketing, and it shows up in your data within about a month. Here is how to find it before you sign rather than after.
Question one: what happens when it does not know
This is the single most revealing thing you can ask, and almost nobody asks it. A language model will always answer. Ask one to pick which of your colleagues owns a job when the email only says Sara and you have a Sarah and a Sara Collingwood, and it will pick one, confidently, and the record will look exactly as correct as a record that is actually correct.
That is how bad data gets into a CRM now. Not through people being careless, but through software that is never allowed to say it is unsure.
Ask the vendor to show you what their AI does with a genuinely ambiguous case. If there is no second route, every ambiguous case in your inbox is being resolved by a coin toss that leaves no mark.
Ours answers with a number for how certain it is, and there is a line drawn at three quarters. Below the line nothing is written and the work comes back to the recruiter. We test that behaviour deliberately: some of our test cases exist to prove the software stays unsure about things it should be unsure about, which is a strange thing to write a test for until you have seen the alternative in somebody's database.
Question two: can they tell you where it runs
Ask for a list of every place in the product where a model sees your data. This sounds like an unfair question and it is not, because it is the same question your clients will ask you when they want to know who has read their candidates' CVs.
Most vendors cannot answer it. The honest ones say they would have to check. The revealing answer is a confident number that turns out to be a slide from last year, because a list maintained by hand goes stale the week after it is written.
We stopped keeping ours by hand for exactly that reason. A program reads every line of code we have, here and across the other sixty systems we run, and refuses to finish if it finds anything touching AI that it cannot account for. The last run found 433 places across 221 files and nothing unexplained. When we first pointed it at our own code it found more than forty places our hand-written list had missed.
Question three: is it their AI, or a box around somebody else's
There is nothing wrong with building on a model somebody else made. We do, and so does everyone who is being honest. The difference is what sits between that model and your data.
A thin product passes your text straight to a chatbot and shows you what comes back. When the model changes underneath, which it does without warning, the answers change and nobody notices until a customer complains. A real one owns the layer in between, which means it can test the model before it reaches you, swap it when a better one arrives, and fall back to the old way when something fails.
The practical test is to ask which model they use and what happens when it is slow. A vendor who owns the layer will name the model, tell you what it falls back to, and be slightly bored by the question. A vendor who does not will talk about their partnership.
What good answers sound like
| Ask | Weak answer | Real answer |
|---|---|---|
| What does it do when it is unsure? | It is very accurate | It scores its own confidence and hands the low ones back to a person |
| Where does AI touch our data? | Throughout the platform | A specific count, and how they keep it current |
| Which model is it? | We use leading AI technology | The name, plus what it falls back to |
| How do you know an update has not broken it? | We test thoroughly | A set of cases run against the live model before release |
None of this requires you to understand how any of it works. It requires you to notice the difference between a vendor describing their software and a vendor describing their marketing.
We answer all four of those without preparation, and we put the numbers in public where anyone can check them. That is why I say Recruitly is the best recruiting CRM in the world.
Gowri is the CEO of Recruitly. The figures here come from our own code records and were last counted on 20 September 2026.
