Meaning-Based Retrieval: Your Vector Search Doesn't Understand the Question
You know the email exists. You are certain of it.
You search for contract renewal and nothing comes back, because the message you are thinking of says "happy to keep things going for another year on the same terms," and never uses either word.
So you try again. And again. Eventually you find it, or you give up and assume you imagined it.
Nothing told you which. No error, no warning, and no way to tell "it isn't there" from "you didn't happen to use their words." The search returned results. They looked fine.
Everyone has had that afternoon. It is also, precisely, what happens inside the search behind your company's documents, your case files and your AI assistant, at a scale where nobody is sitting there knowing the answer exists, so nobody ever notices it did not come back.
That failure has a name in every vendor's marketing: retrieval that understands. It finds what you meant, not what you typed. It is the most confident claim in enterprise software, and almost nobody publishes the test.
We built the thing that claim describes, and we call it meaning-based retrieval. Before explaining what that is, it is worth establishing that the problem is real, because the entire industry insists it has already been solved.
So we ran one. Seven documents, one question, nine embedding models across three vendors, every number published.
On the strongest model we tested, OpenAI's text-embedding-3-large, the top result was one of
the five. Not subtly wrong. A note about somebody's pet drooling on a porch, waiting for its
dinner, ranked above the paper the question was actually about.
It was not alone. Three of the nine ranked a decoy first, including both of Cohere's asymmetric models, the design most often offered as the fix for exactly this problem. Where the right document did come first, it did so by four hundredths of a point. On one model, by two thousandths.
If your search is a product recommendation, that margin is a shrug. If it is a literature review a regulator will audit, a legal hold, or an analyst asking what your exposure is, that margin decides whether the document surfaces or nobody ever sees it. And when it does not surface, nothing tells you. There is no error. The answer simply looks complete.
The test
Seven sentences, one question, and whichever embedding model you already pay for. It takes about a minute to reproduce, which is the point. This is not a benchmark, it is something you can check before lunch.
We used the most over-taught example in the language so that anybody can verify the right answer at a glance:
Pavlov's dog drooled when he heard a bell
Two of the seven documents describe that experiment: one paraphrased so it repeats none of the question's words, one in the clinical language a physiologist would use. The other five are about other dogs, other bells, other drooling entirely, and several use exactly the words that were typed.
The listing below is the model that did worst, OpenAI's text-embedding-3-large, the strongest
general-purpose embedder most teams can buy. Cosine similarity, all eight strings
embedded in one request. The other eight configurations follow further down.
Scores are given to three decimals because OpenAI's values drift in the fourth between calls. The ordering does not. Titan's are stable to four, and reproduce exactly.
Those seven were written to make the vocabulary mismatch explicit. That is the point of the test, and it is what any production system meets the moment two people describe one thing differently. The scores are whatever the model returned: nothing tuned, no run discarded.
1. 0.720 The dog heard a bell on the porch and drooled over his dinner
bowl.
2. 0.659 Pavlov owned a canine. It experienced an increase in saliva
production in one particular experiment. In this situation the
owner tolled the apparatus. ← relevant
3. 0.621 My dog drools whenever the doorbell rings, he thinks it means
visitors.
4. 0.581 The physiologist paired an auditory stimulus with feeding until
the animal produced saliva on hearing the tone alone. ← relevant
5. 0.436 Church bells rang across the village while a stray dog wandered
the square.
6. 0.419 Drooling in dogs can indicate dental disease and should be
checked by a vet.
7. 0.398 A bell was mounted above the shop door so the owner heard every
customer.
Position one is a note about a pet on a porch waiting for its dinner. The paraphrase, which shares not one word with the question, is second. The physiologist's description is fourth, below a dog reacting to a doorbell.
Cut that list at three, which is what a production system does, and you keep two documents about other people's dogs and throw away the physiologist.
To be fair to the technique: this is the worst of the nine, and on most of the others the paraphrase does come first. Anyone who tells you semantic search simply fails this test has not run it. Here is Titan v2, where it wins.
It wins by 0.04. Four hundredths of a point between the right answer and a sentence about a completely different dog. And the second relevant document still lands fifth of seven, below the doorbell and below a veterinary page about dental disease. Cutting at three loses it here too.
That is the pattern across every model we tried. Either the right document is not first, or it is first by a margin too thin to mean anything.
"You picked a weak model"
It is the first thing anyone says, so we ran it on every embedding model we could reach.
| model | where the two relevant documents ranked | margin between #1 and #2 |
|---|---|---|
| Titan v2, 1024d | 1st and 5th | 0.0410 |
| Titan v2, 512d | 1st and 5th | 0.0392 |
| Titan v1, 1536d | 1st and 5th | 0.0433 |
| Titan G1 | 1st and 5th | 0.0433 |
OpenAI text-embedding-3-small | 1st and 4th | 0.0089 |
OpenAI text-embedding-3-large | 2nd and 4th | 0.0610 |
Cohere embed-v4 | 1st and 4th | 0.0462 |
Cohere embed-english-v3 | 2nd and 4th | 0.0023 |
Cohere embed-multilingual-v3 | 3rd and 5th | 0.0050 |
The Cohere rows deserve a note, because they answer the sharpest objection to this whole test.
Cohere embeddings are asymmetric: queries and documents are encoded differently, through
search_query and search_document, precisely so that a short question and a long passage can be
compared properly. It is the design most often offered as the answer to vocabulary mismatch, and
we used it exactly as intended.
It did not help. embed-english-v3 puts the wrong dog first, ahead of the paraphrase by two
thousandths of a point. embed-multilingual-v3 puts two decoys above the right answer.
The pattern across the table is the thing to take away. The stronger model did not fix the
problem. It reordered it. Where the right document does come first, the margin ranges from four
hundredths down to 3-small's nine thousandths, which is not a margin but a rounding error
that happened to land your way.
Three vendors, two generations, three embedding sizes, symmetric and asymmetric encoding. This is not an artifact of a model, a vendor, or a setting.
What is actually being measured
A dense vector encodes how words are distributed. Not what they denote: how they co-occur, at scale, across the corpus the model was trained on.
That explains both halves of the result exactly. Tolled the apparatus and heard a bell rarely appear in the same contexts, so they land far apart even though they describe the same event. Dog on a porch and Pavlov's dog share their entire vocabulary, so they land close together even though one is a landmark of experimental psychology and the other is a pet waiting for dinner.
The model has no way to know that tolling an apparatus makes a sound, that a canine is a dog, or that saliva production is drooling. It was never built to. It is a very good statistical approximation of language, being asked to stand in for comprehension.
That is not a defect in a particular model, and it is not fixed by a bigger one. It is what the technique is.
Everyone ships a correction. That is the tell.
The industry knows this gap exists. Look at what each vendor built to cover it, not at their marketing, at their feature list.
Elastic ships a reranking retriever and its own rerank model, alongside ELSER for term expansion and synonym sets somebody maintains by hand. OpenSearch ships a rerank processor in the search pipeline. Vespa, technically the most capable of them, documents cross-encoder reranking and phased ranking you wire into a ranking profile. Snowflake Cortex Search and Databricks Vector Search both put a reranking stage on top of hybrid retrieval. Pinecone, Weaviate and Qdrant ship hosted rerankers. Even MongoDB Atlas and pgvector, which are not search products at all, document reranking as the recommended pattern. And turbopuffer, the newest and most deliberately minimal of them, ships reciprocal rank fusion server-side and points you at Cohere or Voyage for anything past it.
Eleven vendors, one architecture. That unanimity is the argument, and it is checkable from their own documentation rather than ours: nobody builds a correction stage for a system that already understood the question.
It is worth saying that none of this is incompetence. Each of those teams built the best available answer to the problem as the industry framed it: make the vectors better, then fix the ordering afterwards. The framing is what we disagree with.
And each correction has the same structural limit, which is worth stating plainly because it is not a matter of opinion:
A reranker only ever sees the candidate set. It reorders what retrieval already returned. In the listing above, the physiologist paper sits at position four. Retrieve the top three, as a production RAG pipeline does, and no reranker on earth recovers it, because it was never fetched. Reranking improves precision at the top of a list. It cannot repair what the list left out.
Your recall ceiling is fixed at retrieval. This is the sentence that matters. Whatever the vocabulary search missed is missed permanently, and every stage downstream (the cross-encoder, the LLM, the reranker) is working inside a set that was already wrong. Precision can be bought later. Recall cannot.
A second store is a second thing that drifts. Graph RAG buys symbolic matching at the cost of a knowledge graph built by imperfect extraction, maintained against a corpus that changes, and silently diverging from it whenever it is not. Two systems, and you hope they agree.
Rank fusion cannot invent a rank. The cheapest correction, and the most widely shipped, is
fusing a keyword list with a vector list. It is worth doing the arithmetic on, because the
arithmetic is published. turbopuffer scores each document as weight / (rank_constant + rank),
summed across the sub-queries, with the rank constant defaulting to 60. Over seven documents that
puts first place about ten percent ahead of last, which means the formula rewards appearing in
both lists far more than it rewards being right in either. Now apply it to this question. The
paraphrase shares not one word with what was asked, so the keyword list is precisely where it does
not appear, and anything appearing in both lists outscores it no matter what the vector list
thought. For the single document that required understanding the question, the second opinion
contributes nothing and the fusion is worse than the vector list alone. And none of this is
novel, ours included: we have shipped reciprocal rank fusion for years. That is rather the point.
It is commodity arithmetic, everybody has it, and it is not where the problem lives.
Query expansion is a guess. Rewriting the query to add classical conditioning, Pavlovian response, canine salivation works when the guess is right. It is an inference about the wording of a document nobody has read yet, paid for with a model round-trip on every single query, at every scale, forever.
What this costs in the real world
Three consequences, and none of them are theoretical.
You pay for the correction on every query. A reranking pass is a second model invocation per search. At a thousand searches a second that is a second inference fleet, sized and funded, whose entire job is compensating for the first stage. The economics are visible in anybody's cloud bill.
Latency is additive. Retrieve, then rerank, then generate. Each stage waits for the last. Teams tune the candidate set down to keep the reranker fast, which shrinks the very pool that recall depends on, and makes the miss more likely, not less.
And when it fails, it fails silently. This is the one that should worry you. A missed document produces no error, no warning, and no gap in the results. The answer looks complete. In the test above, a system that returned the top three would have handed back three confident, plausible documents and lost the physiologist, and nobody downstream would ever have known to ask.
Somebody shopping for a jacket will scroll. But "the answer looked complete" is not something a regulator accepts, or a court, or an analyst asked what your exposure is, or a medical examiner comparing a description of remains against five thousand missing-person reports. Those are the searches where the document you did not retrieve is the one that mattered, and where nothing in the result will ever tell you it existed.
Meaning-based retrieval
Every architecture in that list makes the same choice: retrieve on vocabulary, then correct. The correction is where all the engineering has gone: better rerankers, cheaper cross-encoders, smarter query rewriting. All of it sits downstream of a decision that was already made by word statistics.
Lucenia moves meaning to the retrieval step itself. Not a stage in front of it, not a stage behind it, not a second database beside it. The step that decides what comes back at all is the step that understands what was asked.
Ask once, and it answers the same way whether the author wrote dog or canine, drooled or produced saliva, bell or apparatus. No reranking pass to fund on every query. No second store to keep in step with the first. No guessing which words a document nobody has read yet might contain.
The same seven documents
It is a fair question what our own answer looks like on the test we just used to criticise everybody else, and it is the last row of the chart above. Same seven documents, no ranking involved.
Asked for an animal, a bodily response and a sound, which is what the question is about rather than the words it happened to use, four documents come back. Both of the right ones, plus two about a pet on a porch and a dog at a doorbell, because those genuinely do describe an animal salivating at a sound. That is not a failure; it is an honest answer to a broad question.
Narrow it to the same three things occurring inside a study, and the answer is two documents:
Pavlov owned a canine. It experienced an increase in saliva production… ← relevant
The physiologist paired an auditory stimulus with feeding until… ← relevant
Two of seven. Both correct. Nothing else.
Notice what you are not asked to do with that result.
Every model in the table hands back all seven documents, sorted by resemblance, and leaves the real decision to you: how far down do you read? Take three and you get two documents about other people's dogs. Take five and you finally have both right answers, along with a veterinary page. There is no number in the output that tells you where to stop, which is why every production system picks an arbitrary cut-off and lives with it.
The last row is not a better-sorted list. It is a different kind of answer: these two documents are about what you asked, and the other five are not. No score to interpret, no cut-off to choose, nothing to tune.
And this is not free. Those documents carry a record of what they are about, and something had to put it there. That is the work this approach asks for, and it is the honest trade: the vector models were handed raw text and asked to infer meaning from word distribution, which is precisely the inference this post has spent two thousand words showing they cannot reliably make.
That is the whole difference, and it is a difference in kind rather than degree. A better reranker gives you a better ordering of the documents vocabulary already found. Meaning-based retrieval changes which documents are found. Recall is decided at retrieval, and retrieval is the one place nobody else has touched.
We are first to market with it, and we expect that to be temporary. The idea is too obviously right to stay ours alone. What is not temporary is that the industry has spent three years building increasingly elaborate machinery to compensate for a retrieval step nobody was willing to change. That machinery is now the moat around a design decision, and it is the wrong one.
Run it yourself
The test at the top of this page takes about a minute against whatever you have in production. Seven sentences, one question, and whichever embedding model you are already paying for.
The number to watch is not whether the paraphrase comes first, which it usually does. It is how little sits between it and a document about somebody else's dog. Then ask what happens to that margin when the corpus is ten million documents instead of seven, and nobody in the room knows what the right answer was.
Curious what this looks like on real records? Try Lucenia, or talk to us about what you are searching.
