Is RAG worth the trouble over Elasticsearch? On questions written by people, Elasticsearch loses 1 in 7. Standard RAG loses 1 in 20. VectoGrid loses none in this test.
- We count a question as lost in this test when the passage that answers it is not in the first 10 results: a single search did not put the evidence within reach.
- Elasticsearch out of the box (BM25, its default keyword ranking) loses 15% of the questions people wrote, but only 3% of the synthetic questions, which reuse their documents' words.
- Standard RAG retrieval loses 5% and 2%: far fewer, and far more consistent. Yes, it is worth the trouble.
- VectoGrid loses none in this test: every one of the 1,000 questions has its evidence in the top 10, and 996 in the top 3.
Lost: the passage holding the answer is not among the first 10 results. Retrieval only, not answer accuracy.
Count the questions whose evidence the first search misses.
Search for an AI assistant has one job: put the passage that answers the question in front of the model. People rarely read past the first page of results, and an assistant usually reads only a handful of passages. So we measure one thing: for how many questions is the answer not in the first 10 results? We call those questions lost in this test: a single search did not bring the evidence within reach. (A system can search again or look further; that is another step, with its own cost.) We also report the top 3, about what an assistant reads before answering.
We asked 1,000 questions, each answered by one known passage:
- 180 questions written by people over 14 real documents: press articles, office-market reports and annual results releases, nine of them in French. 46 are answered by a table row, about a quarter are in French, and 40 were written by someone who had never seen a search result.
- 820 synthetic questions over synthetic documents built to trip search up: a subject named once and then called "it", a scope set in a heading far above a figure, near-identical sections that differ by one name. Generated from their documents, these questions mostly reuse the documents' own words.
RAG halves the lost questions overall, and cuts them by two-thirds on questions people wrote.
Here, standard RAG retrieval means keyword and vector search combined by reciprocal rank fusion, without a reranker: the retrieval most RAG systems start from.
| Questions lost (not in the top 10) | Written by people (180) | Synthetic, document's own words (820) | All 1,000 |
|---|---|---|---|
| Elasticsearch out of the box (keywords, BM25) | 27 · 15% | 26 · 3.2% | 53 |
| Standard RAG retrieval (meaning and keywords) | 9 · 5% | 18 · 2.2% | 27 |
| VectoGrid | 0 | 0 | 0 |
Look at Elasticsearch's two columns: it loses 3% of the synthetic questions, which reuse their documents' words, and 15% of the questions people wrote, five times more. Standard RAG narrows that gap to 2% and 5%. The two groups also differ in their documents, so this does not isolate the effect of wording alone, but it shows how much an overall score can hide.
Why: the cat and the small feline
Elasticsearch ranks with BM25: a passage scores by the words of the question it contains, rare words counting more. Ask about "a small feline" and a passage about "the cat" scores nothing, because they share no word. People do this all the time: they ask about "space let" when the report says "take-up", or in French about a report in English.
RAG adds meaning search: an embedding model turns every passage and every question into a vector, a position in a space where texts that mean the same thing sit close together, even when their wording differs, including across the languages the embedding model supports. Merged with keyword search, which stays best at exact names, codes and figures, each finds what the other misses.
Why the overall score is not enough
Our test leans towards exact wording: most of its questions are synthetic and reuse their documents' words, which favours keyword search. Overall Elasticsearch lost 53 of 1,000; on the questions people wrote, 27 of 180. In this test, the overall score makes keyword search look stronger than its result on the questions people wrote. Across our two groups, standard RAG varied far less: its result depended much less on which questions were asked.
VectoGrid: from 27 lost to none.
Standard RAG still loses 27 of the 1,000, and leaves 72 outside the top 3. VectoGrid adds three steps, each measured as it was added:
| All 1,000 questions | Lost (not in the top 10) | Not in the top 3 | Written by people, in the top 3 |
|---|---|---|---|
| Elasticsearch out of the box | 53 | 103 | 72% |
| Standard RAG retrieval | 27 | 72 | 85% |
| + the relevant document identified | 22 | 57 | 86% |
| + a reranker reading question and passage together | 7 | 29 | 98% |
| + what each passage depends on but does not say: VectoGrid | 0 | 4 | 99% |
The relevant document
Many questions name their source indirectly, by its subject, period or publisher, while several reports hold similar figures. VectoGrid keeps a short description of every document, from its title, headings and opening, and the documents closest to the question give their passages a boost.
A reranker
The first stage finds candidates; a cross-encoder then reads each one together with the question and scores it again. The largest single step: lost questions from 22 to 7, and the people's questions found in the top 3 from 86% to 98%. Its cost depends on the number and length of the candidates it scores per question, not on the size of the library.
What a passage depends on but does not say
A passage often relies on context outside its own words: a company named at the start of a section and then called "the group", a heading that sets the region its figures belong to, an abbreviation defined pages earlier. At ingest, VectoGrid writes a short context line for each passage, without calling an AI model, carrying what the document said earlier that the passage depends on. Search reads it alongside the passage. The line helps find the passage; it is never quoted as evidence: a value is only ever supported by the passage itself.
Reading the documents well matters as much: section titles recognised as titles, tables keeping their row labels and periods. Two reading fixes made during this work apply to every row of both tables; on VectoGrid's full retrieval they rescued three questions.
What this says, and what it does not.
- "Out of the box" means BM25 with the standard settings. We ran it on OpenSearch, the open-source fork of Elasticsearch (Lucene's BM25, standard analyzer), on an earlier 960-question version of this test: 91 questions outside the top 3, against 92 for VectoGrid's own keyword index. The tables use VectoGrid's index for this row.
- Elasticsearch can do RAG too. It supports vector search, hybrid retrieval and, in its paid tiers, semantic reranking: configured that way it is a RAG retrieval itself. This compares the two approaches, not two products. We have not measured Elastic's own reranker.
- VectoGrid runs on OpenSearch. With VectoGrid's steps on top, OpenSearch gave the same results as VectoGrid's default index on the earlier test. The index stores the passages; the steps above decide which ones reach the model.
- Mostly synthetic. 820 of the 1,000 questions are synthetic and favour exact wording. The 180 written by people are the closer guide to everyday use, and they are where the gap is widest.
- The 4 still outside the top 3 are all in the top 6: a table ratio whose abbreviation its document never spells out; a French question answered just below tables about the same measure; a synthetic question tied with near-identical sections; a question whose passage comes behind contact pages bearing the name it asks about.
How the test was run
- One collection held every document; every question searched all of it. Passages of about 140 tokens, each read with its section path and its parent section.
- Embeddings: bge-m3 (open weights, multilingual). Reranker: a 1-billion-parameter open-weights cross-encoder, reading the best 40 keyword and best 40 vector results and passages of the closest documents.
- Qdrant in process for the main runs; OpenSearch 2.19 in a container for the comparison. Every configuration read the documents the same way. Questions that need evidence from several passages are measured separately. Run on 1 November 2026.
Retrieval is where a grounded answer starts. For what happens after it, see VectoGrid scores 90% on FinanceBench, with every answer and its source published; for how the index grows, From one report to millions; for a library a thousand times larger, A billion documents.
Is your search losing questions?
Published 1 November 2026 · Philippe Vergnet, VectoGrid