A smaller vector that finds more On 445,810 passages, Qwen3-Embedding-0.6B kept to 256 bits a vector found more answer pages than bge-m3 at full length. Its query prompt had to be copied word for word.
- The question: can a vector be cut short, keeping only its first numbers, to save memory at scale without missing the answer?
- bge-m3, the embedder we use, trims poorly. Keeping its first 256 of 1,024 numbers found only half of the true nearest passages.
- Qwen3-Embedding-0.6B, the same size class, did better at every length. 109 answer pages at full length against bge-m3's 98, and 110 with 1-bit search over its first 256 numbers: a quarter of the bits, nothing lost.
- Through the whole search, keyword search and reranker included, Qwen3 put the answer page in the top 5 for 80 to 81 questions, against 66 for bge-m3, a gap unlikely to be chance (p = 0.02 to 0.03).
- The prompt is part of the model. With its model card's query instruction copied exactly, Qwen3 found 109 answer pages; with no instruction, 102; with our own rewording of it, 84.
FinanceBench: 150 questions over 368 reports, answer pages marked by people. Exact search, so the index itself plays no part, except in the full-search test.
At scale, memory is the bill.
A vector database keeps something about every passage in memory so it can search quickly. With 1-bit vectors, as VectoGrid uses by default, each passage costs 1 bit per number: 128 bytes for a 1,024-number vector, while the full vector waits on disk to rescore the best candidates.
Some embedders are trained so that the first numbers of each vector carry the most meaning (Matryoshka embeddings). Uber reported keeping 256 numbers of its vectors for under 0.3% loss of recall and nearly half the storage. We wanted to know two things: how badly bge-m3, which is not trained this way, takes a trim, and whether a model that is would do better.
Every passage, every question, exact search.
We searched all 445,810 searchable FinanceBench passages for the 150 open questions by exact search, comparing each question with every passage, so no index could blur the result. Two measures: how many of the true 10 nearest passages a trimmed vector still found, and how often the page people marked as the answer appeared in the top 40.
Each embedder was tried at full length, trimmed to its first numbers, and as 1-bit vectors over the first numbers with 40 times as many candidates rescored by the full vector.
| Setup | Bits in memory | bge-m3: answer pages | Qwen3-0.6B: answer pages |
|---|---|---|---|
| Full vector | — | 98 | 109 |
| 1-bit, all 1,024 numbers | 128 B | 98 | 109 |
| 1-bit, first 512, full rescore | 64 B | 96 | 108 |
| 1-bit, first 256, full rescore | 32 B | 95 | 110 |
| 1-bit, first 128, full rescore | 16 B | 82 | 96 |
| First 256 numbers only, no rescore | — | 89 | 98 |
Answer page in the top 40, of 150 questions. Vector search alone. Qwen3 with its model card's query prompt.
bge-m3 loses its nearest neighbours fast. Its first 256 numbers found 49% of the true 10 nearest passages, and Qwen3's 58%. Yet the answer pages held far better than those figures suggest: most of what drops out is likely passages similar to each other, not the one that answers.
The rescore is what makes the trim work. Cut the vector and keep searching with the short one, and bge-m3 loses 9 answer pages, Qwen3 11. Search with the short bits and rescore with the full vector, and Qwen3 loses none.
The lead survives keyword search and reranking.
Vectors are only the first step. VectoGrid also runs a keyword search, and a reranker scores the top 40 of each before the best are kept. So we ran that too, with each embedder: vector top 40 plus keyword top 40, reranked by Nemotron, answer page in the final results.
| Embedder | Top 3 | Top 5 | Top 10 |
|---|---|---|---|
| bge-m3 | 55 | 66 | 84 |
| Qwen3-0.6B | 64 | 81 | 95 |
| Qwen3-0.6B, 1-bit over 256, full rescore | 63 | 80 | 96 |
Qwen3 was ahead at every cut-off, even at a quarter of the bits. Each embedder still found some answers the other missed: in the top 5, Qwen3 alone found 25 and bge-m3 alone 10. In paired tests the top-5 gap is unlikely to be chance (p = 0.02 for the full vector, 0.03 for 256 bits); the top-3 and top-10 gaps point the same way without proof (p = 0.09 to 0.15).
Copy the model card's prompt exactly.
Qwen3 expects an instruction in front of each question, and nothing in front of passages. We first wrote our own version of it, "Instruct: Given a question, retrieve passages that answer the question" in place of the card's "Instruct: Given a web search query, retrieve relevant passages that answer the query". Qwen3 found 84 answer pages and looked clearly worse than bge-m3.
Before believing it, we checked our own work. Fresh vectors of stored passages matched the stored ones (cosine 0.9998), so nothing was misplaced. Then the same questions with no instruction: 102. Then with the card's exact text: 109.
| Query prompt | Answer pages, top 40 |
|---|---|
| Our rewording of the instruction | 84 |
| No instruction | 102 |
| The model card's instruction, exactly | 109 |
An instruction-tuned embedder is trained on its prompts. Change the wording and you are testing a different model.
We nearly reported that the recommended prompt hurt. It did not: our paraphrase did. A recommended setting is worth copying exactly, then testing against none, on your own documents.
Less memory than a quarter suggests.
The bits are not all that sits in memory. The search graph keeps links between vectors too, roughly as much again as today's 1-bit vector at common settings (our estimate). So 256 bits instead of 1,024 brings memory per passage from about 256 bytes to about 160: some 40% less, not 75%.
It is not faster. On a 4-vCPU Qdrant server holding the 445,810 Qwen3 vectors, a top-40 search took 13.9 ms (median) on the full vector's bits and 19.7 ms on 256 bits; eight at a time, 197 and 170 searches a second. Both rescore 1,600 candidates by their full vectors, and that, not comparing bits, sets the pace. Shorter bits buy memory, and cost a little speed.
What this says, and what it does not.
- 150 questions, one benchmark. The top-5 gap passed a paired test; the others point the same way without proof.
- English reports. Both models are multilingual, but we did not test French or other languages here.
- Slower to embed on a CPU. On the same 4-vCPU server with sentence-transformers' defaults, Qwen3 embedded 0.23 passages a second against bge-m3's 1.52, and took 1.3 s per question against 0.4 s, although the two models are about the same size. We did not find the cause. Hosted, both cost the same and Qwen3 ran the whole collection in 27 minutes. For a deployment that must embed on its own CPUs, that gap matters as much as the accuracy.
- One server for speed. Search speed was measured on one 4-vCPU server with the client on the same machine, so the rates are lower bounds; the gap between the two layouts is what matters.
- The same passages for both. Chunking was fixed, and so were the reranker and the keyword search; only the embedder changed.
- Hosted embedding. Both ran on DeepInfra. Embedding the whole collection with Qwen3 cost $0.69.
How the test was run
- 445,810 searchable passages of FinanceBench's 368 reports, parsed to markdown by our reader; each passage's stored text embedded, the same text for both models, with no prompt on passages.
- Exact search in NumPy, cosine similarity. 1-bit: the sign of each number, compared by agreement, top 400 or 1,600 candidates (40 times the 10 or 40 kept), rescored by the full vector.
- Full search: Qdrant (bge-m3 at search effort 1,600) or exact search (Qwen3) for the vector top 40, the collection's own bm25 for the keyword top 40, and Nemotron Rerank VL 1B over their union.
- Answer pages: FinanceBench's evidence pages, with the same page alignment as our earlier tests.
For how a VectoGrid index grows, see From one report to millions, and for the largest systems built so far, What is the biggest RAG ever built?
Sizing search for a large library?
Published 8 November 2026 · Philippe Vergnet, VectoGrid. Revised the same day: results with the model card's exact query prompt replace those with no prompt, and search speed measured.