The queries vector search can't see

The last two pieces covered what embeddings measure and what chunking destroys. This one covers what's left after both of those are right: what vector search still misses even when the embeddings are good and the chunks are correctly sized, and what lexical search and reranking rescue.

The query that breaks the demo

Every vector search demo runs on queries like "health effects of smoking." Here's a query from my actual work: O. Reg. 176/26. That's the shape of an Ontario regulation citation, the kind of string my law tracker matches with a regex because it identifies exactly one thing in the world.

Embeddings are bad at it for a reason you can derive from week 3. An embedding places text near its semantic neighbors, and a registry identifier has no semantic neighbors. "176/26" appears in training data rarely and interchangeably with a thousand other number pairs. The tokenizer shreds it into arbitrary subtokens.

Two nearly identical citations, differing in one digit, refer to entirely different laws while embedding almost identically, which is the exact inverse of what retrieval needs. The user typing an identifier has maximum precision of intent, and the vector index meets them with minimum precision of representation.

The same failure covers error codes, part numbers, gene names, statute sections, ticker symbols, anything where the meaning is the exact string. In regulated domains those queries aren't the long tail, they're the daily bread.

What BM25 rewards

The standard lexical scorer is BM25, and the canonical writeup is Robertson and Zaragoza's "The Probabilistic Relevance Framework: BM25 and Beyond" (Foundations and Trends in Information Retrieval, 2009), which is worth reading once in your life because the formula is a few ideas, each visible in the demo below.

A term scores by inverse document frequency: rare in the corpus means informative, so "176/26" carries enormous weight while "the" carries none. Term frequency in the document counts, but with diminishing returns (the k1 parameter caps how much repetition can buy). And longer documents get gently penalized (the b parameter), so a term match in a focused snippet beats the same match buried in a wall of text.

Here it is running in your browser over a six-document toy corpus, with per-term score contributions shown so you can see which words carry each result:

Concept queryIdentifier queryBoth at once

Query

#1 · doc 2O. Reg. 176/26 amends the fee schedule for building permit applications effective July 1.

3.60

o.: 1.03 (idf 1.03) · reg.: 1.03 (idf 1.03) · 176/26: 1.54 (idf 1.54)

#2 · doc 4O. Reg. 214/26 revokes the previous height exception for lots fronting on a laneway.

2.06

o.: 1.03 (idf 1.03) · reg.: 1.03 (idf 1.03)

#3 · doc 1The maximum permitted height of a detached house is 10.0 metres in a residential zone.

0.00

#4 · doc 3Maximum lot coverage in a residential zone is 33 percent of the lot area.

0.00

#5 · doc 5A tall building near a residential area requires additional shadow studies before approval.

0.00

#6 · doc 6Permit fees are set out in the schedule and reviewed annually by the municipality.

0.00

Try the two preset queries. "Maximum building height" is a concept query, and BM25 does fine because the words happen to match. Then try the identifier query and watch the score concentrate almost entirely in the citation tokens, the rarest terms in the corpus, pulling one document to the top with nothing close behind it. That certainty on exact strings is what no embedding gives you, and the demo shows you why: the score is made of the rare term's rarity.

Hybrid: run both, fuse the rankings

Since the two retrievers fail on different queries, the standard move is to run both and merge. The fusion method most stacks use is Reciprocal Rank Fusion (Cormack, Clarke and Büttcher, SIGIR 2009): each document scores the sum of 1/(k + rank) across the lists it appears in, k conventionally 60. No score normalization, no tuning across incomparable score scales, just ranks.

A document that both retrievers like rises; a document only one retriever can see still surfaces. RRF's whole pitch is that it beats fancier fusion while being four lines of code, and my experience hasn't contradicted the paper.

In my research pipeline the two halves already exist side by side in the same Postgres schema: an HNSW vector index for cosine search over embeddings, and a GIN full-text index over titles and abstracts, sitting a few lines apart in the same migration. Postgres full-text ranking isn't literally BM25 (that's what dedicated engines give you), but it's the same family of lexical scoring, and having it next to pgvector means hybrid retrieval is a SQL query away rather than a second database away. That adjacency was a deliberate choice I made without fully being able to justify it at the time. This is the justification.

There's also a degenerate case of "beyond vector search" that's easy to miss because it's too simple: when the query is an identifier and your data has a key, the right retrieval is a lookup. Enacted doesn't run a search for a citation like this at all: Ontario regs resolve through the e-Laws API's own identifiers, not a full-text query. A search engine of any kind would only add ways to be wrong, and part of retrieval engineering is noticing when the problem is a join.

Reranking, and an honest accounting

The third tool is the reranker, and here's where I report rather than testify: I don't run one in production yet.

What it is: the retrievers above are bi-encoders, embedding query and document separately, forced to compress each into one vector before they ever meet. A cross-encoder reranker reads the query and a candidate together, attention flowing across both, and scores the pair. That's far too slow to run against a corpus, and fast enough to run against a shortlist, so the pattern is retrieve 50 cheaply, rerank to 10 carefully.

What it rescues is everything pairwise that single vectors compressed away: which entity did what to which, whether the passage answers the question or merely discusses it. The week 3 problem, relatedness versus assertion, gets its partial fix here, because the cross-encoder can read.

Where I'd add one first is finding-level search in NoeticMap, where the corpus is 1,536-dimensional soup of claims that are all about the same topics, and the difference between "supports" and "mentions" is what users want ranked. When I do, it goes in behind the eval harness, because a reranker is a model choice, and week 8 of this series is about what happened last time I trusted a model choice without a golden set.

Vector search for meaning, lexical search for strings, a key lookup when you have a key, fusion because queries don't announce which kind they are. None of it is exotic. The failure mode is only ever running one of them and assuming it covers the others.