Lab 2 of 6
Keyword plus vector search, then a reranker. Toggle each stage and watch hit@5 move.
BM25 matches words. Vector search matches meaning. Hybrid merges both lists. A reranker reads each candidate next to the question and reorders the top results. Each stage costs time.
Hybrid scores hit@5 0.823. Adding a reranker on top gives roughly:
Measured: hybrid 0.823 to hybrid + rerank 0.943, a gain of 0.120. Median latency went from 7.9 ms to 39.9 ms.
Switch a mode off to see the gap it closes. Source: RESULTS-rag.json, n=141, k=5.
MRR rewards the right document being first, not just present.
Rerank is the biggest quality gain here and the only stage that visibly costs latency: p50 7.9 ms to 39.9 ms. Whether that is worth it depends on your latency budget; the alert rule in lesson 6 sets ours at p95 under 500 ms.
On a separate 30-question check, a fine-tuned reranker (496 training pairs) moved hit@5 from 0.967 to 1.000. n=30 is small. Treat it as a direction, not a result to quote.