A second, slower model that reorders the top candidates.
Made for you now: slide lessons from basics to production, usually with code and a quiz. See a finished one
First-stage retrieval is fast and recall-oriented: it pulls, say, the top 50 candidates cheaply. A reranker then reads the query and each candidate together and scores true relevance, keeping the best few.
An embedding compares two precomputed vectors. A cross-encoder reranker processes query and passage jointly, so it sees their interaction, but it must run once per candidate at query time. That is why it only runs on a shortlist.
Add reranking when the right passage is usually in the top 50 but not the top 5. Check that first on real questions; if it is not in the top 50, fix chunking or retrieval instead.
Step 1
Retrieve top-50
Vector (and keyword) search, fast.
Step 2
Rerank
Cross-encoder scores each query + passage pair.
Step 3
Keep top-5
Best passages only, in score order.
Step 4
Prompt
Pass them to the LLM as context.