← Projects·AI Engineer
rag-recall
Most RAG demos show you an answer and stop there. The honest question is: did retrieval even surface the passage the answer should come from? rag-recall answers that with a reproducible scorecard (recall@k, MRR@k, hit-rate, faithfulness %, hallucination flag) over an in-repo hand-labeled set of 35 (question, gold_chunk_id) pairs.
Why this exists
A small RAG service that answers questions over a bundled doc set and ships a retrieval scorecard, so you can see whether retrieval actually found the right passage instead of taking the answer on faith.
The four metrics on every run:
- recall@k — did the gold chunk land in the top-k?
- MRR@k — how highly was it ranked?
- hit-rate — fraction of questions with the gold chunk retrieved.
- faithfulness % + hallucination flag — are the answer’s claims actually grounded in the retrieved chunks?
Reproduce in one command
python3 -m rag_recall.evaluateA reviewer clicks it: open the Streamlit app, hit Run eval to see the live scorecard, then go to the Ask tab and watch the cited chunks and faithfulness score update.
What’s transferable
The same scoring harness pattern (labeled eval set + per-query metric dashboard) drops directly into real RAG stacks — just swap the stdlib TF-IDF retriever for your embedding model + vector store and reuse the scorecard. The bundle proves the pattern without begging for an API key.