01
PDF · MD · NotionParse
Layout-aware extraction preserves headings, tables, and source lineage.
24 sourcesAN AI ENGINEERING PROJECT BY DEVI PRASADV1.0 / 2026
RDPRASSY
RAG SYSTEM.
A retrieval system by Devi Prasad that makes every engineering decision visible—from ingestion and hybrid search to reranking, grounded citations, evaluation, and full-stack observability.
RAG0.96FAITHFULNESS
01HYBRID RETRIEVAL
02RERANK
03CITE + VERIFY
LIVE BASELINE50 GOLDEN QUERIES
162chunks indexed
0.95mean reciprocal rank
42msretrieval p95
100%citation coverage
01Live retrieval workbench
Ask it. Then inspect every decision.
The public demo runs a deterministic hybrid retriever—no API key, no fake loading, no hidden score.
INDEX HEALTHY
Try
RDPRASSY GROUNDED ANSWER
CONFIDENCE84%
The system prevents unsupported answers with three gates: every claim must trace to retrieved context, at least two passages must agree, and low-confidence requests are refused below a 0.62 rerank threshold . Retrieved text is also treated as untrusted data, so embedded instructions cannot override the generation policy . Quality is continuously checked with faithfulness and abstention metrics in CI .
Sources3 / 4 selected
02System design
From raw document to defensible answer.
Every stage exposes an artifact you can inspect, benchmark, and replace through typed provider boundaries.
02
Structure-awareChunk
420-token targets, 64-token overlap, content hashes, and deduplication.
162 chunks03
BM25 + semanticRetrieve
Two candidate lists are balanced with reciprocal-rank fusion.
top 2004
Relevance modelRerank
Coverage, semantic fit, title affinity, and freshness reduce context noise.
top 405
Grounded answerGenerate
Citation validation and a confidence gate keep unsupported claims out.
0.62 gateINGESTION_RUN_2026_07_28COMPLETED
08:42:01 discovered 24 source documents
08:42:04 parsed pdf=8 markdown=11 notion=5
08:42:08 removed 7 duplicate content blocks
08:42:12 produced 162 structure-aware chunks
08:42:19 indexed 68,124 tokens · 0 failures
08:42:19 ✓ index is query-ready
Duration18.4s
Est. cost$0.0048
Changed12%
Failures0
PLUGGABLE PROVIDERS
OpenAICoherepgvectorQdrantPineconeLocal / deterministic
03Evaluation & observations
Quality is a release gate, not a feeling.
A fixed golden set, retrieval diagnostics, and ablation runs turn “it seems better” into a decision you can defend.
Faithfulness+4.1%
0.96
vs. lexical baselineAnswer relevance+2.8%
0.91
vs. lexical baselineContext precision+11%
0.88
vs. lexical baselineRetrieval p95−18ms
42ms
vs. lexical baselineABLATION STUDY
50 golden queriesWhat actually moved quality?
| Configuration | Faith. | Ctx. precision | MRR | Observation |
|---|---|---|---|---|
| BM25 only | 0.84 | 0.68 | 0.72 | Misses paraphrases |
| + semantic retrieval | 0.89 | 0.77 | 0.86 | Recall improves |
| + fusion & reranking | 0.96 | 0.88 | 0.95 | Current baseline |
ENGINEERING NOTES
Observations from the trace.
- 01Reranking earned its latency.
+11 points of context precision for ~9ms p95 overhead.
- 02Chunk boundaries mattered more than overlap.
Heading-aware chunks beat fixed windows on procedural questions.
- 03Abstention improved user trust.
Threshold 0.62 balanced answer coverage with unsupported-claim risk.
04Builder & source
RDPRASSYDevi Prasad Choudhary
Designed, built, and measured by Devi Prasad.
I build production-minded AI systems where retrieval quality, observability, and product experience are treated as one engineering problem. This project is an open, inspectable example of that approach.
BUILDERDevi Prasad Choudhary · rdprassy
FOCUSApplied AI · RAG · evaluation · observability
INTERFACENext.js · React · Edge API
RETRIEVALBM25 · semantic · RRF · rerank
QUALITYGolden set · CI gates · feedback