Adeyemi-authentic/naija-law-rag

# Naija Law RAG A document-intelligence assistant that answers questions about **Nigerian law** the 1999 Constitution, federal tax legislation, and Lagos State tenancy law **strictly from the source documents**, cites every claim to a verified span, **refuses** what the documents do not cover, and **streams** its answer behind a hardened API. It is legal *information*, not legal advice. > Built on the reusable grounded-RAG playbook (see `docs/SKILL.md`): the retrieval > brain and the production body carry over from the reference build; the corpus, > the gate threshold, the system prompt, and the evaluation set are new. ## What it does - **PDF + text ingestion** — drop `.pdf`, `.txt`, or `.md` into `rag/docs/`; the ingester extracts, de-hyphenates, and paragraph-reflows the text before indexing. - **Hybrid retrieval + reranking** — dense (Voyage embeddings) + BM25 keyword, fused by Reciprocal Rank Fusion, then re-scored by a cross-encoder reranker. - **Confidence gate** — an out-of-scope question is refused *before* any generation call (no cost, no hallucination surface). - **Cited answers** — each retrieved passage is a citable document block; the model API attaches the exact source span behind each claim, so citations are copied from the source, not typed by the model. - **Coverage check** — flags a confident answer whose grounding is thin. - **Streaming UI** — tokens stream live over Server-Sent Events; citations land when the answer completes. ## Architecture ``` INGEST (offline) docs/*.pdf|txt|md -> clean+reflow -> smart chunk -> embed (Voyage) -> Qdrant (local) / pgvector (prod) (+ BM25) QUERY (per request) question (+ history) -> contextualize (rewrite an elliptical follow-up into a standalone query) -> retrieve: dense + BM25 fused by RRF -> top-N candidates -> rerank (cross-encoder) -> top-k, best first -> confidence gate: top-1 score refuse (no model call) -> generate with native citations -> coverage check -> stream answer + verified sources ` …