AppajiGouda/Satya-LLM-Hallucination-Guardrail

A Neuro-Symbolic AI guardrail that uses DeBERTa NLI and Wikidata SPARQL queries to detect and intercept LLM hallucinations in low-resource domains. # Project Satya: Neuro-Symbolic Hallucination Detection for LLMs ## 🎯 Overview Large Language Models (LLMs) inherently suffer from hallucinations due to their probabilistic nature, especially when queried on low-resource or geographically specific domains. **Project Satya** is an active "Uncertainty Gate" and firewall designed to mathematically detect, intercept, and ground factual inconsistencies before they reach the user. Instead of relying on fragile prompt engineering, this system implements a **Neuro-Symbolic architecture** that combines stochastic probability auditing with hard symbolic logic (Knowledge Graphs) to ensure high-stakes queries are factually grounded. ## 🧠 The Architecture Pipeline The system operates in four sequential stages to act as an active guardrail: 1. **The Suspect (Stochastic Interrogation):** The base LLM is forced to answer the same query multiple times at a high temperature ($T=0.9$). Factually grounded models remain consistent; hallucinating models suffer from entropy and narrative drift. 2. **The Interrogator (Semantic Consistency):** A `DeBERTa-v3` Cross-Encoder evaluates the generated samples. Instead of basic string matching, it mathematically calculates the **Contradiction Score** between outputs. High contradiction halts the pipeline. 3. **The Surgeon (Triplet Extraction):** If the model passes internal consistency, the raw text is parsed through a `Babelscape/rebel-large` seq2seq model to extract deterministic relation triplets `(Subject, Relation, Object)`. 4. **The Judge (Knowledge Graph Verification):** The extracted triplets are automatically queried against the **Wikidata Knowledge Graph** via a custom SPARQL wrapper to ensure the entities share a verified, real-world relationship. ## 📊 The Experiment: Geographical Bias Audit To test the pipeline, I designed a comparative hypothesis: *"LLMs exhibit severe geographical bias, resulting in higher hallucination rates for Indian Legal queries compared to US Constituti …