Iovann/Ayinon

Ayinon is an intelligent platform that helps citizens in Benin easily find public service information, including procedures, required documents, eligibility, and practical details through natural-language search. # 🇧🇯 Ayinon Backend **Ayinon** is a state-of-the-art **Retrieval-Augmented Generation (RAG)** assistant backend built with **NestJS**. It is designed to ingest public service descriptions from Benin's official public services portal (service-public.bj), index them using a hybrid retrieval mechanism, and answer user queries with highly grounded citations. --- ## 🏗️ Architecture & RAG Pipeline ``` [Data Source] service-public.bj │ ▼ ┌─────────┐ │ Scraping│ └────┬────┘ ▼ ┌─────────────────────────────────────────┐ │ Ingestion Pipeline │ │ - SHA-256 Hash Change Detection │ │ - Incremental sync using RunId │ └────────────┬──────────────────────┬─────┘ │ │ ▼ ▼ ┌─────────────┐ ┌─────────────┐ │ PostgreSQL │ │ Qdrant │ │(Metadata & │ │(Embeddings &│ │ Full Text) │ │ Chunks) │ └──────┬──────┘ └──────┬──────┘ │ │ └──────────┬───────────┘ ▼ ┌───────────────┐ │ Hybrid Search │ │ (Vector + FTS)│ └───────┬───────┘ ▼ ┌───────────────┐ │ RRF Fusion │ └───────┬───────┘ ▼ ┌───────────────┐ │ LLM Gen │ │ (Strict RAG & │ │ Citations) │ └───────────────┘ ``` 1. **Ingestion & Versioning**: Structured service pages (containing procedures, required documents, fees, and delays) are loaded into **PostgreSQL**. To prevent redundant vector computations, the system computes a SHA-256 `contentHash` for each service. If the hash hasn't changed, vector generation is skipped. A rolling `runId` strategy is used to clean up orphaned chunks when documents are modified and re-indexed. 2. **Queue-based Processing**: Long-running indexing processes are offloaded to **BullMQ** (powered by **Redis**) to prevent blocking HTTP endpoints. 3. **Hybrid Retrieval**: Queries are resolved using a combination of **Qdrant vector search** (semantic understanding) and **PostgreSQL full-text search** (keyword match), fused together using **Reciprocal …