omarfarouk228/togolm

First open-source AI knowledge layer for Togo - 62K+ documents, RAG API, fine-tuned LLM. Built for developers, startups and institutions in francophone West Africa # TogoLM > **The first open-source AI infrastructure focused on Togo** TogoLM is an open-source AI knowledge layer for Togo — a complete pipeline from raw web scraping to a fine-tuned LLM and public REST API that developers, startups, and institutions can build upon. --- ## What it does | Layer | Description | |-------|-------------| | **Corpus** | 62 000+ structured Togolese documents — laws, government data, press, education — from 55+ sources | | **RAG Engine** | Retrieval-Augmented Generation over the Togolese corpus | | **Public API** | REST endpoints consumable by any developer or app | | **Admin API** | Protected endpoints for corpus management, API key CRUD, query analytics | | **Fine-tuned LLM** | Mistral 7B adapted to the Togolese context (training in progress) | | **SDKs** | Official JS/TS and Python clients (`sdk/`) for the public API | ## Why Togolese public data is scattered across dozens of government portals and absent from the training sets of international LLMs. TogoLM provides a reusable, open infrastructure layer for Togo and francophone West Africa. --- ## Repository structure ``` togolm/ ├── corpus/ │ ├── scrapers/ │ │ └── spiders/ # 35 Scrapy spiders — one per source │ └── datasets/ # Scraped JSONL files (gitignored) ├── rag/ │ ├── generation/ # LangChain LCEL chains, prompts, LLM config │ ├── retrieval/ # Vector + fulltext search, query enrichment │ ├── indexation/ # Chunker, cleaner, embedder, ingestor │ └── orchestration/ # LangGraph query graph, intent classification ├── db/ # Shared PostgreSQL connection (get_conn) ├── api/ │ ├── app/ │ │ ├── main.py # FastAPI entry point │ │ ├── core/ # auth.py, rate_limit.py, models.py │ │ └── features/ │ │ ├── admin/ # router.py, service.py, schemas.py │ │ ├── auth/ # register, me │ │ ├── corpus/ # public stats │ …