Ephrem0001/Amharic-document-QA-system

# Amharic Civil Service Document QA (RAG + Fine-Tuned Embeddings) RAG-based Q&A system for Amharic civil service documents: FastAPI, PostgreSQL, FAISS, and **open-source only** embedding/generator models. No paid APIs. Supports embedding fine-tuning and evaluation with standard IR metrics. ## Open-source only This project uses **only open-source models** (no commercial or paid APIs): | Component | Model | License | Link | |------------|--------|---------|------| | Embedding | `intfloat/multilingual-e5-small` | MIT | Hugging Face | | Generator | `Qwen/Qwen2.5-1.5B-Instruct` | Tongyi Qianwen License 2.0 (permissive) | Hugging Face | Override with env vars `EMBEDDING_MODEL` and `GENERATOR_MODEL`; use only models with permissive open-source licenses. **Selected models (defaults):** Embedding = `intfloat/multilingual-e5-small`, Generator = `Qwen/Qwen2.5-1.5B-Instruct`. No need to pick others unless you want to experiment; see Model selection for alternatives. ## Getting started 1. **Ingest PDFs from `docs/` folder:** ```bash python scripts/ingest_docs_folder.py ``` Place your Amharic PDFs in `docs/`, then run this to index them all. 2. **Or create mock PDFs** (if you have no documents yet): ```bash python scripts/create_mock_pdfs.py ``` This writes `docs/mock_civil_service_1.pdf` and `docs/mock_civil_service_2.pdf`. Upload them via `POST /documents/upload`. 3. **Build eval questions** (no chunk IDs needed at first): - List chunk IDs after upload: `python scripts/build_eval_questions.py --list-chunks` - Generate a template with `expected_pages` only: `python scripts/build_eval_questions.py --from-pages --out data/evaluation/eval_questions.jsonl` - Edit the file: set `document_id` (from upload response) and add more questions. - Backfill chunk IDs from the DB: `python scripts/build_eval_questions.py --backfill-chunk-ids --input data/evaluation/eval_questions.jsonl --out data/evaluation/eval_questions_with_chunks.jsonl` - Use the `--out` file for evaluatio …