nebats2/amharic-rag

Amharic Rag AmharicRag is a ready to ship FastAPI application that processes PDF documents, detect amharic and english languages, converts them into vector embeddings, and stores them in a Dockerized Qdrant vector database. This enables accurate and efficient document retrieval to provide context for AI-powered chat prompts. Core Functions are Configuring OpenAI chat(api key) Uploading Zip file for PDF traversing, and proccessing Detecting and splitting Amharic and English texts in the document content Vectorizing and embedding to a Qdrant Database (on-premises setup) Retrieval and chat to LLM model(OpenAI) Caching and embedding QA (question and answers) based on rewards for efficient document retrieval and chat You must set and configure your OpenAI api key and model for chating. Embedding and vector store are on-premises ollama and qdrant vector db. please follow the installation steps properly Authentications The API relies on an OAuth2 Jwt token for user authentication. For simplicity, only one root user is available, and you can make changes to the username, and hashed and raw password under app/settings/.env. Once Authentication : Bearer =jwt_token token at the header is required. Uploading Documents Documents can only be uploaded as a Zip file with a pdf document type only. The zip file can have one or more sub-directories and/or pdf files inside. The uploaded zip files will be temporarly stored in the app/data/upload directory for further processing and embedding. Vectorizing and Embedding Text splitting, vectorization and embedding parameters are configuratble under the app/settings/.env . You can make changes according to the models, splitting, and embedding requirements. Ollama is the emebeddings and qdrant as main vector store, all installed and run inside the docker environment (on-premises). Before vectorization, the contents are separated to amharic and english texts for better embedding and optimal vectorization. OpenAI configuration …