birehan/LLM-Finetuning-for-Amharic-Language

LLM Finetuning for Amharic Language # LLM-Finetuning-for-Amharic-Language ## Enabling Quality Embedding and Text Generation for Amharic Language Welcome to the LLM-Finetuning-for-Amharic-Language repository! This project is initiated by AIQEM, an African startup specializing in AI and Blockchain solutions. The primary objective is to enhance the impact of technological innovations in the Ethiopian and African business landscape. The latest flagship project, Adbar, is an end-to-end AI-based Telegram Ad solution that optimally places ads to different Telegram channels through a network of bots and extensive data analysis. ## Business Need As Telegram gains prominence as a messaging platform, AIQEM recognizes the necessity to adapt its advertising strategy to better align with this evolving ecosystem. The focus is on improving the effectiveness of promotional efforts by integrating powerful AI capabilities for Amharic text manipulation. Specifically, the project aims to create an Amharic RAG pipeline that generates creative text ad content for Telegram channels based on campaign information, including brand details and product information. The success of this project ensures that advertisements are not only catchy but also highly relevant to the Telegram community. To achieve this, the technology must possess quality Amharic text embedding and text generation capabilities. The approach involves choosing a suitable open-source Large Language Model (LLM), such as Mistral, Llama 2, Falcon, or Stable AI 2, and further fine-tuning it to meet the business objectives. ## Project Overview The project is organized into several tasks to achieve the business goals: 1. **Literature Review & Huggingface Ecosystem (Task 1):** - Understand key concepts and methods related to LLMs. - Explore the Huggingface ecosystem for inference and fine-tuning. - Review relevant literature and resources. 2. **Load an LLM and Use It for Inference (Task 2):** - Set up the work environment. - Choose an open-source LLM and load i …