tarikuamisganaw/amharic-news-summerizer

# 🇪🇹 Amharic News Summarization (ByT5) An NLP pipeline for abstractive news summarization in **Amharic**, fine-tuned on the MasakhaNEWS dataset. This project leverages Google's **ByT5** (Byte-level Text-to-Text Transfer Transformer), which is highly effective for low-resource and morphologically rich languages like Amharic because it operates directly on UTF-8 bytes, eliminating the need for language-specific tokenizers. ## 🎯 Project Goals - Build a foundational multilingual AI capability for Ethiopian local languages. - Enable automated summarization for frontline health and news contexts. - Provide a baseline model that can be integrated into larger RAG (Retrieval-Augmented Generation) or Voice-to-Voice pipelines for low-bandwidth environments. ## 🛠️ Tech Stack - **Framework:** PyTorch, Hugging Face `transformers` - **Model:** `google/byt5-small` - **Dataset:** `masakhane/masakhanews` (Amharic subset) - **Evaluation:** ROUGE metrics via `evaluate` library - **Language:** Python ## 📊 Dataset & Preprocessing - **Source:** MasakhaNEWS (Amharic `amh` subset). - **Task:** Headline generation (using the article `text` as input and `headline` as the target summary). - **Preprocessing:** Inputs are prefixed with `"summarize: "` and truncated/padded to 512 tokens. Targets are padded to 128 tokens. ## 🚀 Training Configuration The model was trained using the Hugging Face `Seq2SeqTrainer` with the following optimizations: - **Batch Size:** 4 (per device) - **Epochs:** 3 - **Learning Rate:** 3e-4 - **Mixed Precision:** FP16 enabled for CUDA devices to optimize GPU memory. - **Evaluation:** Evaluated every 200 steps using ROUGE scores. ## 💻 Usage ### 1. Installation ```bash pip install transformers datasets evaluate torch sentencepiece ``` ### 2. Inference Once the model is trained and saved locally (or downloaded from the Hugging Face Hub), you can run inference: ```python from transformers import ByT5Tokenizer, ByT5ForConditionalGeneration # Load model and tokenizer …