Elyablegese/Amharic-character-generation

A transformer-based model designed to generate Amharic text at the character level, trained and evaluated on . This repository contains all code, data processing scripts, figures, and results referenced in the accompanying thesis. # Amharic NLP | Transformer Model # AmharicGPT (Optimized for AWS EC2 p3.2xlarge) A character-level generative transformer model trained on Amharic text using the GPT architecture. Optimized for GPU acceleration on AWS EC2 p3.2xlarge instances with NVIDIA V100 GPUs. ### Features - Character-level text generation for the Amharic language. - Trained on a cleaned and preprocessed Amharic text corpus. - Utilizes modern deep learning techniques for sequence prediction. ### Dataset The dataset consists of a large collection of Amharic text. ## Installation ```bash # Clone repository git clone github.com cd Amharic-character-generation # Create and activate virtual environment python -m venv venv source venv/bin/activate # Linux venv\Scripts\activate # Windows # Install dependencies pip install -r requirements.txt # Preprocess data python src/preprocess.py ``` ### Training Training the Amharic text generation model on a CPU is possible but highly inefficient due to the computational demands of transformer-based architectures. For practical purposes, GPU acceleration is strongly recommended. As a benchmark, training on an NVIDIA GeForce RTX 3040 Super with the default hyperparameters (50 epochs, batch size of 64, learning rate of 1e-4) completed in approximately around 4 hours. i. Preprocess the dataset (preprocess.py): Cleans the Amharic dataset by removing noise and irrelevant content. ii. The training script (train.py) includes the following steps: 1. Model Building: Define and compile the model. 2. Data Preprocessing: Load and clean the dataset, then convert it to sequences of characters. 3. Training: Train the model using the training data, with validation on the validation set. 4. Hyperparameters such as batch size, learning rate, and number of epochs can be adjusted in the script. ``` # Train model (GPU recommended) python src/train.py \ --data-dir data/processed \ --save-dir results/checkpoints …