surafiel-habib/Transformer-Based-Amharic-to-English-Machine-Translation-with-Character-Embedding-and-Combined-Regul

# Transformer-Based Amharic-to-English Machine Translation with Character Embedding and Combined Regularization Techniques Amharic, the official working language of the Federal Government of Ethiopia, is a Semitic language characterized by its morphological complexity and limited digital resources. These attributes pose significant challenges for natural language processing tasks, particularly machine translation. This research introduces a Transformer-based Amharic-to-English neural machine translation model tailored to address these challenges. Key features of the model include: - **Character-Level Embeddings: Captures Amharic’s intricate morphological structure and effectively handles out-of-vocabulary words. - **Combined Regularization Techniques: Employs dropout with Elastic Net, L1, and L2 regularizations in the point-wise feed-forward network to enhance training stability and model generalization. ## Performance Highlights The proposed model demonstrates substantial improvements over previous state-of-the-art methods for Amharic-to-English neural machine translation benchmarks: - **BLEU Score: Achieved a BLEU score of 40.59, which is 7% higher than the prior state-of-the-art benchmark. - **Best Regularization: The combination of L2 regularization with dropout applied to the point-wise feed-forward network achieved the best translation performance. - **Parameter Efficiency: Reduced the model's parameters from 75 million to 5.4 million, significantly improving computational efficiency while maintaining high accuracy. ## Experimental Insights The research addresses the unique challenges posed by low-resource and morphologically rich languages like Amharic. Extensive experiments showcase: - Improvements in test accuracy and reduction in loss. - Enhanced translation fidelity compared to models using word-level embeddings. - Insights into leveraging character-level embeddings and combined regularization techniques to overcome challenges associated with low …