ZigaLarissa/Kinyarwanda_English_Translator

# Machine Translation with LSTM - Tourism Dataset This project focuses on building a machine translation model using an LSTM (Long Short-Term Memory) neural network. The goal is to translate between two languages, English and Kinyarwanda, using a tourism-related dataset. The notebook walks through data preprocessing, model building, and evaluation. ## Table of Contents 1. Overview 2. Dataset 3. Preprocessing 4. Model Architecture 5. Training & Evaluation 6. Results 7. Future Work 8. Requirements 9. How to Run 10. Contributors ## Overview This project builds a machine translation model to convert text from English to Kinyarwanda (or vice versa). Using an LSTM-based encoder-decoder architecture, the model is trained to predict sequences in the target language based on input sequences from the source language. ## Dataset - **Input Dataset**: A tourism dataset containing English and Kinyarwanda phrases. mbazaNLP/NMT_Tourism_parallel_data_en_kin - **File Format**: The data is stored in TSV format (`tourism_train_data.tsv`) and contains columns for `source` (English) and `phrase` (Kinyarwanda). - **Preprocessing**: The text data is cleaned by lowercasing, removing punctuation, and stripping extra whitespace. ## Preprocessing Before training the model, the data goes through several preprocessing steps: 1. **Text Cleaning**: Lowercasing, removing punctuation, and extra spaces. 2. **Tokenization**: Using Keras' `Tokenizer` to convert text to sequences of integers. 3. **Padding**: Padding sequences to ensure uniform input size using `pad_sequences`. 4. **Vocabulary Size**: Vocabulary sizes for both languages are determined based on the tokenized sequences. ## Model Architecture The translation model is built using an encoder-decoder architecture with LSTM layers: - **Encoder**: Converts source sequences (English) into hidden states. - **Decoder**: Takes the encoder's states and generates target sequences (Kinyarwanda). - **Embedding Layer**: Used in both encoder and dec …