Dala-Innovation/DALA-TRANSLATE

An Itsekiri-to-English neural machine translation system # Itsekiri-English Translation System: Technical Documentation ## Model Architecture ### Overview The Itsekiri-English translation system uses a sequence-to-sequence (Seq2Seq) neural machine translation architecture with encoder-decoder components. The model is implemented in TensorFlow and specifically designed for the low-resource Itsekiri language. ### Components #### 1. Encoder The encoder processes the source Itsekiri text and converts it into a fixed-length context vector: - **Embedding Layer**: Converts tokenized input words into dense vector representations (dimension = 256) - **GRU Layer**: Gated Recurrent Unit that processes the sequence and generates hidden states - Units: 512 - Returns both sequences and state - Uses glorot_uniform initialization #### 2. Decoder The decoder takes the context vector from the encoder and generates the target English translation: - **Embedding Layer**: Converts tokenized target words into dense vector representations (dimension = 256) - **GRU Layer**: Processes the embedded target sequence with the encoder's context - Units: 512 - Returns both sequences and state - **Dense Layer**: Output layer that produces probability distributions over the target vocabulary ### Translation Process 1. **Input Processing**: - Text normalization (handling diacritics) - Tokenization using a pre-trained tokenizer - Padding to fixed length 2. **Dictionary Lookup**: - First checks for direct word translations in the dictionary - For compound words, attempts to translate individual parts 3. **Neural Translation**: - If dictionary lookup fails, uses the neural model - Encoder converts the input text to a context vector - Decoder generates the translation one token at a time 4. **Post-processing**: - Proper capitalization - Removing extra whitespace - Formatting the final translation ### Model Training The model is trained using: - Teacher forcing methodology - Categorical cross-entropy loss - Adam optimizer ### Tokenizers The s …