13Aluminium/Synthetic-Multilingual-Code-Switch-Data-Generation-A-Pipeline-for-Low-Resource-Languages

# Synthetic-Multilingual-Code-Switch-Data-Generation-A-Pipeline-for-Low-Resource-Languages ## Research project still going on --- ### Generation_2 #### Start-Notes on Generation_2 During the last generation, I understood that we lacked that contextual switching, so I went and tried to find a paper which might have implemented something like this, and as always chinese researcher had done something like this in 2019, for mandrin-english in the paper named , so naturally i have tried to replicate this approach, but there was a problem, they had used a dataset, which is a *Mandarin-English Code-Switching* Speech and transcript datast, but for *Hindi-English* I couldn't find anything, so I took a small dataset from kaggle just to try the approach, still this is a long-shot. #### About the Paper ---- ### Generation_1 This is a systematic approach to generate contextually accurate code-switched sentences (English-Hindi) using machine translation, transliteration, and linguistic rules. The goal is to create natural-sounding sentences where words from Hindi (embedded language) are mixed into an English (matrix language) framework, adhering to syntactic and semantic constraints. **METHODOLOGY** 1. *Translation*: We have used `googletrans` ``` def translate_to_hindi(input_sentence): translated = translator.translate(input_sentence, src='en', dest='hi') return translated.text ``` 3. *Transliteration*: Used `indic-transliteration` ---> e.g., "बाजार" → "bazaar" ``` def transliterate_to_english(hindi_sentence): return transliterate(hindi_sentence, sanscript.DEVANAGARI, sanscript.ITRANS) ``` 4. *POS Tagging & Syntactic Analysis*: Used `spaCy--en_core_web_sm` ``` def pos_tagging(input_sentence): doc = nlp(input_sentence) return [(token.text, token.pos_) for token in doc] ``` Here main rules were: - `NOUN`, `ADJ` and `PROPN` are prioritized for switching. - `VERB`, `ADP`(pprepositions), `DET`(determiners) are retained in english for the sake of preserving synta …