abhipratap98/LLM_project-

Explored BLOOM-1.7B’s multilingual capabilities via word embedding similarity, multilingual NLI probing (XNLI), and cross-lingual transferability. Analyzed embedding alignment, layer-wise semantic encoding, and zero-shot performance on low-resource languages. # How Does a Multilingual LM Handle Multiple Languages? ## Abstract In this study, we explore the multilingual capabilities of the pre-trained **BLOOM-1.7B** model through a series of tasks: **similarity between word embeddings across languages**, **probing for language understanding**, and **cross-lingual transferability**. The results show that BLOOM-1.7B effectively encodes semantic relationships across languages and demonstrates cross-lingual transferability, with performance depending on **language similarity** and **resource availability**. Our analysis offers insights into BLOOM’s potential for **zero-shot learning** in low-resource languages. --- ## 1 Introduction ### 1.1 Background Transformer-based LLMs have shown great progress in multilingual NLP. Understanding what linguistic information these models capture—syntax, semantics, morphology—is critical. Probing techniques allow us to analyze model representations and cross-lingual capabilities. This project investigates the multilingual abilities of **BLOOM-1.7B**, focusing on: - Word embedding similarity across languages - Probing for syntactic and semantic understanding - Cross-lingual transferability to low-resource languages --- ## 2 Related Work - Models: **BERT**, **XLM-R**, **mBART**, **mBERT**, **GPT-based** - Probing studies show higher transformer layers capture semantics; lower layers syntax - Cross-lingual transfer critical for **low-resource languages** - Embedding alignment (e.g. **MUSE**) remains a key challenge BLOOM-1.7B leverages **shared subword tokenization** and **cross-lingual pretraining**, providing a promising architecture for multilingual tasks. --- ## 3 Methodology ### Task 1: Similarity between Word Embeddings - 992 words translated into **English, French, Spanish, Chinese, Hindi** - Embeddings extracted, normalized, compared using **cosine similarity** - Results show consistent semantic alignment across languages ### Task 2: Probing to Understand Model Behavio …