kkrish-tech/low-resource-language-reasoning

Evaluation framework for measuring LLM reasoning and translation performance across low-resource languages using glossary-guided prompting and semantic similarity metrics. # 🧠 Low-Resource Language Reasoning Framework An evaluation framework designed to measure how effectively Large Language Models (LLMs) reason and translate across low-resource languages. This project explores whether structured prompting strategies and linguistic guidance improve model performance on languages with limited training data. The framework evaluates model outputs using automated scoring techniques and compares reasoning quality through translation and question-answering tasks. --- ## 🧠 Technologies ### AI / NLP * Python * LLaMA 3.1 * Large Language Models (LLMs) * Natural Language Processing (NLP) * Prompt Engineering * Chain-of-Thought Prompting ### Evaluation & Metrics * SentenceTransformers * BERTScore * Cosine Similarity * Semantic Similarity Evaluation ### Data & Visualization * Pandas * Matplotlib * Regular Expressions (Regex) ### Research Concepts * Low-Resource Languages * Translation Evaluation * Interlinear Glossed Text (IGT) * Morphological Analysis --- ## ⚙️ Features ### 🌍 Low-Resource Language Evaluation Evaluates model performance across languages with limited available training data. ### 🧠 Glossary-Guided Reasoning Provides linguistic gloss information to guide translation and reasoning tasks. ### 📝 Multiple Evaluation Tasks Measures performance through: * Multiple-choice grammar questions * Open-ended translation tasks ### 📊 Automated Performance Scoring Generates evaluation metrics automatically using: * Accuracy * Cosine Similarity * BERTScore ### 📈 Visualization & Reporting Creates plots and CSV summaries to compare language performance. ### 🔄 Repeatable Benchmarking Pipeline Runs the same evaluation process across multiple datasets and languages. --- ## 🔧 The Process The project was built as a structured evaluation pipeline: ### 1. Dataset Preparation Language datasets were prepared using: * Multiple-choice grammar questions * Interlinear glossed text (IGT) * Ground-truth translations ### 2. Prompt C …