lingo-iitgn/eka-eval-demo

The Framework for Multilingual Low-Resource Language Evaluation. --- # EKA-EVAL Demo EKA-EVAL Demo Evaluation Framework for Low-Resource Multilingual Large Language Models --- # Overview **EKA-EVAL** is a unified framework for evaluating **Large Language Models (LLMs)** across **low-resource multilingual languages**. Most existing evaluation frameworks focus heavily on **English and high-resource languages**, while requiring complex CLI workflows and configuration files. **EKA-EVAL solves this with a Zero-Code Web Interface**, enabling researchers to run multilingual evaluations directly from a browser. ### What the framework provides | Capability | Description | | -------------------------- | ------------------------------------------------- | | 🌍 Multilingual Benchmarks | 55+ benchmarks including 23 multilingual datasets | | 🖥 Zero-Code UI | Run evaluations without editing configs or code | | 📊 Visual Analytics | Interactive charts and model comparisons | | 🤖 AI Diagnostics | Automatic analysis of model failures | | ⚡ Modular Framework | Easily extend with new models and datasets | The **eka-eval-demo repository** provides the **complete UI-based evaluation platform**, combining a **React frontend** with a **FastAPI backend**. --- # Zero-Code Evaluation Interface The **web interface** allows users to perform full evaluations without writing code. ### Workflow ``` Select Model → Choose Benchmarks → Configure Parameters → Run Evaluation → Analyze Results ``` Users can: • select multilingual benchmarks • configure prompts and inference parameters • monitor evaluation progress • visualize benchmark performance --- # UI Features --- ## Benchmark Selection Dashboard Users can build evaluation suites by selecting benchmarks from multiple categories. Supported domains include: | Category | Examples | | --------------- | ------------ | | Reasoning | ARC, MM …