kidist-amde/amharic-ir-benchmarks
Domaine:
natural language processingType de record:
model
Créateur:
kid
Hôte:
Official codebase for the ACL 2025 Findings paper: Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval.
# Amharic-IR-Benchmarks
This repository accompanies our ACL 2025 Findings paper:
**"Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval"**
✨ We provide a reproducible benchmark suite for Amharic information retrieval, including:
* BM25 sparse baseline
* Dense embedding models (RoBERTa / BERT variants fine-tuned for Amharic)
* ColBERT-AM (late interaction retriever)
---
## 👐 Features
- **Pretrained Amharic Retrieval Models** Includes ( RoBERTa-Base-Amharic-Embd, RoBERTa-Medium-Amharic-Embd, BERT-Medium-Amharic-Embd, and ColBERT-AM for dense retrieval.)
- **Hugging Face model & dataset links for easy access**
- **Training, evaluation, and inference scripts for reproducibility**
- **Benchmarks BM25 (sparse retrieval), bi-encoder dense retrieval, and ColBERT (late interaction retrieval) for Amharic.**
- **MS MARCO-style dataset conversion script & direct dataset links**
---
## 📁 Repository Structure
```
amharic-ir-benchmarks/
├── baselines/ # BM25, ColBERT, and dense Amharic retrievers
│ ├── bm25_retriever/
│ ├── ColBERT_AM/
│ ├── colbert-amharic-pylate/
│ └── embedding_models/
├── data/ # Scripts to download, preprocess, and prepare datasets
├── scripts/ # Shell scripts for training, indexing, evaluation
├── utils/ # Utility functions
├── amharic_environment.yml # Conda environment
├── requirements.txt
└── README.md
```
---
## ⚙️ Installation
### Conda (Recommended)
```bash
conda env create -f amharic_environment.yml
conda activate amharic_ir
```
Or using pip:
```bash
pip install -r requirements.txt
```
---
## 📚 Datasets
We use two publicly available Amharic datasets:
| Dataset | Description | Link |
| ---------------- | ----------------------------------- | ---------------------------------------------------- …