AbeloByte/NER_Amharic

# 🇪🇹 Amharic NER System **Named Entity Recognition for Amharic** using XLM-RoBERTa and semi-supervised learning (self-training with pseudo-labeling). --- ## 📁 Project Structure ``` NER/ ├── configs/ │ └── config.py # Hyperparameters & label definitions ├── data/ │ ├── data_loader.py # CoNLL/text loaders & NERDataset │ ├── labeled/ │ │ └── sample.conll # Sample labeled data (BIO format) │ └── unlabeled/ │ └── sample.txt # Sample unlabeled Amharic sentences ├── preprocessing/ │ ├── text_normalizer.py # Amharic-specific text cleaning │ └── tokenization.py # XLM-R tokenizer + label alignment ├── models/ │ └── ner_model.py # XLM-RoBERTa token classification ├── training/ │ ├── trainer.py # Supervised training loop │ └── semi_supervised.py # Self-training pipeline ├── evaluation/ │ └── metrics.py # Entity-level P/R/F1 (seqeval) ├── inference/ │ └── predictor.py # Single-text NER prediction ├── utils/ │ └── helpers.py # Seed, device, save/load utilities ├── templates/ │ └── index.html # Web UI template ├── app.py # FastAPI web server ├── main.py # CLI entry point ├── requirements.txt # Python dependencies └── README.md # This file ``` --- ## 🚀 Setup ### 1. Install dependencies ```bash pip install -r requirements.txt ``` ### 2. Prepare data - **Labeled data** goes in `data/labeled/` in CoNLL format (one token + label per line, blank lines between sentences): ``` አበበ B-PER አዲስ B-LOC አበባ I-LOC ውስጥ O ይኖራል O ። O ``` - **Unlabeled data** goes in `data/unlabeled/` as plain text (one sentence per line). --- ## 🏋️ Training ### Supervised Training Train on labeled data only: ```bash python main.py train --data data/labeled/sample.conll --epochs 5 ``` Options: - `--epochs N` — number of training epochs (default: 5) - `--lr 2e-5` — learning rate - `--ba …