wolderufael/NER-for-Amharic-Telegram-eCommerce-Channels

# LLM's for Amharic Named Entity Recognition (NER) This repository contains scripts and data related to the fine-tuning of a Language Model (LLM) for **Amharic Named Entity Recognition (NER)**. The project focuses on processing data collected from Telegram-based e-commerce channels in Ethiopia and involves labeling, tokenizing, and scrapping product names, prices, and locations. ## Project Structure ├── .github. │ └── workflow │ └── test.yaml ├── .venv ├── .vscode │ └── settings.json ├── data │ ├── adamagebeya_telegram_data.csv │ ├── conll_format_data.txt ├── notebook │ ├── data_processing_labeling.ipynb │ └── scrap_label_tokenize.ipynb ├── script │ ├── data_processor_labler.py │ └── telegram_scrapper.py ├── .env ├── .gitignore ├── README.MD ├── requirements.txt The following is an overview of the file structure for the project: ## Contents ### 1. `.github/workflow/test.yaml` - Contains a GitHub Actions configuration for Continuous Integration (CI), specifically designed to run unit tests across multiple Python versions. ### 2. `data/` - Contains various datasets used for NER tasks: - **adamgebeya_telegram_data.csv**: Dataset scraped from the 'Adama Gebeya' Telegram channel. - **conll_format_data.txt**: Data in CoNLL format, useful for training NER models. ### 3. `notebook/` - Jupyter Notebooks: - **data_processing_labeling.ipynb**: Steps for cleaning, labeling, and processing the scraped data. - **scrap_label_tokenize.ipynb**: Notebook for scraping Telegram channels and tokenizing messages. ### 4. `script/` - Python scripts: - **data_processor_labler.py**: Script to tokenize, label and process the data for NER training. - **telegram_scrapper.py**: Script for scraping Telegram channels to gather data for the project. ### 5. `.env` - Configuration for environment variables. ### 6. `.gitignore` - Specifies which files and directories Git should ignore. ## Setup 1. **Clone the repository**: ```bash git clone github.com …