tigrem/Amharic_E-commerce_Data_Extractor

This project focuses on fine-tuning LLM’s for Amharic Named Entity Recognition (NER) system that extracts key business entities such as product names, prices, and Locations, from text, images, and documents shared across different these Telegram channels. # Amharic Named Entity Recognition (NER) Dataset Labeling This project focuses on creating a manually labeled dataset in CoNLL format for Named Entity Recognition (NER) in Amharic sales messages. The goal is to identify and tag specific entities such as **Products**, **Prices**, and **Locations** within the text. ## Project Structure (So Far) This repository contains the scripts used for: * **Task 1:** Initial dataset loading and setup for text processing. * **Task 2:** Interactive manual labeling of a subset of the Amharic messages into the CoNLL format. ## Tasks Completed ### Task 1: Data Preparation and NLTK Setup **Objective:** To load the raw dataset (assumed to be a CSV file with a 'Message' column) and prepare the environment for text processing, specifically tokenization using NLTK. **Details:** * The project expects a CSV file containing Amharic messages, with at least two columns: 'Message ID' and 'Message'. * The `nltk` library is utilized for word tokenization. Crucially, the `punkt` and `punkt_tab` tokenizers are downloaded to ensure proper functioning of `word_tokenize` for Amharic text. This step is handled programmatically at the start of the labeling script. ### Task 2: Manual CoNLL Labeling **Objective:** To manually label a subset (30-50) of the Amharic messages in the CoNLL format, a standard format for NER tasks. **CoNLL Format:** Each token (word) is placed on a new line, followed by its corresponding entity label, separated by a tab. Blank lines are used to separate individual sentences or messages. **Entity Types Used:** * **B-Product**: Beginning of a product entity. * **I-Product**: Inside a product entity. * **B-LOC**: Beginning of a location entity. * **I-LOC**: Inside a location entity. * **B-PRICE**: Beginning of a price entity. * **I-PRICE**: Inside a price entity. * **O**: Outside of any defined entity. ### Task 3: Fine Tune NER Model **Objective: Fine-Tune a Named Entity Recognition (NER) model to extract key entities (e.g …