Yosef-Ali/amharic-hnet-llm

# Amharic H-Net LLM & Benchmark Platform **Project Status:** Draft → Review → Final **Target Release:** Q4 2025 (December 1, 2025) **Owner:** Yosef (Developer / Orchestrator) ## 🧭 Overview This project implements an Amharic-language large language model using H-Net's dynamic chunking architecture and delivers an interactive, open benchmark platform hosted on Cloudflare. ### Purpose Build an Amharic LLM using H-Net's success in Chinese-style segmentation to create better Amharic tokenization and provide the community with open benchmark datasets and a simple UI. ### Goals - Train a proof-of-concept H-Net Amharic model using raw byte/character-level data - Evaluate performance against standard Amharic LLMs - Release open-source pipeline, datasets, and benchmark platform ### Success Metrics 1. H-Net Amharic model achieves at least 10% relative error reduction in QA/translation vs baseline 2. Benchmark platform with >3 tasks and UI hosted online 3. At least one collaborator or external tester uses the dataset within 3 months ## 🏗️ Project Structure ``` amharic-hnet-llm/ ├── data/ # Data collection and preprocessing ├── training/ # H-Net training pipeline ├── evaluation/ # Benchmark datasets and evaluation scripts ├── frontend/ # Next.js web UI ├── scripts/ # Utility scripts and automation ├── docker/ # Containerization files └── docs/ # Documentation ``` ## 🚀 Quick Start ### Prerequisites - Python ≥ 3.9 - PyTorch - Node.js and Next.js - Docker (optional) - Cloudflare Pages CLI ### Installation ```bash # Clone the repository git clone cd amharic-hnet-llm # Install Python dependencies pip install -r requirements.txt # Install Node.js dependencies for frontend cd frontend npm install cd .. ``` ## 📋 Development Phases ### Phase 1: Data & Prep (Weeks 1-2) - [ ] Amharic data scraper/collector (Wikipedia, news, public domain) - [ ] Cleaning & normali …