thepriya-bit/NLP-Digital-Preservation-Annotation-Platform

A collaborative platform for collecting, annotating, verifying, and preserving low-resource regional languages datasets for NLP research. # NLP & Digital Preservation Annotation Platform An end-to-end platform for Assamese language annotation, translation, verification, and dataset export. Built for AI/ML dataset creation with community-driven quality control. ## Architecture ``` ┌─────────────────┐ ┌──────────────────┐ ┌────────────┐ │ React + TS │────▶│ FastAPI Backend │────▶│ PostgreSQL │ │ (Vite + TW) │ │ (Python 3.13) │ │ (DB 15) │ └─────────────────┘ └──────────────────┘ └────────────┘ │ │ │ └── Local File Storage (audio) │ ┌────┴────┐ │ Browser │ └─────────┘ ``` ## Features - **Role-Based Access**: Contributors (annotators), Verifiers, and Admins - **Audio Upload**: Record voice via browser → upload to local storage - **QA Rule Engine**: Validates Assamese Unicode, filters toxic content - **Syntax Tagging**: POS tagging, Named Entity Recognition for Assamese - **Verification Flow**: Human review with +1/-1 voting, trust score tracking - **Dataset Export**: CSV, JSON, and Parquet formats for AI/ML use - **Admin Console**: User management, orphan cleanup, platform stats ## Quick Start (Docker) — Step by Step ### Prerequisites - Docker Desktop (Windows) or Docker Engine + Docker Compose ### Step 1: Build and start all services ```bash docker compose up --build -d ``` This starts three containers: - **Frontend** (React + Nginx): localhost - **Backend** (FastAPI): localhost - **Database** (PostgreSQL 15): localhost:5432 Wait ~30 seconds for all services to be healthy. You can check with: ```bash docker compose ps ``` All three should show "Up" status. ### Step 2: Seed the database (IMPORTANT — do not skip) The database starts empty. You must run the seed script to populate it with 50 Assamese phrases and 3 default users: ```bash docker compose exec backend python scripts/seed_phrases.py ``` You should see output like: ``` Created admin user (id=1) Created user riddhi (id=2) …