EphremTil17/berana

Brana: Bridging ancient liturgy and AI. This research project fine-tunes TranslateGemma-3-12B for high-fidelity translation of low-resource Ge'ez and Amharic ecclesiastical texts into English. Features a modular pipeline including Surya-powered OCR layout analysis, custom syntax auditing via SLMs, and rigorous theological benchmarking. # 📜 Berana: Liturgical Ge'ez Translation & OCR Pipeline ## 1. Project Abstract **Berana-Trans** is an advanced, research-grade Machine Learning and Data Engineering pipeline designed to extract, translate, and benchmark low-resource Ethiopian liturgical texts (Ge'ez and Amharic) into formal English. The primary objective of this repository is to process complex, triple-column PDF manuscripts, digitize them using layout-aware OCR, and prepare "Gold Standard" datasets for benchmarking against Large Language Models (specifically `TranslateGemma-12b-it-GGUF`). ## 2. The Crucial Engineering Decision: OCR Library We have explicitly selected **Surya OCR (v0.17.1)** over Tesseract. * **The Problem:** Liturgical PDFs feature dense, triple-column layouts (Ge'ez | Amharic | English). Standard OCR reads horizontally, destroying the semantic pairings. * **The Solution:** Surya provides state-of-the-art native Layout Analysis and Line-Level Bounding Box extraction. It identifies discrete columns before applying text recognition, ensuring zero cross-column data bleeding. ## 3. Tech Stack & Dependencies (Dev Environment) * **OCR & Layout Analysis:** `surya-ocr >= 0.17.1` (Class-based Predictor API) * **GPU Compute:** PyTorch `2.10.0+cu130` (CUDA 13.0 native) * **Vision Transformers:** `transformers >= 4.48.0, = 0.129.0` * **Image Processing:** `pdf2image`, `opencv-python-headless` (for pre-processing deskew/binarization) * **Environment:** Python 3.10+, 64GB System RAM, NVIDIA RTX 3060 Ti GPU (8GB-VRAM), CUDA 13.0 Toolkit. ### Known Dependency Constraints | Package | Constraint | Reason | |---------|-----------|--------| | `torch` | `==2.10.0+cu130` | Must match system CUDA 13.0 toolkit. Install via PyTorch cu130 index. | | `transformers` | `>=4.48.0, =0.36.0, _vNN/data/auto_labels_tasks.json` Import that JSON into Label Studio for manual verification. For exact Local Files path configuration, import/export procedure, and troubleshooting, see the Label Studio workflow guide …