AlvinSMoyo/2XYDqXDc6wzA716j
Domain:
natural language processingRecord type:
model
Creator:
Alv
Host:
MonReader Cognitive Engine — a multi-modal AI pipeline (CNN • OCR • NLP • TTS) that turns a Shona hymnbook into accessible audio. Colab-first.
# MonReader Cognitive Engine: A Multi‑Modal AI Pipeline
> **Turn a physical Shona hymnbook into an interactive, accessible, and audible experience.**
MonReader is a six‑phase, end‑to‑end AI system spanning **Computer Vision (CNN)**, **OCR**, **NLP**, and **Text‑to‑Speech**. The engine detects page turns, extracts text from images, understands cross‑lingual meaning, and finally **speaks** the content in a custom‑cloned Shona voice.
---
## 🔭 Project Overview
The mission was to build a complete pipeline that could:
1. **Visually detect** page turns to trigger a scanner.
2. **Accurately extract** Shona text from scanned images.
3. **Analyze and align** meaning across Shona ⇄ English.
4. **Synthesize** high‑quality audio in a custom‑cloned Shona voice.
This was accomplished over six distinct phases, each building upon the last.
---
## 🗺️ The Six Phases of Development
### 📸 Phase 1 — Image Classification Pipeline
**Essence:** An end‑to‑end **image classification** project to flawlessly distinguish a page being turned (**flip**) vs held steady (**notflip**)—the trigger for scanning.
**Highlights**
* **Transfer Learning with CNNs:** A rigorous bake‑off across leading architectures.
* **AlvinNet + Lion Optimizer:** A lightweight, custom CNN refined with **Lion**.
* **Performance & Explainability:** Near‑perfect **F1 score**; **Grad‑CAM** verified feature focus.
---
### 📖 Phase 2 — From Scanned Page to Digital Text (OCR on the Shona Hymnal)
**Essence:** A focused **OCR** challenge on high‑resolution ELCZ Shona hymnbook images.
**Highlights**
* **Competitive OCR:** **EasyOCR** vs **Tesseract** head‑to‑head.
* **Quantitative Evaluation:** Winner selected via **Levenshtein distance**.
---
### 🧠 Phase 3 — Hybrid AI for Hymn Transcription & Analysis
**Essence:** Beyond standalone OCR—build a **Hybrid AI Pipeline** comparing **Vision‑Language Models (VLMs)** against a two‑step flow where a **PLM** corrects OCR.
**Highlights**
* **Modern Model Bake‑Off:* …