chongj23/shona_mandarin_dictionary_prototype

A interface build by python streamlit, retrieve data to approach function of dictionary. # Shona-Mandarin Dictionary Prototype An end-to-end NLP dictionary system that provides cross-lingual retrieval, segmentation, and vector embeddings between Shona and Mandarin Chinese. --- ## 📌 Project Overview This repository contains preprocessing code, segmentation methods, vector embedding pipelines, evaluation notebooks, and a web interface for searching and querying Shona-Mandarin translations. Raw datasets and corpora used by the project are **not included in this repository**. They should be downloaded separately from their original sources as described below. --- ## 📁 Repository Structure ```text ├── Database/ │ └── mandarin_shona_dictionary_database.7z # Shona and Mandarin database │ ├── Preprocessing/ │ ├── shona_preprocessing.ipynb # Shona text cleaning and normalization │ ├── shona_segmentation_methods.ipynb # Segmentation algorithm experiments │ ├── shona_embedding.ipynb # Vector embedding generation for Shona │ ├── mandarin_preprocessing_v2.ipynb # Mandarin text tokenization & processing │ ├── mandarin_embedding_v2.ipynb # Vector embedding generation for Mandarin │ ├── shona_segmentation_boundary_evaluation.ipynb # Evaluation of Boundaries Shona Segmented Word │ └── shona_mandarin_evaluation.ipynb # Evaluation metrics and model performance │ └── Interface/ ├── app.py # Main application entry point (Streamlit / Flask) ├── db.py # Database connection & query execution ├── retrieval.py # Vector similarity search and retrieval logic ├── segmentation.py # Master segmentation handler ├── shona_segmenters.py # Custom Shona word/morpheme segmenter implementations ├── flatcat_model.tar.gz # Trained segmentation/morphological model ├── requirements.txt # Python dependencies └── env.example # Environment variables template ``` …