RuvaS20/zvirevo_word-embeddings

We've developed word embeddings for the Shona language using Word2Vec and FastText models, enabling powerful word similarity and analogy calculations. # Video Demo: Click on the image # Zvirevo: Shona Language Word Embeddings Zvirevo is a project aimed at bridging the digital divide for Shona speakers through advanced natural language processing techniques. We've developed word embeddings for the Shona language using Word2Vec and FastText models, enabling powerful word similarity and analogy calculations. ## Creators This project was developed by: - **Daisy Tsenesa** - **Ruvarashe Sadya** ## Key Features - Word embeddings for the Shona language - Trained using Word2Vec and FastText algorithms - Supports word similarity calculations - Enables analogy computations (e.g., "mambo" - "murume" + "mukadzi" = "mambokadzi") - API deployed on Google Cloud Run for easy integration ## Data Sources Our word embeddings were trained on a corpus of Shona text collected from various sources: 1. Shona Dictionary: A comprehensive source of formal Shona text. 2. Shona Language Corpus: Kwayedza news articles and randomly chosen websites covering a wide range of topics in Shona. 3. Belebele Dataset: A multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. 4. VOA News: A collection of Shona news articles and short stories. 5. Shona Bible and Quran We are grateful to these sources for providing the rich textual data necessary for training our models. ## Why Zvirevo? Shona, spoken by over 80% of Zimbabwe's population, is underrepresented in modern NLP technologies. Zvirevo aims to change that by providing foundational NLP resources for Shona, paving the way for more advanced applications like chatbots and virtual assistants in the future. ## Usage Our API allows developers to easily integrate Shona language processing into their applications. Use it for: - Improving search relevance for Shona content - Enhancing machine translation systems - Developing educational tools for Shona language learners - Powering recommendation systems for Shona-language e-commerce platforms ## A …