Tokenization and Morphological Analysis for Malagasy

Domain:

natural language processing

Record type:

softwarepaper

Creator:

DalLiaMac

Editor:

ConKin

Publisher:

CSL

Host:

We present a tokenizer and finite-state morphological analyzer (Beesley and Karttunen, 2003) for Malagasy, based primarily on the discussion of Malagasy morphology in Keenan and Polinsky (2001) and Randriamasimanana (1986). Words in Malagasy are built from roots by means of a variety of morphological operations such as compounding, affixation, and reduplication. We analyze productive patterns of nominal and verbal morphology, describing genitive compounding and suffixa- tion for nouns, and various derivational processes involving compounding and affixation for verbs. Our work offers a computational analysis of Malagasy morphology, and forms the basis of our computational grammar and lexicon of Malagasy within the framework of the PARGRAM project.

Visit

ora.ox.ac.uk

Languages

Malagasy Malagasy, Merina

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies

Gender-inclusive NLP research has documented the harmful limitations of gender binary-centric large

VerChol -- Grammar-First Tokenization for Agglutinative Languages

Tokenization is the foundational step in all large language model (LLM) pipelines, yet the dominant

henok555/ethiopic-tokenization

# Consonantal Factorization Before Byte-Pair Encoding Improves Token Efficiency and Root Consistency

Morphological Analysis of Sahidic Coptic for Automatic Glossing

elgenio123/Malagasy-Senti-Analysis

Sentiment analysis on the Malagasy language

Reconciling morphological and molecular approaches: The integrative taxonomy of Leucobryum (Leucobryaceae) in the Malagasy region

International audience Abstract Leucobryum (Leucobryaceae) is a genus of moss (Bryoph