Faysel-Abdella/Qalat-Amharic-Tokenizer

misge10/amharic-tokenizer

Amharic Segmenter and tokenizer

This is a simple script that split an Amharic document into different sentences and tokenes. If you find an issue, please let us know in the GitHub (https://github.com/uhh-lt/amharicprocessor/issues)

gashudemman/amharic-sentencepiece-tokenizer

Bonnief/amharic-nllb-tokenizer

dagim19/amharic-tokenizer-bpe

This a code to train a bpe tokenizer using the amharic alphabets and amharic letters. The trained to

Mequanent/amharic-tokenizer-fast