dagim19/amharic-tokenizer-bpe

This a code to train a bpe tokenizer using the amharic alphabets and amharic letters. The trained tokenizer is available for use at dagim/amharic_tokenizer # amharic-tokenizer-bpe This a code to train a bpe tokenizer using the amharic alphabets and amharic letters. The trained tokenizer is available for use at here