Applying Hybrid Approach of Complexity Annotator Tool and Machine Learning for Lexical Complexity Detection and Simplification in Amharic Text

Text complexity refers to how difficult a document is for target readers to understand. The Amharic language includes complex and unfamiliar words that may cause low-literacy readers to misunderstand the document. In addition to human readers, such text complexity also poses challenges for NLP applications like machine translation. To detect and reduce this type of complexity for one of the Semitic family, low-resourced and morphologically reached language Amharic, we have applied a hybrid approach of complexity annotator tool and machine learning. The Amharic text complexity annotator tool is built using 1113 complex terms. Then based on the annotated dataset we have developed complexity classification and simplification models. For the classification experiment, we used 26.4k sentences to train Long Short-Term Memory, Bidirectional Long Short-Term Memory, and Bidirectional Encoder Representations from Transformers pre-trained models. To embed these sentences, we have built Word2Vec and BERT embeddings. using 9756 vocabularies. The experimental result of these models scores an accuracy of 84.1\%(LSTM), 86\%(BiLSTM), and 90\%(BERT). Based on the experimental result the BERT model has better classification accuracy, because of its ability to handle long-term information dependency. The simplification generation for the identified complex term using cosine similarity results 92\% for top-ranked and 53\% for lowest-ranked simplest equivalents from five top-generated words.