rasta-nitzsche/Bert-For-Algerian-Dialect-DariBERTo

We developped 2 BERT models that can understand Algerian dialect (Darija). One model is from scratch (You should execute the code on a server for many hours to train the model as we could not do this), we called it DariBERTo and 1 pretrained model from the Arabic language. # Bert-For-Algerian-Dialect-DariBERTo ## About We developped 2 BERT models that can understand Algerian dialect (Darija). One model is from scratch (You should execute the code on TPU servers for many hours to train the model to get good results), we called it DariBERTo. One pretrained model for the Arabic language from moha. Link: huggingface.co. ## Data Collection We got the needed data by web scrapping via APIs three social networks: Facebook, Twitter, YouTube. We got over 500 000 raw data but will shorten due to the preprocessing. ## Preprocessing Delete URL's, ponctuation, repeated caracters, additionnal blanks, numbers and emojis... Normalize arabic numbers that means letters by subbing them with their correct value (.kh=5, h=7,q=9, ou=2, t=6). Some manual cleaning with excel to avoid some sentences with no sense. ## Credits CodeEmporium playlist for Transformers: youtube.com. Transformers from scratch: towardsdatascience.com. Krish Naik Deep playlist for BERT implementation: youtube.com. Search paper for BERT: arxiv.org.