abdouaziz/bert-base-wolof

A pretrained language model Bert for the wolof language # bert-base-wolof: Unsupervised Language Model Pre-training for Wolof **bert-base-wolof** is pretrained bert-base model on wolof language . ## bert-base-wolof model | Model name | Number of layers | Attention Heads | Embedding Dimension | Total Parameters | | :------: | :---: | :---: | :---: | :---: | | `bert-base-wolof` | 6 | 12 | 514 | 56931622 M | ## Using bert-base-wolof with Hugging Face's Transformers ```python >>> from transformers import pipeline >>> unmasker = pipeline('fill-mask', model='abdouaziiz/bert-base-wolof') >>> unmasker("kuy yoot du [MASK].") [{'sequence': '[CLS] kuy yoot du seqet. [SEP]', 'score': 0.09505125880241394, 'token': 13578}, {'sequence': '[CLS] kuy yoot du daw. [SEP]', 'score': 0.08882280439138412, 'token': 679}, {'sequence': '[CLS] kuy yoot du yoot. [SEP]', 'score': 0.057790059596300125, 'token': 5117}, {'sequence': '[CLS] kuy yoot du seqat. [SEP]', 'score': 0.05671025067567825, 'token': 4992}, {'sequence': '[CLS] kuy yoot du yaqu. [SEP]', 'score': 0.0469999685883522, 'token': 1735}] ``` You can try this model directly in the huggingface hub 🤗 bert-base-wolof ## Training data The data sources are Bible OT , WOLOF-ONLINE ALFFA_PUBLIC ## Contact Please contact abdouaziz@gmail.com for any question, feedback or request.