techthiyanes/bean

BEnchmarking Africa NLP (BeAN 🫘), A framework for easy evaluation of language models on several Africa NLP datasets # BeAN 🫘 BEnchmarking Africa NLP, A framework for easy evaluation of language models on several Africa NLP datasets ### List all datasets You can list all the available datasets by specifying a particular task or language; see below for example ```python3 from bean.datasets.list_datasets import list_datasets print(list_datasets(task='text_classification')) ## Returns [{'name': 'swahili_news', 'languages_covered': ['swahili'], 'available_on_huggingface': True, 'dataset_link': 'huggingface.co'}, {'name': 'swahili_news', 'languages_covered': ['swahili'], 'available_on_huggingface': True, 'dataset_link': 'huggingface.co'}] ``` ## 🫘 Onboarded Datasets ### Classification | Dataset | Website | Paper | Public | Split | Download Link| languages | Data size | | :--- | :----: | ---: | ---: | ---: | ---: | ---: |---: | | swahili_news | website | - | Yes | `train` `dev` | link | `sw` | 29545 | hausa_voa_topics | website | paper | Yes | `train` `dev` `test` | link | `ha` | 2917 | yoruba_bbc_topics | website | paper | Yes | `train` `dev` `test` | link | `yo` | 1908 | Davis/Swahili-tweet-sentiment | website | - | Yes | `train`| link | `sw` | 2263 | Iyanuoluwa/YOSM | website | paper | Yes | `train` `dev` `test` | link | `yo` | 1500