google-research-datasets/Amplify_SSA

An annotated dataset of 9,003 adversarial queries in seven Sub-Saharan African languages. # Overview Current AI models often fail to account for local context and language, given the predominance of English and Western internet content in their training data. This hinders the global relevance, usefulness, and safety of these models as they gain more users around the globe. Amplify Initiative, a data platform and methodology, utilizes expert communities to collect diverse, high-quality data to address the limitations of these models. The platform is designed to enable co-creation of datasets, provide access to high-quality multilingual datasets, and offer recognition to data authors. In partnership with local researchers situated in these countries, the pilot demonstrated an end-to-end approach to co-creating data with over 200 experts in sensitive domains (e.g., physicians, teachers, religious leaders, bankers, lawyers, human rights advocates). This approach, implemented with an Android app, resulted in an annotated dataset of 10,044 adversarial queries in seven languages (e.g., Luganda, Swahili, Chichewa)—capturing nuanced and contextual information related to key themes such as misinformation and public interest topics. This dataset in turn can be used to evaluate large language models for their safety and relevance to an African context. The dataset and accompanying data exploration Google Colaboratory notebook are provided in this repository. # Repository Folder Structure * data: Contains The primary dataset used in this project, stored as a comma-separated values file. * code: Contains a Google Colaboratory notebook providing initial data exploration and visualizations of the dataset. * taxonomies: Contains files defining the taxonomies and their corresponding definitions used in the annotations of the dataset. # Data Schema | Column Name | Type | Description | | ----------- | -------- | ----------- | | query_id | string | Unique identifier of the query. Contains 2-3 letter country code followed by a series of numbers. | | query …