Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures

Domaine:

natural language processing

Type de record:

paperdataset
Créateur:
Ousidhoum, NedjmaMyuPerJin
Hôte:avatar
We present our shared task on evaluating the adaptability of LLMs and NLP systems across multiple languages and cultures. The task data consist of an extended version of our manually constructed BLEnD benchmark (Myung et al. 2024), covering more than 30 language-culture pairs, predominantly representing low-resource languages spoken across multiple continents. As the task is designed strictly for evaluation, participants were not permitted to use the data for training, fine-tuning, few-shot learning, or any other form of model modification. Our task includes two tracks: (a) Short-Answer Questions (SAQ) and (b) Multiple-Choice Questions (MCQ). Participants were required to predict labels and were allowed to submit any NLP system and adopt diverse modelling strategies, provided that the benchmark was used solely for evaluation. The task attracted more than 140 registered participants, and we received final submissions from 62 teams, along with 19 system description papers. We report the results and present an analysis of the best-performing systems and the most commonly adopted approaches. Furthermore, we discuss shared insights into open questions and challenges related to evaluation, misalignment, and methodological perspectives on model behaviour in low-resource languages and for under-represented cultures. SemEval-2026 Task Description Paper. Data and resources are available at \url{github.com

Visit

arxiv.org

Tasks

commonsense reasoningquestion answering

Tags

Computation and Language

Similaires

BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and LanguagesHausaNLP at SemEval-2026 Task 7: Prompt-based Hausa Cultural Question AnsweringSemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)HRM and knowledge migration across culturesVisually Grounded Reasoning across Languages and CulturesSemEval Task 1: Semantic Textual Relatedness for African and Asian Languages

BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages

Large language models (LLMs) often lack culture-specific knowledge of daily life, especially across diverse regions and non-English languages. Existing benchmarks for evaluating LLMs' cultural sensitivities are limited to a single language or collected from onli

HausaNLP at SemEval-2026 Task 7: Prompt-based Hausa Cultural Question Answering

SemEval-2023 Task 12: Sentiment Analysis for African Languages (AfriSenti-SemEval)

We present the first Africentric SemEval Shared task, Sentiment Analysis for African Languages (Afri

HRM and knowledge migration across cultures

Most discussions of knowledge, knowledge management and knowledge transfer, especially of human reso

Visually Grounded Reasoning across Languages and Cultures

The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or d

SemEval Task 1: Semantic Textual Relatedness for African and Asian Languages

We present the first shared task on Semantic Textual Relatedness (STR). While earlier shared tasks p