Investigating Cultural Alignment of Large Language Models

Domaine:

natural language processing

Type de record:

paper

Créateur:

AlkElNAlKDia

Hôte:

The intricate relationship between language and culture has long been a subject of exploration within the realm of linguistic anthropology. Large Language Models (LLMs), promoted as repositories of collective human knowledge, raise a pivotal question: do these models genuinely encapsulate the diverse knowledge adopted by different cultures? Our study reveals that these models demonstrate greater cultural alignment along two dimensions -- firstly, when prompted with the dominant language of a specific culture, and secondly, when pretrained with a refined mixture of languages employed by that culture. We quantify cultural alignment by simulating sociological surveys, comparing model responses to those of actual survey participants as references. Specifically, we replicate a survey conducted in various regions of Egypt and the United States through prompting LLMs with different pretraining data mixtures in both Arabic and English with the personas of the real respondents and the survey questions. Further analysis reveals that misalignment becomes more pronounced for underrepresented personas and for culturally sensitive topics, such as those probing social values. Finally, we introduce Anthropological Prompting, a novel method leveraging anthropological reasoning to enhance cultural alignment. Our study emphasizes the necessity for a more balanced multilingual pretraining dataset to better represent the diversity of human experience and the plurality of different cultures with many implications on the topic of cross-lingual transfer. ACL 2024 (Main)

Visit

arxiv.org

Similaires

Investigating Bias in Bulgarian in the Context of Large Language ModelsEvaluating Cross-National Value Alignment in Large Language Models: Challenges and Limitations of Survey-Based ApproachesFrom Facts to Folklore: Evaluating Large Language Models on Bengali Cultural KnowledgeTowards Multimodal Cultural Context Modeling for African Languages in Large Language ModelsLost in Translation: Safety Alignment Failures in Nepali and Code-Switched Variants of Instruction-Tuned Large Language ModelsNavigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models

Investigating Bias in Bulgarian in the Context of Large Language Models

Abstract This paper investigates the detection and annotation of bias in Bulgarian

Evaluating Cross-National Value Alignment in Large Language Models: Challenges and Limitations of Survey-Based Approaches

Large language models (LLMs) often exhibit cultural biases, raising questions about their alignment

From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge

Recent progress in NLP research has demonstrated remarkable capabilities of large language models (L

Towards Multimodal Cultural Context Modeling for African Languages in Large Language Models

This preliminary work addresses the critical gap in multimodal Large Language Models (LLMs) for Afri

Lost in Translation: Safety Alignment Failures in Nepali and Code-Switched Variants of Instruction-Tuned Large Language Models

Large Language Models (LLMs) are increasingly deployed in multilingual settings, yet safety alignmen

Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models

As LLMs are increasingly deployed in global applications, the importance of cultural sensitivity bec