Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

On the Universality of Deep Contextual Language Models

Domaine:

natural language processing

Type de record:

paper
Créateur:
BhaGoyDanCho
Hôte:avatar
Deep Contextual Language Models (LMs) like ELMO, BERT, and their successors dominate the landscape of Natural Language Processing due to their ability to scale across multiple tasks rapidly by pre-training a single model, followed by task-specific fine-tuning. Furthermore, multilingual versions of such models like XLM-R and mBERT have given promising results in zero-shot cross-lingual transfer, potentially enabling NLP applications in many under-served and under-resourced languages. Due to this initial success, pre-trained models are being used as `Universal Language Models' as the starting point across diverse tasks, domains, and languages. This work explores the notion of `Universality' by identifying seven dimensions across which a universal model should be able to scale, that is, perform equally well or reasonably well, to be useful across diverse settings. We outline the current theoretical and empirical results that support model performance across these dimensions, along with extensions that may help address some of their current limitations. Through this survey, we lay the foundation for understanding the capabilities and limitations of massive contextual language models and help discern research gaps and directions for future work to make these LMs inclusive and fair to diverse applications, users, and linguistic phenomena. 9 pages

Visit

arxiv.org

Tasks

language modelingtransfer learning

Tags

Computation and Language

Similaires

Contextual Phenotyping of Pediatric Sepsis Cohort Using Large Language ModelsContextual Evaluation of Large Language Models for Classifying Tropical and Infectious DiseasesLocalised Contextual Large Language Models (LLM’s) for Personalised Medicine in AfricaThe universality of conversational postulatesOn the non-universality of tonal association ‘conventions’: evidence from CiyaoDetecting Cyberbullying on Twitter Using Natural Language Processing Techniques and Deep Learning Models

Contextual Phenotyping of Pediatric Sepsis Cohort Using Large Language Models

Clustering patient subgroups is essential for personalized care and efficient resource use. Traditio

Contextual Evaluation of Large Language Models for Classifying Tropical and Infectious Diseases

While large language models (LLMs) have shown promise for medical question answering, there is limit

Localised Contextual Large Language Models (LLM’s) for Personalised Medicine in Africa

The universality of conversational postulates

ABSTRACT Grice's analysis of conversational maxims and implicatures is examined in the light of Mal

On the non-universality of tonal association ‘conventions’: evidence from Ciyao

One of the major aims of linguistic theory is to determine what is universal vs . language-specific

Detecting Cyberbullying on Twitter Using Natural Language Processing Techniques and Deep Learning Models

Detecting Cyberbullying on Twitter Using Natural Language Processing Techniques and Deep Learning Models 

Poster presented at the Deep Learning Indaba 2023 by Zandile  Shabangu