Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation

Domaine:

natural language processing

Type de record:

paper

Créateur:

SánEbeHelEsp

Hôte:

Large language models (LLMs) are increasingly used as scholar recommenders, shaping who is seen as an expert in academia. Existing audits remain English-centric, single discipline, and persona-agnostic, leaving the source of output variability poorly understood. To this end, we propose a benchmark that disentangles the effects of model choice and prompt design on recommendations. We audit 43 LLMs by varying persona prompts (language, location, role-and-task) and context (field, seniority, k). Recommended scholars are compared against Semantic Scholar over six scientific disciplines to measure technical quality (factuality, coverage) and social representativeness (diversity, parity). Basic technical quality is driven by model choice, factuality and parity by context, and diversity by location. South Africa prompts yield less factual lists, while Japan prompts yield highly factual but homogeneous lists skewed toward highly productive scholars. Prompt design is thus a non-trivial axis of LLM-based scholar discovery and should be systematically audited alongside model choice. 25 pages (10 main, 2 references, 13 appendix), 6 figures in main, 13 figures in appendix (under-review)

Visit

arxiv.org

Similaires

kuenane/sesotho-llm-promptingTuning LLM-based Code Optimization via Meta-Prompting: An Industrial PerspectiveA multi-LLM explainable food recommendation system based on Deep Learning Un système explicable de recommandation alimentaire multi-LLM basé sur l'apprentissage profondComparing LLM prompting with Cross-lingual transfer performance on Indigenous and Low-resource Brazilian LanguagesWhen Money Comes Up Short: Implementing Resource Coordination Systems to Redefine Planning for Economic DevelopmentPrompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languages

kuenane/sesotho-llm-prompting

Prompt engineering findings for Sesotho (Southern Sotho) using Claude — from a production remittance

Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective

There is a growing interest in leveraging multiple large language models (LLMs) for automated code o

A multi-LLM explainable food recommendation system based on Deep Learning Un système explicable de recommandation alimentaire multi-LLM basé sur l'apprentissage profond

Food recommender systems (FRS) increasingly support dietary decision-making, yet thei

Comparing LLM prompting with Cross-lingual transfer performance on Indigenous and Low-resource Brazilian Languages

Large Language Models are transforming NLP for a variety of tasks. However, how LLMs perform NLP tas

When Money Comes Up Short: Implementing Resource Coordination Systems to Redefine Planning for Economic Development

Urban planning traditionally treats money as the primary tool for e

Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languages

Partly automated creation of interlinear glossed text (IGT) has the potential to assist in linguisti