wilermine/Leveraging-syntactic-dependencies-in-disambiguation-the-case-of-African-American-English

# Leveraging syntactic dependencies in disambiguation: the case of African American English *Link to Prepint* **Authors:** Wilermine Previlon, Alice Rozet, Jotsna Gowda, William Dyer, Kevin Tang, and Sarah Moeller **Date: 5.10.2024** ### Project Description / Abstract: African American English (AAE) has received recent attention in the field of natural language processing (NLP). Efforts to address bias against AAE in NLP systems tend to focus on lexical differences. Whenever the structural uniqueness of AAE is considered, the solution is often to remove or neutralize the differences. This work leverages knowledge about the unique morphosyntactic structures to improve automatic disambiguation of habitual and non-habitual meanings of “be” in naturally produced AAE transcribed speech. Both meanings are employed in AAE but examples of Habitual be are rare in the already limited AAE data. Generally, representing contextual syntactic information improves semantic disambiguation of habituality. Using an ensemble of classical machine learning model swith a representation of the unique POS and dependency patterns of Habitual be, we show that integrating syntactic information improves the identification of habitual uses of “be” by about 65 F1 points over a simple baseline model of n-grams, and as much as 74 points. The success of this approach demonstrates the potential impact when we embrace, rather than neutralize, the structural uniqueness of African American English. ### Requirements See `requirements.txt` for a full list of requirements which can be installed using pip and a virtual environment. ### Running the Project This project aims to classify sentences containing a single "be" are either a habitual or non-habitual usage. #### Scripts *Habitual Rule Tagger.ipynb* should be ran first. It takes in a corpus of sentences containing a single "be" and tags classifies them according to our rule-based method. If you simply want to annotate a corp …