On the Evaluation and Modelling of Context-sensitive Lexical Semantics

A word can change its meaning in different contexts. The evaluation and the modelling of such contextual effect on lexical meaning are pivotal to natural language understanding. This thesis sets out to answer the following two-fold research question: (1) how can we design a reliable evaluation framework that accurately reflects the challenges in contextual lexical semantics? And (2) how can we improve contextual word representations in a data-efficient manner? To address the first question, the thesis starts with systematic analysis on the existing benchmark datasets and stresses the importance to assess the complex word-context interaction. I propose that the evaluation of crosslingual correspondence can effectively assess this interaction. Specifically, I introduced two datasets, i.e. BTSR (Bilingual Token-level Sense Retrieval) and AM2ICO (Adversarial and Multilingual Meaning in Context), to evaluate crosslingual word-in-context correspondences that require the accurate crosslingual modelling of both the target words and their context. BTSR and AM2ICO complement each other in task formulations and language/word coverage. Finally, I show that evaluating and modelling crosslingual word-in-context representations have direct benefit for downstream applications. In a case study, I apply the BTSR task formulation to improve machine translation for rare senses. While improving the current state-of-the-art contextual models typically involves labelled data which is not easy to obtain especially for low-resource languages, the second research question of this thesis aims to elicit better word in context representations from state-ofthe-art contextual models without resorting to labelled data. As an outcome, I designed two novel unsupervised methods (MIRRORWIC and STATICTRANSFORM) that improve either within-word or inter-word contextualisation of the pretrained contextual models both monolingually and crosslingually. In sum, the thesis contributes to the field of computational lexical semantics by providing challenging and accurate evaluation frameworks and efficient modelling techniques that avoid the need for labelled data.