A Multi-Head Attention-Based Transformer Model for Predicting Causes in Aviation Incident

The timely identification of probable causes in aviation incidents is crucial for averting future tragedies and safeguarding passengers. Typically, investigators rely on flight data recorders, yet delays in data retrieval or damage to the devices can impede progress. In such instances, experts resort to supplementary sources like eyewitness tes- timonies and radar data to construct analytical narratives. Delays in this process have tangible consequences, as evidenced by the Boeing 737 MAX accidents involving Lion Air and Ethiopian Airlines, where the same design flaw resulted in catastrophic outcomes. To streamline investigations, scholars advocate for natural language processing (NLP) and topic modeling methodologies, which organize pertinent aviation terms for rapid analysis. However, existing techniques lack a direct mechanism for deducing probable causes. Bridging this gap, this study proposed a transformer-based model for predict- ing likely causes from raw text narrative inputs, leveraging advancements in long-input transformers. By training the model on comprehensive aviation incident investigation reports like those from the National Transportation Safety Board (NTSB), the proposed approach exhibits promising performance across key evaluation metrics, including Bilin- gual Evaluation Understudy (BLEU) with (M=0.727, SD=0.33), Latent Semantic Analysis (LSA similarity) with (M=0.696, SD=0.152), and Recall Oriented Understudy for Gisting Evaluation (ROUGE) with a precision, recall and F-measure scores of (M=0.666, SD=0.217), (M=0.610, SD=0.211), (M=0.618, SD=0.192) for rouge-1, (M=0.488, SD=0.264), (M=0.448, SD=0.257), M=0.452, SD=0.248) for rouge-2 and (M=0.602, SD=0.241), (M=0.553, SD=0.235), (M=0.5560, SD=0.220) for rouge-L, respectively. This demonstrates its potential to expedite investigations by promptly identifying probable causes from analysis narratives, thus bolstering aviation safety protocols