RabiesScan: A Multimodal Deep Learning Framework for Non-Invasive Rabies Detection in Dogs via CNN-LSTM Visual Analysis and Behavioural Assessment

Rabies is a deadly zoonotic virus that causes encephalitis and is thought to claim an average of 59,000 lives worldwide annually, the majority of which occur in Asia and Africa due to lack of control over their respective canine reservoirs. Current diagnostic techniques prior to death are not feasible in low-resource contexts, as they are highly lab-dependent and time-consuming. This paper introduces a new multimodal approach to non-invasive rabies detection, named RabiesScan. The proposed methodology combines two complementary sources of diagnostic information: a spatiotemporal pipeline for visual analysis of input video streams and a structured survey of the symptoms of the disease in terms of aggressive behaviour, paralysis, and excessive salivation. Visual analysis utilizes an InceptionV3 CNN as a spatial extractor of 2,048-dimensional features per input frame, after which classification is performed via a masked Long Short-Term Memory network with two layers to distinguish between the states of Normal and Rabies behaviour. The resulting classification decision together with responses to the structured questionnaire is aggregated to produce a HIGH- or LOW-Risk diagnosis alongside veterinary instructions. The whole model is deployed as a mobile application based on React Native framework and runs inference using Flask-based backend service in Python.