Emile-Lucky-Muhigira/Multimodal-Image-Text-Misinformation-Detection

Multimodal detection uses image–text consistency to flag misinformation. This study adapts a Fakeddit-trained model to African media using a 178-example local dataset. Findings show the original model is weak on African data, but targeted adaptation significantly improves recall and F1-score without losing benchmark value. # Improving Fakeddit for Africa: Domain Adaptation for Multimodal Misinformation Detection This repository contains the final technical-group project for multimodal misinformation detection using image-text consistency features from CLIP and a lightweight logistic regression classifier. The project began with a Fakeddit-trained baseline and then adapted that same model using African-context data so that it performs better on African examples while remaining strong on the original benchmark. ## Team Group name: `Technical Team 1` Team members: - Ishimwe Karekezi Guy Gael — Andrew ID: `iguygael` - Lynne Chepkwony — Andrew ID: `lchepkwo` - Emile Lucky Muhigira — Andrew ID: `emuhigir` The repository includes: - the main experimentation notebook - the African-context dataset and image assets used for adaptation - a deployed Streamlit application for interactive prediction - paper-writing materials for proposal, midterm, and final report stages - collection and annotation documentation ## Project summary Many misleading posts do not fabricate an image completely. Instead, they reuse a real image and attach text that changes the implied location, event, actors, or meaning. This project treats multimodal misinformation detection as a semantic consistency problem: - the image is encoded with CLIP - the text is encoded with CLIP - a feature vector is built from image-text similarity and embedding differences - a logistic regression classifier predicts whether the pair is `misinformation` or `likely_consistent` Our final project focus is not just benchmark performance. It is model improvement: - start from a benchmark-trained Fakeddit model - test how it behaves on African-context data - add African training data to adapt that same model - check whether the improved model helps on Africa without hurting Fakeddit ## Main contributions - A lightweight multimodal misinformation detector built from CLIP (`ViT-B/32`) and logistic regression. - An African-context datas …