KOSbassem/BERT-based-Q-A-Approach-for-MSA-Multi-Dialect-and-Tunisian-Data

This repository implements a question-answering system for three Arabic corpora: MSA, multi-dialectal, and Tunisian Arabic. It uses AraBERTv2, XLM-Roberta, and TunBERT with NER and semantic analysis to improve Q&A in healthcare contexts, covering data preprocessing, training, and evaluation. # Arabic Question Answering System (MSA & Tunisian Dialect) This project implements BERT-based Question Answering (QA) pipelines for Arabic language, covering: - Modern Standard Arabic (MSA) - Mental Health Arabic QA - Tunisian Dialect Health Corpus The system integrates semantic analysis and Named Entity Recognition (NER) to improve answer accuracy. --- ## 1. System Architecture The proposed architecture follows a modular Question Answering (QA) pipeline designed to overcome the input length limitation of BERT-based models (512 tokens) while preserving semantic relevance. The system combines **semantic analysis** and **Named Entity Recognition (NER)** to construct optimized input sequences for Question Answering. ### Architecture Workflow 1. **Input Layer** - User question - Textual context (document, transcript, or answer candidates) 2. **Segmentation** - The context is split into smaller semantic segments. - This step allows fine-grained relevance analysis instead of linear truncation. 3. **Semantic Analysis** - Each segment and the question are encoded using Sentence-BERT embeddings. - Cosine similarity is computed between the question and each segment. - Segments are ranked based on semantic relevance. 4. **Named Entity Recognition (NER)** - NER is applied to identify critical entities (persons, locations, medical terms). - Segments containing key entities are prioritized to preserve factual information. 5. **Sequence Construction** - The final input sequence is built by concatenating: `[CLS] Question [SEP] Selected Segments [SEP]` - Segments are added incrementally until the 512-token limit is reached. 6. **Question Answering Model** - The optimized sequence is fed into BERT-based models (AraBERTv2, XLM-Roberta, TunBERT). - Depending on the corpus: - Answer span prediction is used (MSA corpus) - Sequence classification is used (MentalQR and Tunisian corpus) This architecture ensures that the most relevant and informative content is preserved, i …