haftomdesbele/TRANSFER-LEARNING-AND-KNOWLEDGE-DISTILLATION-for-CHEST-X-RAY-CLASSIFICATION

TRANSFER LEARNING AND KNOWLEDGE DISTILLATION for CHEST X-RAY CLASSIFICATION, This makes the approach particularly suitable for deployment in low-resource healthcare settings, where computational efficiency and reliability are crucial **Transfer Learning and Knowledge Distillation for Chest X-ray Classification** --- **Overview** --- This project uses two powerful and efficient deep learning approaches for chest X-ray classification. A combination of Transfer Learning and Knowledge Distillation to create a high-performing and lightweight model. This makes the approach particularly suitable for deployment in low-resource healthcare settings, where computational efficiency and reliability are crucial. **Introduction** --- Deep Learning techniques like CNN come with 2 problems: Data Scarcity: Training a deep learning model from scratch on limited labeled medical data can lead to poor performance and overfitting. Deployment Constraints: Large, high-performance models are often too slow or resource-intensive for deployment on edge devices or in clinical settings with limited computational power. Our solution addresses these challenges by employing a two-stage approach: Transfer Learning: We fine-tune a pre-trained, large-scale model (the "teacher") to achieve high accuracy on the classification task. This leverages the extensive knowledge the model has already learned from a massive, general-purpose dataset. Knowledge Distillation: We then use the knowledge from this large teacher model to train a smaller, more efficient "student" model. The student model learns to mimic the teacher's nuanced predictions, resulting in a lightweight model that retains a high degree of performance, making it suitable for deployment. **Features** --- Transfer Learning: Fine-tuning of state-of-the-art architecture (ResNet-50) on a chest X-ray dataset. Knowledge Distillation: Implementation of a distillation pipeline to compress a large teacher model into a compact student model. Comprehensive Evaluation: Detailed comparison of the performance of the baseline, teacher, and student models using relevant medical diagnosis metrics (e.g., Accuray, F1-Score, Precision, and Recall). Base …