SOULx1s/Medical-LLM-Fine-Tuning-Model-Compression

A highly efficient LLM fine-tuning and deployment system using QLoRA and 4-bit quantization, optimized for low-resource environments. Demonstrated via a medical speech translation use-case. # Medical-LLM-Fine-Tuning-Model-Compression A highly efficient LLM fine-tuning and deployment system using QLoRA and 4-bit quantization, optimized for low-resource environments. Demonstrated via a medical speech translation use-case. ## 📌 Overview Deploying Large Language Models (LLMs) in real-world, localized environments often encounters severe hardware bottlenecks. This repository demonstrates a highly efficient, end-to-end deep learning pipeline designed to fine-tune and deploy heavy models entirely offline on medium-tier hardware (e.g., single NVIDIA T4, 12-16GB VRAM). By leveraging **Parameter-Efficient Fine-Tuning (PEFT)** and **4-bit Quantization (QLoRA)**, the system drastically reduces memory footprints without catastrophic forgetting. To validate the architecture's efficiency, it is benchmarked on a highly complex, zero-tolerance use case: **Real-Time Clinical Speech Translation (English-to-Arabic)**. ## 🔗 Hugging Face Assets (Models & Data) To maintain strict software engineering best practices, the heavy model weights and the large clinical dataset are not included in this code repository. They are publicly hosted and version-controlled on Hugging Face: * **🤗 Clinical English-Arabic Dataset:** 50,000 highly curated medical sentence pairs (split into Train/Test for rigorous evaluation). * **🤗 Fine-Tuned NLLB Adapter (QLoRA):** The optimal 10-epoch, 4-bit quantized LoRA weights ready for plug-and-play inference. ## ⚙️ Core Architectural Features * **Model Compression & Quantization:** Utilizes 4-bit NormalFloat (NF4) quantization via `bitsandbytes` to shrink the Meta `NLLB-200-distilled-600M` base model, allowing it to run concurrently with Whisper Large. * **Low-Rank Adaptation (LoRA):** Injects trainable rank decomposition matrices into the attention layers, updating only a fraction of parameters during training while keeping base weights frozen. * **Asynchronous Offline Backend:** A custom FastAPI orchestrator seamlessly handles the bidirec …