sharonct/SW_SCLIP

Fine-Tuning CLIP for Swahili Semantic Video Search¶ # Fine-Tuning CLIP for Swahili Semantic Video Search ## Introduction The growing adoption of artificial intelligence (AI) in language processing has largely focused on high-resource languages, leaving underrepresented languages like Swahili with limited support. This research addresses this gap by fine-tuning the pre-trained Contrastive Language-Image Pretraining (CLIP) model to handle Swahili queries effectively. Leveraging transfer learning, the study adapts the model to understand and generate meaningful responses to Swahili text. By utilizing a specialized dataset of Swahili text-image pairs, the fine-tuned model is trained to capture the contextual and semantic intricacies of the Swahili language. The primary objective is to develop a robust system capable of performing semantic video searches based on Swahili queries. This advancement aims to enhance accessibility and usability for Swahili-speaking users, fostering inclusivity in AI technologies. Beyond semantic search, this research contributes to the broader field of low-resource language AI applications. ## Features - **Fine-tuned CLIP model:** Adapts the pre-trained CLIP architecture to understand Swahili text. - **Swahili text-image dataset:** Utilizes a curated dataset of Swahili text-image pairs to improve model performance. - **Semantic video search:** Enables users to search videos based on Swahili queries. - **Transfer learning approach:** Leverages existing AI models to minimize training costs and improve accuracy. ## Dataset The dataset comprises Swahili text paired with relevant images and video frames. This data is structured to enhance the CLIP model's ability to associate Swahili text with visual content effectively. ## Model and Training The project employs: - **Transformers Library:** Utilizing `CLIPProcessor`, `CLIPModel`, and multilingual models such as `XLM-Roberta`. - **PyTorch Framework:** Implements the training pipeline using PyTorch and its dataset handling capabilities. - * …