Potholes present a substantial hazard to road safety, resulting in accidents and impeding the smooth flow of traffic. This issue is particularly salient in developing nations such as Nigeria, where proactive and effective pothole management is imperative. The present study addresses this challenge by advocating a pioneering methodology employing an Enhanced Faster R-CNN algorithm that amalgamates EfficientNet and Faster R-CNN techniques. The primary objective of this model is to enhance pothole detection accuracy, with a specific focus on facilitating the operations of autonomous vehicles within environments characterized by resource limitations. By harnessing the efficiency of the Lightweight Faster R-CNN in conjunction with EfficientNet, the proposed model attains a notable accuracy rate of 97.7%. This performance surpasses that of established architectures including MobileNetV2, ResNet50, VGG16, and Inception V3. These findings underscore the efficacy of the model in real-time pothole detection, thereby underscoring its potential to substantially ameliorate road safety and traffic management in developing regions.