Object detection and class categorization in rainy environment using colour augmentation and heat map generation

Abstract

The development of autonomous systems for self-driving cars is advancing with time. These systems enable interpretability, decisions, and ethical considerations in self-driving vehicles and at the same time ensure the overall safety of these vehicles. But multiple challenges need to be addressed while developing these systems. Some of the challenges commonly observed are detecting objects in the vicinity of the vehicle in adverse weather conditions of rain, low illumination, and occlusion. The vehicles rely on input from sensors. Images captured by sensors in challenging weather conditions contain corruption due to obstructed vision and often result in inefficient object detection. The way autonomous vehicles perceive their input differs significantly from humans and there is a need to better bridge this gap with computer vision. This work aims to train the system on real rain images to better understand the real scenario as compared to simulated rain. This provides deeper insights to the developers of autonomous systems by applying and comparing enhancement techniques to capture the features in an input image. Colour augmentation and Heat map generation techniques are used in this work for object detection and class categorization in a rainy environment. A detailed analysis of these two techniques is showcased. Input image features are enhanced along with the image's quality and clarity. The average precision of the baseline models has significantly increased with a mean average precision of 97 as compared to customized RNN with 54\%, Faster RCNN with 28\% for the stylized data set, and 11\% for synthetic rain.