Enhancing Autonomous Driving in Adverse Weather: Road Surface Classification and Image Restoration Evaluation

 Corresponding author: Prayag Gaonkar Copyright © 2025 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution License 4.0. Enhancing Autonomous Driving in Adverse Weather: Road Surface Classification and Image Restoration Evaluation Prayag Gaonkar * Dougherty Valley High School, San Ramon, CA 94582 USA. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 Publication history: Received 21 October 2025; revised on 29 November 2025; accepted on 01 December 2025 Article DOI: https://doi.org/10.30574/wjarr.2025.28.3.3992 Abstract The integration of artificial intelligence into transportation has pushed for the development of autonomous vehicles, promising improved safety and convenience. Despite numerous advancements, however, autonomous vehicles remain unreliable in adverse weather conditions including rain, fog, snow, and darkness. Current methods of scene mapping rely on LiDAR technology, which tends to fail when exposed to weather that distorts its signals. This issue emphasizes the need for robust image-based systems for suboptimal driving conditions. Two models, CLIP and ResNet, were selected, both of which offer unique approaches toward image-based weather classification compared to conventional models such as convolutional neural networks (CNNs). Using a self-compiled custom dataset of 4800 road images in varying conditions, each model was trained and tested. ResNet-50 was the most effective model, reaching an accuracy of 0.95 on the testing data set. The predictions of road surface weather from the models were compared to empirically determined data to estimate the coefficient of friction, which can be used to maximize safety. Furthermore, an image restoration model—a model that removes weather effects such as raindrops—was analyzed and its performance was measured quantitatively. Algorithmically measuring the confidence of object detection numerically showed the improvement from the original images to the restored images, serving as a reliable evaluation technique. This concept could be used as an optimizer for these models to maximize their performance. Overall, this research reveals the potency of previously unused techniques for the development of autonomous driving and serves as a foundation for future developments involving adverse weather. Keywords: Autonomous Vehicle Safety; Image Classification; Image Restoration; Object Detection 1. Introduction The rapid advancement of technology, particularly in artificial intelligence (AI), has fostered the diversification of various sectors, including transportation. Though the development of autonomous automobiles has promised improved safety, accessibility, and efficiency, there remain glaring challenges that need to be addressed in order to make the usage of autonomous vehicles a lifestyle, rather than a liability. One of the most pressing challenges is their vulnerability to adverse weather conditions. According to the US Department of Transportation, approximately 21% of automobile crashes (~1.2 million/year) in the US are weather-related [1]. Meta-analyses show that crash rates climb by ~71% during rain and ~84% during snow [2]. As the world shifts closer towards fully autonomous transportation, it is essential that self-driving vehicles can detect and respond accordingly to adverse weather conditions, such as snow, rain, and fog. Failure to appropriately account for these conditions compromises safety and efficiency, preventing organizations such as the National Highway Traffic Safety Administration (NHTSA) from approving the deployment of autonomous vehicles on public roads [3]. Though millions of dollars and hours have been put into the development of autonomous vehicles, relatively little has been done to prepare these vehicles for abnormal weather conditions. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 31 Weather-related safety and efficiency concerns will undoubtedly hinder the widespread use of these automobiles as it mitigates its practicality. This research aims to address key limitations in current vehicle sensing and decision-making frameworks under the unfavorable weather conditions of rain, fog, snow, and darkness. These circumstances require different decisions from the autonomous system, depending on the severity and type of weather—a factor often underrepresented in prior research. Most existing weather-detection models tend to classify weather based on environmental features (e.g., sky conditions) rather than the actual road surface—an approach that can be dangerously misleading. For example, while the sky may appear clear, the road could still be wet and slippery. Additionally, unusual weather circumstances call for more advanced object detection systems due to poor visibility in cameras due to precipitation. With unclear images, autonomous systems will be less aware of the surroundings, potentially compromising safety. Current autonomous vehicles in production and development mostly rely on LiDAR (Light Detection and Ranging) technology, which does not address either of these two problems. LiDAR systems create high-resolution 3D maps of environments, which are used by the vehicles to make decisions [4]. The technology functions by emitting laser pulses and tracking the time it takes to hit an object and reflect back, allowing it to determine the shape and distance of objects around it [4]. However, precipitation and harsh weather involving raindrops and snow distort the lasers emitted and thus significantly interfere with the resulting 3D map produced, preventing the vehicle from sensing the objects around it [5]. Finally, LiDAR is unable to determine the surface condition of the road (e.g., dry, wet, or icy) and therefore does not provide an autonomous system with sufficient information to appropriately adjust vehicle speed in response to reduced traction and friction [6]. LiDAR’s struggles in poor weather result in the need and increased reliance on imagebased systems for autonomous vehicle development. One of the primary objectives of this study is to develop a robust image‐based classification model capable of distinguishing the five road conditions (rain, fog, snow, clear, night) with high accuracy. To improve upon past work, the models were tailored to two key aspects that were previously overlooked: • Specificity: the ability of the model to only consider the condition of the road and not external factors. • Adaptability: the models should be robust for nearly any scenario in any location, so they should not be overfit to any particular trait. Once the most effective model is developed and tested, the outputs of the classifier can be integrated with physical models of tire–road interaction: by applying empirically determined coefficients of friction for each road surface category, the system could compute and recommend proportional reductions in vehicle speed to maintain safe stopping distances. These adaptive recommendations could be sent to the longitudinal controller of an autonomous vehicle to make real-time adjustments. In this study, two major groups of models were trained and tested to classify road surface condition, namely ResNet and CLIP. Figure 1 Chart of vehicle stopping distance based on speed; fails to consider surface of road and its friction coefficient To support the development of these models, a diverse and curated dataset of road weather images was compiled from various publicly available sources. However, despite the widespread research and data availability in the field of World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 32 autonomous vehicles, there is little publicly available data in the form of images of weather conditions specifically tailored to vehicles or roads. Therefore, the data used to train the models was collected and combined from a variety of sources, possibly leading to biases in the classification of the images. For example, one dataset might classify a road during twilight time as normal lighting, while another might classify a similar image as nighttime. The limitation of having a lack of standardized and reliable data will reduce model accuracy and will make it less robust to new, unfamiliar weather images and scenes. However, to mitigate geographic and infrastructural bias, the dataset incorporates samples from multiple regions, encompassing a wide range of lighting, weather intensities, and road environments. Moreover, the complied dataset was hand-filtered to ensure that there are no major mis categorizations. The data compiled from the different sources was shuffled and used to train and test the various models for image-based weather detection. The other primary objective of this study is to establish a standardized evaluation framework for existing image‐ cleaning (de-weathering) models. These models aim to restore scene visibility and enhance the clarity of images captured by vehicle-mounted cameras, thereby improving downstream tasks such as object detection, lane following, and semantic segmentation. The lack of consistent benchmarking criteria makes it difficult to compare models objectively or identify the most effective methods for specific weather conditions. The proposed evaluation framework seeks to address this gap by introducing a quantitative metric that assesses not only image restoration quality (e.g., PSNR, SSIM, LPIPS) [7,8], but also task-specific performance, such as object detection accuracy on restored frames. In addition to providing a fair basis for comparison, the framework will be designed as an optimization utility for researchers developing new de-weathering models. By integrating the evaluation algorithm with automated hyperparameter tuning tools (e.g., grid search, Bayesian optimization, or reinforcement learning-based tuning), developers can iteratively adjust network architecture and training parameters to maximize real-world performance under targeted environmental conditions. This approach supports both model-level refinement and broader research progress by enabling reproducible, data-driven improvement of perception systems in autonomous vehicles operating under visually degraded conditions. Together, these proposed frameworks of comprehensive object detection and context‐aware speed adaptation will equip autonomous platforms with the necessary means to perceive and respond to challenging weather scenarios more reliably, leading to safer and more efficient driving behaviors. 2. Methods The first component of this research involves a comparative analysis of underutilized models for image-based weather classification. Although originally designed for general image classification tasks, these models were reprogrammed and optimized to address the specific requirements of weather classification. The second component is exploratory in nature, aiming to establish a method for evaluating image restoration models, given the lack of standardized quantitative performance metrics. Developing a reliable evaluation technique is essential for the effective optimization and practical deployment of both classification and restoration models. 2.1. Data collection and image processing Sufficient and usable data is critical to maximizing the efficiency and accuracy of any machine learning model. However, at the time of research, there was no publicly available dataset of road images with image labels corresponding to varying weather or visibility conditions. As a result, it was key to filter through and preprocess the images in the dataset to maximize the performance of the models. • Dataset Creation: Compiled a diverse dataset of 4,800 images (1000 images for the normal, fog, and night classes; 900 images for the rain and snow classes) by collecting images from a variety of open-source databases. Images that did not fit with the intended classes (i.e. dust storm) or lacked usability (i.e. hard for even a human to classify) were removed from the dataset and replaced with another. This technique of image cleaning is important to improve model prediction accuracy. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 33 Figure 2 Distribution of Images in Custom Dataset [16,17,18,19,20] • Data Segmentation: The dataset was programmatically shuffled and then split into a 85:15 ratio of training to testing images using the train_test_split() function of the Scikit-learn Python library. The resulting split datasets were saved to separate JSON files to ensure that each model was trained and tested on the same sets of images, minimizing bias. • Image Preprocessing: The images were normalized to the same shape, a term that generally refers to three components of an image: the height, width, and channels. All images for standard neural networks were converted to (256, 256, 3), which indicates a 256x256 pixel ratio and three channels, converting all images to RGB. Standardizing the images is critical for the models to extract features and patterns. 2.2. Weather model selection and implementation 2.2.1. CLIP (Contrastive Language-Image Pretraining): CLIP is a pretrained neural network (trained on large image-caption datasets using a contrastive objective that maximizes the similarity between correct image–text pairs while minimizing the similarity of mismatched pairs) developed by OpenAI that learns visual distinctions and similarities in an unsupervised manner [21]. It combines Natural Language Processing (NLP) techniques with traditional Computer Vision (CV) techniques to make class names/descriptors hold significance in the prediction algorithms. CLIP is a dual-encoder multimodal model that uses one encoder for images (e.g., a Vision Transformer or ResNet) and one for text (a Transformer-based language model), trained jointly to align image and text embeddings the shared latent space [21]. This enables “zero-shot” capabilities, recognizing objects or concepts without task-specific training based on descriptive text prompts. Given an image and a set of text descriptions, CLIP can output a similarity score of each description to the image without any additional training [21]. CLIP’s versatility compared to standard supervised models makes it extremely suitable for autonomous vehicles and weather classification, as it is essential that a model can adapt to new, unforeseen circumstances. To improve performance from the pre-trained version, the CLIP model was fine-tuned by training it on the custom dataset that was created. The images were preprocessed using the CLIP ViT-B/32 Preprocessor (an image encoder) and set to 512 “features”, the dimensionality of each image embedding. In other words, this is a tensor of 512 numbers that correspond to a specific image, which is produced from a series of manipulations to the original multidimensional vector representing the image. Then, these image embeddings are compared to the text embeddings (also a 1x512 tensor) of descriptors of each of the five classes (normal, rain, snow, fog, night). An integer value one to five is outputted, corresponding to the class with the text embedding tensor that has the highest similarity to the image embedding tensor [21]. This value is compared to the correct image label, and the CLIP model then receives feedback regarding the correctness of its prediction. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 34 Figure 3 Diagram of CLIP’s Shared Text and Image Embedding Space Figure 4 Methodology of CLIP’s Zero-shot Prediction Capabilities 2.2.2. ResNet (Residual Network): ResNet is a type of Convolutional Neural Network (CNN) and is pretrained on the ImageNet dataset, with over 14 million images. CNNs are especially effective with image classification but tend to struggle with two key problems. • Vanishing Gradients: Gradients, which are indicators of a model’s loss, become very small as layers to the network are added, resulting in little to no learning. During backpropagation, small numbers outputted from functions are multiplied to the gradients and cause their value to diminish over time. After a certain point, the gradient value stops changing, causing the optimizer to stop learning, since there is no change in loss [22]. • Degrading Accuracy: Contrary to intuition, many CNNs have lower accuracy when there are too many layers in the model. This problem is partially caused by overfitting, the concept that an excess number of parameters causes the network to simply “memorize” the training data rather than extracting patterns, resulting in poor performance on new data. Having a surplus of layers can also hurt optimization efficiency [22]. ResNet models, however, tackle both problems. The key mechanism of residual networks is that they learn to compute the change between the input (image) and output (road condition) [22], making it more efficient than in a normal CNN, where the model must learn how to produce the entire output from a given input. Therefore, in contrast to regular CNNs, residual networks have layers that can add directly to the final output by using residual (skip) connections, the ability World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 35 to bypass layers to change the output [22]. In a ResNet model, each layer can contribute to the difference (residual) individually, but in a normal CNN, layers “work together” to produce the output. This feature ensures that each layer is learning and directly contributing to the output, preventing any override by another layer down the line [22]. In other words, each layer is independent, so having a sophisticated model with numerous layers will not cause either of the two problems mentioned above. Figure 5 Structural Differences Between Residual Networks and Standard CNN Models 2.2.3. Model learning and evaluation Loss functions provide quantitative data regarding a model’s performance, guiding the optimizer to improve predictions by minimizing the difference between true values and predictions. The cross-entropy loss function is particularly strong with probabilistic models and is gradient-friendly, making it suitable for both CLIP and ResNet models. Furthermore, the cross-entropy loss function is strong with nonbinary classification, which is required for this task [23]. Specifically, the cross-entropy loss function is defined by 𝐻(𝑝,𝑞)=−∑𝑞𝑘log(𝑝𝑘), where qₖ is the one-hot encoded true label and pₖ is the predicted probability from the SoftMax output [23]. Minimizing cross-entropy encourages the model to assign high probability to the correct class and is especially effective in deep networks due to its smooth, well-behaved gradients. The cross-entropy loss function was paired with the AdamW optimizer, which has many key features tailored to sophisticated, multi-layer image classification in comparison to the standard Adam optimizer [24]. For instance, the AdamW features decoupled weight decay, which prevents weights from growing too large, thereby improving generalization [24]. Since it is “decoupled” from the optimization step, it can compute adaptive learning rates which helps minimize overfitting [24]. The models were evaluated primarily on two metrics, accuracy and loss. Each metric was measured individually for each model on both the testing and training sets. The values were computed at regular intervals of each epoch (one epoch is one complete iteration of the training data through a model), for a total of ten measurements (ten epochs were run for each model). Then, the models were compared and ranked for accuracy based on the validation data set of 720 images. 2.2.4. Mapping speed control to road surface condition To ensure safe and adaptive driving under adverse weather conditions, the output of a road weather classification model is integrated with a physics-based framework for speed control, which could potentially be used in a longitudinal controller in an autonomous vehicle. Each predicted weather class (normal, rain, snow, fog, night) is mapped to an estimated coefficient of friction (μ) based on empirical values reported in transportation safety literature. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 36 Table 1 Mapping of Road Surface Conditions to Plausible Ranges for the coefficient of friction (μ) Road Surface Condition Additional Driving Info Coefficient of Friction (μ) Normal (Dry) Asphalt/Concrete 0.75-0.85 Rain Wet/Slick Road 0.25-0.50 Snow/Ice Covered 0.05-0.25 Fog Lower Visibility 0.60-0.75* Night Lower Visibility 0.65-0.80* *Friction is the same as normal conditions for fog and night, but the stopping distance is higher due to reduced visibility. For calculation, μ is reduced proportionally. Based on the estimated friction coefficients, the system can compute and recommend proportional reductions in vehicle speed to maintain safe stopping distances by the equation 𝑑 = 𝑣2 2𝜇𝑔 , where 𝑑 is the braking distance, 𝑣 is the vehicle speed, 𝜇 is the friction coefficient, and 𝑔 is the gravitational constant. Ideally, additional factors, such as turning radius (if the vehicle is turning) and tire condition (wear and age), should be considered by the system for speed adjustments to maximize safety whilst maintaining efficiency. 2.2.5. Evaluating image restoration model performance raindrops and snow, from images. However, currently, there are no proven methods to quantitatively measure the performance of these models because it is impossible to produce images in the exact same scenes without the poor weather that the restored images can be compared to. Therefore, it is critical to go back to the original purpose of the de-weathering model, which is to make scene recognition from images more accurate. In other words, the deweathering model’s purpose was to improve object detection of images taken with “weathered” lenses. An image restoration model for weather-affected images by Zhu et al. [25] was implemented to evaluate. Since the model was pretrained for rain, snow, and haze (fog) images, images from the validation dataset classified into those categories by the ResNet-50 model were used to evaluate the de-weathering model. It was impractical to de-weather and compare all images in the dataset due to memory and GPU constraints. After implementing and running Zhu et al.’s model, the de-weathered (restored) images were paired with the original images. To evaluate the effectiveness of image restoration models in the context of autonomous driving under adverse weather conditions, an object detection–based performance assessment framework was developed. The DetrForObjectDetection model from the HuggingFace transformers library [26] was implemented due to its capability to output a numerical confidence score for each detected object. This confidence score, a continuous value ranging from 0 to 1, represents the model’s certainty in correctly identifying and localizing an object within the scene. This feature makes DETR particularly suitable for evaluating restoration quality, as it provides quantifiable insight into the impact of de-weathering processes on downstream perception tasks. The evaluation technique consists of applying the object detection model to both the original degraded image (e.g., containing rain, snow, fog) and its corresponding restored image from the de-weathering model. For each detected object, the confidence score generated by the DETR model on the restored image was compared to the score obtained from the original weather-degraded image. These differences in confidence values were then averaged across all detected objects within a single scene to produce an overall net confidence gain for a particular image. Repeating this procedure across a large set of restored and original image pairs produced a statistically meaningful metric of performance attributable to the restoration model, which could be used for further development or analysis. Similar to the training and fine-tuning of weather classification models using adaptive optimizers like AdamW, the proposed evaluation framework also supports the iterative improvement of image restoration models. By using the average confidence gain from object detection as a feedback signal, these models can be adjusted more effectively to enhance both visual clarity and object detectability, both of which are essential for the safe and reliable operation of autonomous vehicles in challenging weather conditions. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 37 3. Results 3.1. Weather classification Four separate models—CLIP, ResNet-34, ResNet-50, and ResNet-101—were trained and tested on the same custom dataset of approximately 4800 road images. Ten epochs were run on each model, producing final validation accuracies as follows: CLIP (0.79), ResNet-34 (0.92), ResNet-50 (0.95), and ResNet-101 (0.93). The loss after each epoch (the complete pass of the training data through the model) was also tracked, as shown in the graphs below. The data below shows the performance of the models under a batch size of 64 and base learning rate of 0.001, both of which seem to be best for the given conditions. Figure 6 Performance (accuracy) of CLIP vs. ResNet-50 during training on validation data set (720 images) Figure 7 Gradual optimization of CLIP vs. ResNet-50 through loss function over iterations of training data (4080 images) Both models, ResNet and CLIP, experienced a significant increase in accuracy and a decrease in loss as more epochs were run. This indicates that the models were learning from the training data, showing the potency of using a custom dataset to refine these models. Although CLIP significantly underperformed in comparison to ResNet-50, it still could be practical to use in the future. Unlike ResNet-50, CLIP’s accuracy did not plateau after ten epochs and steadily continued to increase. Thus, with more data and epochs run, it is likely that CLIP will be able to achieve performance similar to ResNet, if not better. Testing on both the AdamW and Adam optimizers, AdamW outperformed the latter (data for models above) showing better standardization and consistency. World Journal of Advanced Research and Reviews, 2025, 28(03), 030-041 38 Figure 8 Comparison of loss between AdamW and Adam optimizers; AdamW better for generalization due to standardization and adaptability 3.2. Image restoration (de-weathering) analysis classification To determine and evaluate the performance of Zhu et al.’s de-weathering model, the confidence in object detection with the original and restored images was measured and compared. Averaging these values produced a proportion that represents the effectiveness of the model. An example of this process for a single image is shown below. Figure 9 Effect of Zhu et al.’s Image Restoration Model on Object Detection: Rain (527) Table 2 Difference in confidence of object detection between original and restored image: Rain (527) Object Number Original Image (O) Restored Image (R) Difference (R – O) 1 0.848 0.856 +0.008 2 0.890 0.967 +0.077 3 0.710 0.784 +0.074 4 0.879 0.901 +0.022 5 0.814 0.966 +0.152 6 0.995 0.982 -0.013 7 0.984 0.994 +0.010 8 0.654 0.766 +0.112 Average 0.847 0.902 0.055