Evaluating the Performance of Machine Learning Algorithms in Predicting Industrial Equipment Maintenance Costs

Evaluating the Performance of Machine Learning Algorithms in Predicting Industrial Equipment Maintenance Costs Ali Cheraghalikhani¹, Mohammad Mahdi Masoumian²* ¹ Assistant Professor, Department of Industrial Engineering, Tafresh University – chera[email protected] ² M.Sc., Department of Industrial Engineering, Tafresh University – [email protected] Abstract Decision-making in the management of industrial machinery and equipment has always been of great importance, and proper maintenance strategies can significantly reduce operational costs. Selecting an effective maintenance approach requires a high level of accuracy, and one of the key factors contributing to this accuracy is the ability to estimate future maintenance and repair expenses. This study aims to present a precise and scientifically grounded method for predicting maintenance costs using the accuracy and computational efficiency of artificial intelligence. A polynomial regression model is employed as the predictive algorithm to estimate future maintenance expenditures. Finally, a related case study on forecasting maintenance costs for agricultural tractors is presented to demonstrate the applicability of the proposed approach. Keywords: Machine learning, predictive models, Maintenance 1. Introduction Since the invention of computers, humans have continuously sought ways to train these machines to perform desired tasks [3]. With the rapid growth of data science and artificial intelligence, it has become possible to identify complex patterns and relationships among numerous variables using machine learning algorithms and to leverage these insights to achieve practical objectives. Machine learning, as a subfield of computer science, aims to construct models based on data— whether obtained from natural phenomena, generated by human activities, or produced through other algorithms—so that these models can be used to solve real-world problems [4]. Increasing profitability within an organization requires minimizing operational costs. In industries where machinery and equipment constitute a significant portion of total assets, proper management of these resources has a substantial impact on reducing operational expenses. Moreover, the highly competitive nature of modern production and service environments demands rapid responses to customer needs. Equipment failures can disrupt production flow and lead to delays in product delivery. Therefore, reducing operational costs while simultaneously improving equipment reliability are two central objectives in the management of capital-intensive assets [1]. Developing an effective maintenance strategy requires access to accurate and comprehensive data, among which future maintenance and repair costs play a critical role. Reliable estimation of these costs can be achieved using various artificial intelligence techniques such as polynomial regression, support vector machines, or artificial neural networks. In this study, the focus is placed on the application of polynomial regression for cost prediction. The remainder of this paper is organized as follows: Section 2 provides a brief overview of the principles and operation of machine learning algorithms. Section 3 introduces the polynomial regression method. Section 4 describes the data preparation process and model training procedure. Section 5 presents model evaluation techniques and error analysis. Section 6 discusses a relevant case study from the agricultural tractor maintenance domain, and finally, Section 7 offers conclusions and recommendations. 2. Principles and Functioning of Machine Learning Algorithms Machine learning algorithms are, fundamentally, mathematical computational procedures that are implemented using programming languages. In Python, one of the most widely used and versatile programming languages for artificial intelligence and machine learning, implementations of various algorithms are organized into a comprehensive library called scikit-learn, which is readily accessible to users. By selecting an appropriate method for their specific problem, users can readily invoke and employ the desired algorithm from this library. Currently, Python is considered the most prevalent and capable programming language in the field of artificial intelligence and machine learning. Numerous printed and online resources are available to guide practitioners in effectively utilizing the algorithms and libraries provided within Python. This study focuses on the polynomial regression algorithm, for which all necessary regression formulas have been systematically implemented and are accessible through scikit-learn. Machine learning algorithms are generally categorized into three main types: supervised learning, unsupervised learning, and reinforcement learning. • Supervised learning: This category involves datasets with labeled outcomes. Problems involving numerical prediction or classification with a limited number of distinct categories—such as the one addressed in this study—are typically analyzed using supervised learning algorithms. • Unsupervised learning: In this type, datasets lack labeled outcomes. It is applied when the potential outcomes of a specific condition are unknown or when the set of possible solutions is not predefined or limited. • Reinforcement learning: In reinforcement learning, artificial intelligence agents iteratively improve their performance through trial and error, using feedback from prior computations. AI systems designed for strategic games, such as chess, often rely on this approach [3]. 3. Introduction to the Polynomial Regression Model In supervised machine learning, where the primary objective is to predict the value of a target variable based on other features, linear regression algorithms are commonly employed. However, when the relationships among the data are more complex than a simple linear pattern, a linear model alone may be insufficient. A straightforward approach to handle non-linear relationships is to expand the feature set by including polynomial transformations of each feature. Subsequently, a linear model can be fitted using this augmented dataset, effectively capturing non-linear dependencies. This methodology is referred to as polynomial regression [5]. Supervised learning fundamentally entails learning a function from a training dataset. This function maps an input variable 𝑥(which may be a vector of multiple features) to a target variable 𝑦, where 𝑥is termed the predictor and 𝑦the response variable [6]. 4. Data Preparation Methodology In predictive modeling tasks such as classification or regression, raw data cannot usually be employed directly for several reasons: • Machine learning algorithms require numerical input data. • Raw datasets often contain noise and statistical errors that must be corrected. • Complex non-linear relationships may be present, which can significantly affect model performance if not properly handled. Consequently, raw data must undergo preprocessing and refinement before being input into machine learning algorithms. This preparatory step is referred to as the data preparation phase in predictive model implementation [7]. In maintenance and repair applications, certain observations may exhibit exceptionally large deviations from expected values. For instance, the fuel consumption of equipment such as tractors may, under specific circumstances, be recorded with substantial discrepancies from actual usage, or may suddenly increase or decrease due to unusual operational conditions. Such observations, commonly referred to as outliers, should be removed from the dataset, as their presence can significantly increase model error and reduce predictive accuracy. Additionally, textual data must be converted into numerical form, since most machine learning algorithms cannot process string data directly. Several approaches exist for this transformation. When the textual variables have limited diversity—for example, days of the week or seasons— each category can be represented as a separate column in the dataset, with binary indicators (0 or 1) specifying the presence of each category for a given record. In cases where textual variables exhibit high diversity, such as addresses or numerous component names, one can leverage encoding techniques available in libraries such as category encoders. Before feeding the dataset into polynomial regression or any other machine learning algorithm, the data are typically split into training and testing subsets. In this study, 75% of the data were allocated for training and 25% for testing. This split enables the evaluation of model performance and the computation of prediction errors, as described in the following section. 5. Model Evaluation and Error Metrics During the implementation of a machine learning model, data are partitioned into training and testing subsets. Initially, the algorithm is trained on the training dataset, allowing the model to learn the underlying relationships. Subsequently, the trained model is applied to the testing dataset to predict target values such as costs or prices. These predicted values are then compared against the actual observations, and model errors are computed to assess performance. Several metrics can be employed to quantify predictive accuracy, including: • Mean Absolute Percentage Error (MAPE): Measures the average absolute deviation between predicted and actual values as a percentage. • Root Mean Squared Error (RMSE): Provides a measure of the average magnitude of prediction errors, with greater weight given to larger deviations. • Coefficient of Determination (R²): Assesses the proportion of variance in the observed data explained by the regression model, indicating the goodness-of-fit. These evaluation metrics provide a comprehensive understanding of model reliability and predictive performance, ensuring the selected algorithm is appropriate for the given maintenance cost forecasting task. 𝑅𝑀𝑆𝐸= √∑(𝑑𝑖−𝑝𝑖)2 𝑚 𝑖=1 𝑚 (1) 𝑀𝐴𝑃𝐸= 1 𝑚∑|𝑑𝑖−𝑝𝑖 𝑑𝑖| 𝑚 𝑖=1 × 100 (2) 𝑅2= (∑(𝑑𝑖−𝑑) 𝑚 𝑖=1 (𝑝𝑖−𝑝))2 ∑(𝑑𝑖−𝑑)2 . ∑ (𝑝𝑖−𝑝)2 𝑚 𝑖−1 𝑚 𝑖=1 (3) In the above formulas, the variables are defined as follows: • 𝑑𝑖 denotes the predicted cost obtained from the model for the 𝑖-th observation. • 𝑝𝑖 represents the actual recorded cost corresponding to the 𝑖-th observation. • 𝑑ˉ is the mean of the predicted costs across all observations. • 𝑝ˉ is the mean of the actual costs. • 𝑚 indicates the total number of samples used for error measurement. These definitions provide a precise interpretation of the variables employed in the error evaluation metrics, ensuring clarity and reproducibility in predictive model assessment. 6. Case Study: Forecasting Tractor Maintenance and Repair Costs in the Agricultural Sector The management of industrial machinery replacement, particularly tractors, is a critical factor for the timely execution of agricultural operations. Failure to accurately predict the optimal replacement time can lead to reduced machine availability. Due to the stochastic and unpredictable nature of maintenance, the magnitude and characteristics of repair and maintenance costs are strongly influenced by machine quality, management practices, operator skill, and environmental conditions. This underscores the importance of accurately forecasting maintenance and repair costs for machinery, especially tractors [2]. In this study, the forecasted cost components include: • Repair costs (spare parts and labor), • Oil costs (lubricants and oil filters), • Fuel costs (diesel and fuel filters), • Overall maintenance costs, and • Year of purchase and manufacture for each tractor. Before inputting the data into the machine learning algorithm, a standardization procedure was applied. It was necessary to adjust costs for inflation. Moreover, due to the heterogeneity of maintenance costs and initial purchase prices among different tractor types, as well as the distinct operational contexts in which they are employed, all cost variables were standardized using the following index: 𝐶𝐶𝐼𝑡= ∑𝐶𝑡 𝑡0 𝑃𝑃0 × 100 (4) where: • 𝐶𝐶𝐼𝑡is the Cumulative Cost Index at time, 𝑡 for each cost category (fuel, oil, or repairs), • 𝐶𝑡represents the maintenance or operating cost at time 𝑡for each category, and • 𝑃𝑃0denotes the initial purchase price of the tractor. This index either increases or remains constant over the calendar life of the tractor. After standardizing the costs, cumulative indices were computed as follows: Cumulative Repair Cost Index (CCI_repair), Cumulative Oil Cost Index (CCI_oil), and Cumulative Fuel Cost Index (CCI_fuel). These indices were used as the dependent variables in the regression models. In economic decision-making for machinery management, the overall Cumulative Maintenance Cost Index—defined as the sum of the cumulative fuel, oil, and repair indices—is often employed to determine the effective service life of equipment. Regarding service life, the independent variable in the regression model is typically the cumulative operational hours of the tractor. Cumulative operating hours provide the most appropriate representation of a tractor's lifetime, reflecting the total hours the tractor has physically worked. This measure effectively moderates the considerable fluctuations in maintenance costs over time. In this study, the cumulative operating hours for each tractor were estimated based on the frequency of monthly engine oil changes. Table 1: prediction results cost indicator model β1 β2 β3 R2 NAPE CCI repair seccond degree 0.398 0.001 - 0.998 27.37 third dgree 0.046 0.005 1.1E-05 0.999 2.94 exponential 3.993 0.018 - 0.906 62 CCI oil seccond degree 0.054 0.00028 - 0.999 4.02 third dgree 0.047 0.00028 3E-07 0.999 2.89 exponential 1.169 0.014 - 0.848 39.71 CCI fuel seccond degree 0.029 0.0003 - 0.999 8.95 third dgree 0.041 0.00011 5.2E-07 0.999 4.89 exponential 1.009 0.015 - 0.902 28 It was observed that third-degree polynomial regression exhibited the best performance, yielding the lowest prediction error compared to other models. Therefore, the model developed based on third-degree polynomial regression can be employed with a high degree of confidence to forecast tractor maintenance and repair costs, as well as to determine the optimal replacement time. It should be noted that while one might assume that increasing the degree of the regression equation would always enhance model accuracy, this approach may lead to overfitting, ultimately reducing the model's generalization capability. 7. Conclusion After obtaining estimates of potential future costs for machinery and equipment, engineering economic techniques can be applied to develop optimal strategies for maintenance and repair system planning. Since these estimates are derived using machine learning algorithms and the accuracy of the model has been validated, the resulting strategy can be relied upon with substantial precision and confidence. Furthermore, the development of similar studies aimed at predicting equipment failure times can provide valuable insights for designing preventive and predictive maintenance strategies, enhancing operational efficiency and reliability. References [1] ;)شرفت دحاو( ریبکریما یتعنص هاگشناد .)یزاس هنیهب و یزاسلدم( تاریمعت و یرادهگن تیریدم .یناکملگ اضردیمح2009 . [2] ریمعت یاههنیزه ینیبشیپ رد نویسرگر یاهلدم یبایزرا .یفطصم هداز یلو ,نیسحدمحم درف روپسابع ,جریا ربجنر ,سابع یناحور .یشکهز و یرایبآ یاه هزاس یسدنهم تاقیقحت .روتکارت یرادهگن و2007;11 (3). [3] Mitchell TM. Machine Learning. New York: Mcgraw Hill; 1997. [4] Andriy Burkov. THE HUNDRED-PAGE MACHINE LEARNING BOOK. Andriy Burkov; 2019. [5] Aurélien Géron. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. “O’Reilly Media, Inc.”; 2019. [6] Neapolitan RE, Jiang X. Artificial Intelligence. Chapman and Hall/CRC; 2018. [7] Brownlee J. Data Preparation for Machine Learning. Machine Learning Mastery; 2020.