Minimum description length and multi-criteria decision analysis in predictive modelling

Received 3 January 2025, accepted 20 January 2025, date of publication 22 January 2025, date of current version 30 January 2025. Digital Object Identifier 10.1109/ACCESS.2025.3532815 Minimum Description Length and Multi-Criteria Decision Analysis in Predictive Modeling PETR SILHAVY 1, KATEŘINA HLAVÁČKOVÁ-SCHINDLER 2, AND RADEK SILHAVY 1 1Faculty of Applied Informatics, Tomas Bata University in Zlín, 760 01 Zlín, Czech Republic 2Data Mining and Machine Learning Research Group, Faculty of Computer Science, University of Vienna, 1010 Vienna, Austria Corresponding authors: Radek Silhavy ([email protected]) and Kateřina Hlaváčková-Schindler (katerina.schindlerov[email protected]) This work was supported by Tomas Bata University in Zlín, Faculty of Applied Informatics, under Project RO30246061025/2102. ABSTRACT Accurate model selection is essential in predictive modelling across various domains, significantly impacting decision-making and resource allocation. Despite extensive research, the model selection process remains challenging. This work aims to integrate the Minimum Description Length principle with the Multi-Criteria Decision Analysis to enhance the selection of forecasting machine learning models. The proposed MDL-MCDA framework combines the MDL principle, which balances model complexity and data fit, with the MCDA, which incorporates multiple evaluation criteria to address conflicting error measurements. Four datasets from diverse domains, including software engineering (effort estimation), healthcare (glucose level prediction), finance (GDP prediction), and stock market prediction, were used to validate the framework. Various regression models and feed-forward neural networks were evaluated using criteria such as MAE, MAPE, RMSE, and Adjusted R2. We employed the Analytic Hierarchy Process (AHP) to determine the relative importance of these criteria. We conclude that the integration of MDL and MCDA significantly improved model selection across all datasets. The cubic polynomial regression model and the multi-layer perceptron models outperformed other models in terms of AHP score and MDL criterion. Specifically, the MDL-MCDA approach provided a more nuanced evaluation, ensuring the selected models effectively balanced complexity and predictive accuracy. INDEX TERMS Multicriteria decision analysis, minimum model length, machine learning, model selection prediction, MDL-MCDA. I. INTRODUCTION Accurate model selection is essential in predictive modelling across various domains. The efficacy of predictive models influences decision-making processes and resource allocation. Despite extensive studies comparing multiple predictive models, the model selection approach still needs to be explored. Model selection is intrinsically tied to the objectives of prediction and understanding, with its essence captured through the formalisation of loss and risk, as declared by Petropoulos et al. [1] and by Friedman [2]. The issues of model selection lie in navigating through the complex relationship between independent variables and the dependent variable underpinned by both observable and unobservable factors. The literature identifies two broad The associate editor coordinating the review of this manuscript and approving it for publication was Alba Amato . categories of variables influencing dependent variables: explanatory variables, which are observable, and unobservable variables, which include factors such as measurement errors or omitted independent variables. This issue has been discussed in various problem domains. A. HOW MODELS CAN BE SELECTED: MAIN ISSUES Model selection is a crucial challenge within all prediction tasks, bridging the gap between theoretical constructs and practical applications. This section describes the core aspects of model selection, covering the fundamental issues, model evaluation and construction methodologies, and the selection process. The multicriteria approach, assumptions underlying model selection strategies, and the interplay between explanatory variables and unobservable factors influencing the outcome variable are essential to our discussion. The evaluated datasets are often more complex because they 19388 2025 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling contain more features, allowing more independent variables to be used. Issues in model selection are mainly related to selecting a model that will fit data well, keep a complexity level low, and provide a reasonable, accurate prediction. For each prediction task, the following is to be evaluated [3]: •Complexity level – The existing methods of assessing model quality are often based on assumptions of randomness of variables and may, therefore, be sensitive to extreme values. On the contrary, some methods make few assumptions about randomness, but their inherent generality may alter their results. This means that while assumption-based methods can be very accurate under ideal conditions, they may fail with non-ideal data, whereas assumption-light methods are more flexible but can sometimes offer less precise insights. •Class of models for a specific system – It is also important to use a good set of predictive models and select the most relevant method for model construction. •Evaluation criteria – Selecting the most relevant evaluation criteria for prediction models is crucial. A model evaluation criterion, based on its distance to the theoretical quantity, assesses the performance in predicting a model. Also, criteria can often be in conflict. The prediction task is challenging due to unknown relationships between the variables involved. A common approach is to create multiple models that represent these relationships differently. The task then becomes evaluating and comparing these models to select the best one, where the ‘‘best’’ is taskspecific. Four benchmark problem domains were chosen to validate the proposed evaluation methods: effort estimation, predicting glucose levels, gross domestic product (GDP) prediction, and stock price prediction. These domains require accurate predictions in both technical and financial fields and the versatility and robustness of the proposed MCDA-MDL evaluation framework is demonstrated in our work. The first prediction task is in software engineering, focusing on predicting software development efforts. This task is crucial for project management because accurate estimates help to plan, budget, and allocate resources. Simple and accurate predictive models are valuable as they are easier for project managers and stakeholders to understand and use, ensuring better project control and success. The second prediction task is from the medical and health science, explicitly predicting glucose levels. Accurate glucose level predictions are essential for managing diabetes, as they help to monitor and maintain optimal glucose levels and prevent complications. This task represents a broader challenge in medical research, where accurate predictions are necessary for effective patient care and treatment planning. The third prediction experiment is related to the gross domestic product prediction. Those predictions are essential for company financial planning and life cost prediction. Knowing the gross domestic product prediction is mandatory for many businesses and public administration. The fourth domain is related to stock market prediction. This was included as being a typical representation of the time series. Also, this is an important task in economic and financial analysis. By choosing these four significant problem domains, the study aims to show the versatility and robustness of the proposed MCDA-MDL evaluation framework over data science applications. The software engineering, medical and financial analysis/economic tasks highlight the need for practical and easy-to-use models in each field. B. OBJECTIVES OF THE WORK Model selection simplifies the process by reducing the number of possible models to a limited set. However, it remains a challenging problem because it requires defining what makes a good model and how to measure its quality. These definitions should align with the primary goal of the study. Although this seems straightforward, in practice, the methods used to create and evaluate models often need to align better with the study’s objectives. To address the challenges of model selection in the presence of conflicting error measurements, we propose the integration of Minimum Description Length (MDL) and Multi-Criteria Decision Analysis (MCDA). The MDL principle [4],[5] helps balance the model’s complexity with its ability to fit the data. By minimizing the minimal description length, MDL provides a robust way to prevent overfitting and select models that generalize well to new data. Integrating the MCDA approach is essential when error measurements conflict. MCDA helps to incorporate an error score, which fuses more than one error criterion. MCDA score can be understood as a goodness-of-fit part of MDL. By combining MDL and MCDA, we can enhance the model selection process, ensuring that the selected model fits the data well and effectively meets the task’s objectives. This integrated approach provides a structured framework to navigate the complexities of model evaluation and selection. C. RESEARCH QUESTIONS For this work, the following research questions have been set: •RQ1: How does the Minimum Description Length (MDL) and Multi-Criteria Decision Analysis (MCDA) integration affect predictive model selection? •RQ2: What advantages does the MDL-MCDA have compared to the MDL-RSS1in predictive model selection? D. MAIN CONTRIBUTIONS OF THE WORK In this paper, we address critical challenges in predictive modeling and model selection by introducing a novel methodological framework that integrates the strengths of the Minimum Description Length (MDL) principle with Multi-Criteria Decision Analysis (MCDA). While traditional 1MDL-RSS is the common MDL having the residual sum of squares at the goodness-of-fit criterion. VOLUME 13, 2025 19389 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling MDL relies on Residual Sum of Squares (RSS) as a measure of goodness-of-fit, this approach often falls short in scenarios where multiple, conflicting error criteria must be balanced, particularly in complex, real-world datasets. By integrating MDL with MCDA, we extend the scope of model evaluation beyond a single error metric, allowing for a more robust and nuanced assessment that accounts for multiple evaluation criteria. This innovation not only enhances the reliability of model selection but also addresses critical gaps in traditional MDL approaches. Below, we detail the specific contributions that underscore the novelty and practical value of our proposed framework: •MDL and MCDA Integration: Introduces a novel integration of Minimum Description Length (MDL) with Multi-Criteria Decision Analysis (MCDA) to improve predictive model selection by resolving conflicting error measurements. •Comparative Analysis: Evaluates MDL-AHP vs. MDL-RSS integrations for better handling of complex datasets, showing practical benefits. •Domain Applications: Assesses the methodology across multiple domains. •Benchmark Datasets: Validates the framework using datasets from software engineering, medical, and finance domains. •Enhanced Selection Framework: Demonstrates that MDL-MCDA improves model selection by balancing complexity and accuracy. •Impact of MDL-MCDA: MDL-MCDA outperforms traditional MDL or MCDA in model selection across datasets. •MDL-MCDA vs. MDL-RSS: Shows MDL-MCDA selects models with better generalization compared to MDL-RSS. These contributions advance the understanding and implementation of model selection methodologies, offering to apply MDL with MCDA to various domains requiring precise and reliable predictive modelling. E. PAPER ORGANIZATION The rest of the paper is organised as follows. Section II provides a comprehensive overview of existing research and methodologies related to model selection, MDL, MCDA, AHP, and RSS. Section III details the methods used in our work, including data preparation, model implementation, evaluation measures, and the integration of MDL and MCDA. Section IV presents the results of the experiments using various datasets and predictive models. It includes a comparison of the performance of different models based on AHP, MDL with AHP, and MDL with RSS. Section V discusses the implications of the results, the effectiveness of the integrated approach, and its applicability to different problem domains. Finally, Section VI summarizes the study’s main findings, highlights the contributions, and suggests directions for future research. II. RELATED WORK A wide range of viable prediction models are available across different industries, making it difficult to determine the optimal one, especially when faced with conflicting error measures. The Minimal Description Length (MDL) was introduced to address this issue in [4]. MDL is an alternative to the Akaike Information Criterion (AIC), which was introduced as a recognised method for automatic model selection [6]. While the Akaike Information Criterion (AIC) is highly efficient in selecting models within the same class and comparing non-nested models, such as linear and non-linear models, it cannot automatically choose models from different prediction model classes, such as exponential smoothing and autoregressive models. To address this limitation, the Bayesian information criterion (BIC) from Schwarz was introduced, which, in the same vein as AIC, evaluates the fit of the data with a complexity penalty. However, the BIC imposes a more substantial penalty for complexity than the AIC. Nevertheless, this method still requires further development to assess models within the same class. Villegas et al. [7] suggest employing support vector machines (SVM) to identify the most suitable prediction model from a range of alternatives, given that model variables (such as the degree of accuracy and the fitted parameters) may change over time. The researchers discovered that utilising SVM leads to a greater overall predictive accuracy. Ghobbar and Friend [8] devised a predictive error forecasting technique for assessing demand prediction models in the airline manufacturing sector based on their factor levels. They employed mean absolute percentage error (MAPE) as the criterion for evaluation but did not account for hybrid prediction models that incorporate personal information. Oh and Morzuch [9] assessed eight demand prediction models using six performance measures that evaluate bias and forecast error, including MAPE, MAE, RMSE, AIC, and BIC. Their study revealed that the choice of prediction model varied based on the performance measures employed. Taylor and McSharry [10] evaluated six distinct prediction models to estimate electricity demand across ten European countries. They used MAPE and MAE as evaluation measures and discovered that the rankings generated conflicting outcomes, except for the top-performing model, which consistently ranked first. Petropoulos et al. [1] and Han et al. [11] investigated the use of subjective expert judgment in prediction model selection, revealing that the chosen models outperformed those selected through AIC based on evaluation measures such as MAE, MAPE and MASE. Furthermore, it has been shown that collective judgment is superior to a single decision and statistical selection methods. Davydenko and Fildes [6], for instance, explored the effectiveness of MAPE and median average percentage error (MdAPE) in assessing judgmental adjustments to statistical prediction. They concluded that relying solely on MAPE to determine a model’s performance is insufficient due to inconsistent results between MAPE and other error measures. The study suggests 19390 VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling that future research should develop an approach for selecting the optimal model when evaluating multiple error measures, particularly in the face of conflicting results. Multiplecriteria decision analysis (MCDA) is a widely-used approach for addressing complex problems involving multiple, often conflicting, objectives [12]. Selecting predictive models using MCDA can be particularly useful when different error measures, such as mean squared error and mean absolute error, provide conflicting guidance on the optimal model. Comparing AHP and TOPSIS, two prominent MCDA methods, we consider how they can be applied in this model selection context. The Analytic Hierarchy Process (AHP) [13] is a structured technique for organizing and analyzing complex decisions. AHP involves decomposing a problem into a hierarchy of goals, objectives, and alternatives and then using pairwise comparisons to derive priorities for the alternatives. In the model selection domain, AHP could be used to establish a hierarchy with the overall goal of minimizing prediction error, with sub-objectives of minimizing MSE, MAE, and potentially other relevant measures. Each candidate model would then be evaluated against these criteria, with AHP providing a composite score to guide the final model selection [14],[15],[16],[17]. The perceptron, a fundamental building block of neural networks, has also been explored for effort estimation. A study by [18] demonstrated the potential of perceptron-based models to capture non-linear relationships, characteristic for effort estimation problems and the potential to improve traditional estimation techniques. While neural network and deep learning models have shown promising results, their performance is heavily dependent on the quality and characteristics of the input data. Proper feature engineering, data preprocessing, and hyperparameter tuning are crucial for achieving reliable and accurate effort estimation using these advanced techniques. Hyperparameter optimization can significantly impact the model’s predictive capabilities and generalisation, such as the number of hidden layers, neurons, and the learning rate. Neural networks and deep learning models promise to improve software effort estimation. Their ability to model complex, non-linear relationships in data makes them well-suited for this task. Continued advancements in neural network architectures, training algorithms, and hybrid modelling approaches will likely enhance their accuracy and applicability in software engineering. Various methods for evaluating prediction models in different domains, primarily using error measures and information criteria like AIC and BIC. However, employing AIC and BIC to assess models restricts the comparison to models within the same class. Moreover, further research is needed to determine an appropriate approach to evaluating multiclass demand prediction models based on several interdependent error measures and to select the best model based on the simultaneous use of multiple error measures [19]. III. METHODOLOGY A. RESEARCH DESIGN This work evaluates the integration of minimum description length (MDL) and multi-criteria decision analysis (e.g. AHP) in selecting predictive models. To achieve this, a series of steps were taken during experimental work. This involves data preparation, model implementation, evaluation, and comparison. We employ four datasets covering software engineering, medical and financial problem domains. These cover several domains and sizes and are also a combination of natural and synthetic samples. The considered model classes include multiple linear regression (MLR), Ridge regression, Lasso regression, Elastic net regression, quadratic and cubic regression (i.e. polynomial regression with degrees 2 and 3), and a feed-forward neural network (FF-NN) with various configurations, which will be specified. The performance of these models will be evaluated using criteria: Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), Adjusted R-squared (adjR2), Prediction at 25% (Pred(0.25)), and Weighted Quantile Loss (WQL). Additionally, we will apply the Analytic Hierarchy Process (AHP) and Minimum Description Length (MDL) principles to aid in model selection. The procedure will involve several steps (Figure 1). First, a dataset is prepared and processed by handling missing values, normalising data, and splitting them into training and test sets. Then, the predictive models are trained using the training data. After training, they evaluate each model using the specified criteria on testing data. The MDL principle will help quantify the complexity and goodness-of-fit for each model, focusing on the total description length, which includes the model structure, parameters, and data encoding. MCDA, specifically AHP, will evaluate models based on multiple criteria. This involves making pairwise comparisons to determine the relative importance of each criterion in the final score for model selection. Finally, the models selected using traditional methods will be compared with those chosen through the integrated MDL and MCDA approach. This comparison will determine if the combined approach improves predictive accuracy and model simplicity. The expected outcomes of this work include identifying the impact of integrating MCDA methods (e.g. AHP) for MDL and comparing to selection using AHP only, or MCDA with RSS. Moreover, we obtain insights into how MDL and MCDA can improve model selection and understand the impact of different variables on model performance. B. EVALUATING MEASURES The evaluation and comparison of models involve a detailed analysis of various models’ performance, focusing on their ability to predict or explain the dependent variable accurately. Let us consider a sample D= {(xi,yi),i=1, . . . , n},of VOLUME 13, 2025 19391 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling FIGURE 1. Model design flowchart. variable values yiand ˆyi, where yirepresents the actual value and ˆyiis the predicted value. Mean Absolute Percentage Error (MAPE) is a measure of prediction accuracy of a forecasting method, expressing the accuracy as a percentage. It is defined as: MAPE =1 n n X i=1 yi− ˆyi yi ×100.(1) Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) are common evaluation measures assessing the average magnitude of prediction errors. MAE is defined as: MAE =1 n n X i=1yi− ˆyi(2) and RMSE is defined as: RMSE =v u u t 1 n n X i=1 (yi− ˆyi)2.(3) The Median Absolute Percentage Error (MdMAPE) provides a robust measure by focusing on the median of the percentage errors, defined as: MdMAPE =median  yi− ˆyi yi ×100(4) The Adjusted Coefficient of Determination (Adjusted R2) is an enhancement of the regular R2metric that adjusts for the number of predictors in the model. It provides a more accurate measure of goodness of fit than R2by considering model complexity. Adjusted R2is defined as: Adjusted R2=1−(1 −R2)(n−1) n−k−1(5) where nis the number of observations, kis the number of predictors, and R2is the coefficient of determination on set D. Pred(0.25) evaluates the proportion of predictions that fall below a specified error threshold, such as 25%. It is useful for assessing the overall model’s predictive accuracy within an acceptable error range. Pred(0.25) is calculated as Pred(0.25) =1 n n X i=1 I yi− ˆyi yi <0.25(6) where Iis an indicator function that equals 1 if the condition is true and 0 otherwise. The Weighted Quantile Loss (WQL) [20] measures how well a predictive model performs across different quantiles of the target variable’s distribution. It is particularly useful in scenarios where it is important to understand the model’s performance across various data distribution segments. The WQL is given by: wQL(τ)=PN i=1Lτ(yi,ˆyi(τ)) PN i=1|yi|(7) where τis the quantile level (e.g., quartils), yiis the observed value at the i-th data point, and ˆyi(τ) is the predicted quantile value at the i-th data point for the quantile level τ. The quantile loss function, Lτ(yi,ˆyi(τ)), is defined as Lτ(yi,ˆyi(τ)) =(τ−1{yi<ˆyi(τ)})(yi− ˆyi(τ)), where 1{yi< ˆyi(τ)}is an indicator function that equals 1 if yi<ˆyi(τ) and 0 otherwise. 1) DISCUSSION ON EVALUATION MEASURES Model performance is assessed using various criteria: Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Median Absolute Percentage Error (MdMAPE), Adjusted R2, and Predictions Level (Pred(0.25)). Each criterion offers a distinct perspective on model evaluation, capturing different facets of accuracy, robustness, or complexity. However, these measures may occasionally produce conflicting results, necessitating careful interpretation. The selection of these criteria reflects their ability to balance accuracy, robustness to outliers, and the trade-off between model fit and complexity. Accuracy-focused measures: •MAPE measures percentage errors, providing an intuitive view of relative accuracy for stakeholders. •MAE averages error magnitudes, offering a straightforward overall accuracy measure without outlier bias. •RMSE highlights large errors, useful for significant deviations but sensitive to outliers, potentially conflicting with MAPE. 19392 VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling Robustness against outliers: •MdMAPE captures median percentage errors, ensuring robustness to outliers and complementing RMSE and MAE. Model fit and complexity: •Adjusted R2measures variance explained, accounting for predictors to balance fit and complexity. •Pred(0.25) measures predictions within 25% error, prioritizing consistent accuracy over complexity metrics. Balancing these criteria is crucial for developing robust models. Measures like MAPE, MAE, RMSE, MdMAPE, and Pred(0.25) focus on predictive accuracy, while complexity-oriented measures like Adjusted R2provide insights into generalizability. By evaluating multiple metrics, a comprehensive understanding of the model’s strengths and weaknesses emerges. Evaluation measures can be grouped based on whether they should be minimized, maximized, or zeroed for optimal performance: Minimization Criteria: •Mean Absolute Percentage Error (MAPE): emphasizes relative prediction errors. •Mean Absolute Error (MAE): captures average error magnitudes. •Root Mean Squared Error (RMSE): penalizes larger errors, highlighting extreme deviations. •Median Absolute Percentage Error (MdMAPE): offers robustness to outliers. Maximization Criteria: •Adjusted Coefficient of Determination (R2): balances variance explanation and model complexity. •Proportion of Predictions Below 25% Error (Pred(0.25)): emphasizes practical predictive accuracy. •Weighted Quantile Loss: ensures balanced performance across quantiles. This diverse set of evaluation criteria ensures the model is both accurate and generalizable, meeting practical needs while avoiding overfitting or overemphasis on specific error types. C. MINIMUM DESCRIPTION LENGTH The Minimum Description Length (MDL) principle is a formal method of inductive inference that balances the model complexity and goodness of fit. This principle is rooted in information theory and aims to avoid overfitting by penalising model complexity. MDL was introduced by Rissanen [4] and further developed by Rissanen et al. in e.g. [21],[22],[23], and by Grünwald and Roos in [5] and [24]. MDL is based on the idea that the best model for a given set of data is the one that allows for the shortest overall description of the data and the model itself. The total description length is the sum of the data and model encoding lengths. Mathematically, the total description length L(D,M) can be expressed as: L(D,M)=L(M)+L(D|M) (8) where L(M) is the length of the description of the model M and L(D|M) is the length of the description of the data Dgiven the model M. MDL prefers models that balance simplicity (short model description) and accuracy (short data description given the model) in the sense that the best model Dis minimizing Eq. (8). This approach penalizes more complex models unless they significantly improve the data fit. MDL can bring advantages where overfitting is a concern and model interpretability and simplicity are valued. MDL helps select models that generalise well to new data by penalizing model complexity. MDL focuses on the total length of encoding both the model and the data, ensuring that the model chosen is the one that best compresses the data. This means MDL inherently balances model fit and complexity by minimizing the information required to describe the model and the data it explains. Unlike AIC and BIC, which are derived from statistical considerations, MDL directly addresses the issue of overfitting by penalizing unnecessarily complex models, thus often leading to models that generalize better to new data. This makes MDL a robust criterion for selecting models that are not only accurate but also parsimonious, enhancing predictive performance and interpretability. 1) TWO-PART MDL CODES The minimal description length as defined by Eq. (8) is in the literature called a two-part MDL. We point in the beginning, since MDL is a principle, there can be various encodings of the models from a class of models and thus there can be various MDL functions corresponding to the general scheme from Eq. (8). We will now come to a more formal explanation of the MDL principle and its encodings. First we briefly explain the orginal theory from Rissanen [4],[22] and his followers [5], [24] as it was developed for the case that a conditional probability distribution p(y|x) is known. Secondly we explain MDL when we only know about the model, from which the data are generated, that it is a member of a class of functional models [25]. 2) ENCODING OF MODELS WITH KNOWN PROBABILITY OF THE DATA GENERATION PROCESS We define a model for the prediction problem as a conditional probability distribution p(y|x) over and input space X, i. e. in other words, Py∈Yp(y|x)=1 (where the output space Y can be theoretically also an infinite). A model class is a set of models depending on a parameter vectorθ θ θ, i.e. M= {pθ θ θ,θ θ θ∈ 2 2 2}. Usually, 2 2 2is a subset of a multivariate Euclidean space. Shannon in [26] proved the following fundamental statement in information theory, known under the name ShannonHuffman code. If a sender and receiver agreed in advance on a model pand both know the input xi,i=1, . . . , n then there exists code to transmit the values yi,i=1, . . . , n losslessly with codelength (up to at most one bit on the whole VOLUME 13, 2025 19393 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling sequence) Lp(y|x)= − n X i=1 log2p(yi|xi) (9) where y,xis a shortened notation for set y1, . . . , yn, x1, . . . , xn, respectively (which are from Y,Xrespectively). The one additional bit in the Shannon-Huffman code is present only once for the whole data set [27] and with large data sets is negligible. Thus it will be omitted from the encodings. We do no need to know the practical implementation of compression algorithms but we consider only the theoretical bit length of their associated encodings. We want to measure the amount of information contained in the data, and how it is represented by the model. So we will directly work with codelength functions. Probability distribution function pcan be understood as the data generating process and in general it is not known but can be approximated from the data. To quantify the complexity of the computational models for prediction (and in general for a supervised learning problem) can be done e.g. by parameter counting. An informationtheoretic way to use the Occam razor principle in terms of the simplest model with a good generalization is the minimum description length (MDL), introduced by Rissanen [4] and further developed by Rissanen, Barron, Yu in e.g. [21],[22], [23], and by Grünwald and Roos in [5],[24]. Encodings in which the parameters of a model are at first transmitted to the receiver and then the data using these parameters are encoded, have been called two-part codes and introduced by Grünwald [5]. Let Lparam(θ θ θ) be any encoding scheme for parameters θ θ θ∈ 2 2 2and let θ θ θ∗be any parameter. The corresponding two-part codelength is Lθ θ θ∗(y|x)=Lparam(θ θ θ∗)+Lpθ θ θ∗(y|x) =Lparam(θ θ θ∗)− n X i=1 log2pθ θ θ∗(yi|xi).(10) Pn i=1log2pθ θ θ∗(yi|xi) is called the goodness-of-fit. The objective is to find θ θ θ∗at the minimum of (10) over all parameterizations. 3) ENCODING OF MODELS WITH KNOWN FUNCTIONAL CLASS OF THE DATA GENERATING PROCESS When pis known, it is clear that the minimum of (10) is equivalent to the maximum likelihood estimate (MLE). However, what makes the MDL principle so generic is that it can be generalized to the functional cases, i.e. instead of p probability, fas a general function can be considered about which is only known to be a member of a class of candidate models, see e.g. [25]. It means that about the model, from which the data are generated, is only known to be a member fl(.|θ θ θl) of a class of models M= {fl(.|θ θ θl),θ θ θl∈2 2 2l, θlj ∼πlj(θlj), l=1, . . . , m,j=1, . . . , kl}(11) where mis the number o models in M,θ θ θl=(θl1, . . . , θlkl) is a kl-dimensional parameter vector associated with fland 2 2 2lis a parameter space for θ θ θl.πlj(θlj) is introduced merely to simplify the encoding process as an artificial device to minimize the description length. It is assumed that every flis known except for θ θ θl, and that different flmay have different number of parameters kl. Given a set of observed data, the goal is to find the ‘‘true’’ flfrom Mas well as to estimate the parameter θ θ θiassociated with it. In this sense is (10) replaced by L(y)=L(ˆ θ θ θl)+L(y|ˆ θ θ θl) (12) where L(ˆ θ θ θl), L(y|ˆ θ θ θl) are code lengths for encoding fl(.|ˆ θ θ θl) and ‘‘yconditioned on fl(.|ˆ θ θ θl)’’ respectively. L(y|ˆ θ θ θl) is called the goodness-of-fit. Rissanen in [21] proved that if ˆ θlj is an MLE computed from njdata points and if nis large, then the precision of θlj can be effectively encoded with 1 2log2njbits. Rissanen derived a well-known form when all the parameters θlj are to be estimated by using all data points of size n. For subset selection in regression analysis, based on [22] it is MDL(kl)= − log2fl(y|ˆ θ θ θl)− kl X j=1 log2πlj(ˆ θlj)+kl 2log2n. (13) where klis the number of the regressors. Moreover, for n large, the choice of πlj(ˆ θlj) is relatively unimportant, as the resting summands in (13) are dominating [22]. So in practice for high n, term πlj(ˆ θlj) can be omitted for MDL. 4) MDL WITH MULTI-OBJECTIVE GOODNESS-OF-FIT We propose to replace the goodness-of-fit measure, which are commonly used in the MDL literature, namely MAE,MAPE RMSE etc. by the multiobjective goodness-of-fit measure. We will utilize this idea of both probabilistic and functional representation of models described above. If the probability function pis known: The second part in Eq. (10) is a goodness-of-fit of the model ppθ θ θ∗on data set D. In this paper, we replace in the value Lpθ θ θ∗(y|x):= Pn i=1−log2pθ θ θ∗(yi|xi) by L8θ θ θ∗(y|x)= − log28θ θ θ∗(y|x) (14) where 8θ θ θ∗(y|x) is a multi-objective criterion, and similarly, as above, the first part Lparam(θ θ θ∗) is the encoding of the selected model. The objective is to find θ θ θ∗at the minimum of (14) over all parameterizations. If only function flis known, the goodness-of-fit in (13) L(y|x):= − log2fl(y|ˆ θ θ θl)= n X i=1 −log2fl(yi|ˆ θ θ θl) (15) will be replaced analogically by (14). 19394 VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling 5) CONSIDERED MACHINE LEARNING METHODS Our work incorporated three types of machine learning methods which we call model classes: multiple linear regression (LR), multiple linear regression with a penalization term (penLR), polynomial regression (polREG) up to degree 3, and feed-forward neural networks (FF-NN). In this work, multilayer perceptrons with two and three hidden layers are used. In the following subsections, we construct the MDL descriptions of the above models for the multi-objective goodness-of-fit. 6) MDL FOR LINEAR REGRESSION WITH MULTI-OBJECTIVE GOODNESS-OF-FIT Giurcăneanu et al. in [28] constructed several informationtheoretic criteria for the variable selection by multiple linear regression assuming that the noise follows a Gaussian distribution. We will use their MDL derived from the stochastic complexity [29]. However, we replace their goodness-of-fit with the multi-objective goodness-of-fit. We denote k= |γ γ γ| the number of non-zero values in the binary vector γ γ γ, i. e. the number of regressors, and we can assume that k>0. Let β β βγ γ γ∈Rk+1be the vector of the unknown regression coefficients within the γ γ γ-subset. The matrix Xγ γ γis given by the columns of Xthat correspond to the γ γ γ-subset and the regression equation is y=Xγ γ γβ β βγ γ γ+εγ γ γ,(16) where y=(y1, . . . , yn) is the dependent variable and εγ γ γare Gaussian distributed with zero-mean and unknown variance τγ γ γ. Under the assumption that matrix Xγ γ γhas full-rank, the maximum likelihood (ML) estimates are ˆ β β βγ γ γ=(X⊤ γ γ γXγ γ γ)−1X⊤ γ γ γy(17) and ˆτγ γ γ= ∥y−Xγ γ γˆ β β βγ γ γ∥2 2/n(18) where ˆτγ γ γis a goodness-of-fit in Eq. (12) on regressor from γ γ γ. Paper [28] evaluated MDL of these regressions with independent variables indexed by γ γ γas functions depending on vector yand γ γ γas MDLLR(y, γ γ γ)=n−k 2log2ˆτγ γ γ+k 2log2 ∥Xγ γ γˆ β β βγ γ γ∥2 2 n −log20(n−k 2)−log20(k 2)+n 2log2(nπ) (19) where 0denotes the Euler integral of the second kind. In our MDL, we propose to replace ˆτγ γ γin (19) by a multi-objective criterion 8ˆ β β βγ γ γ(yi|xi), i.e. MDLLR 8(y,γ γ γ)=n−k 2log2( ˆ 8ˆ β β βγ γ γ(y|x) n) +k 2log2 ∥Xγ γ γˆ β β βγ γ γ∥2 2 n −log20(n−k 2)−log20(k 2)+n 2log2(nπ). (20) The objective is to find β β β∗at the minimum of (20) over all parameterizations ˆ β β βγ γ γand combinations of γ γ γ. 7) MDL FOR PENALIZED LINEAR REGRESSION WITH MULTI-OBJECTIVE GOODNESS-OF-FIT We express the encoding of the penalization part in regression as 1 2log2λfor fixed values of MLE of ˆ β β β. We use the same encoding of the regularization parameter for Lasso, Ridge and Elastic penalization. However, we stress that MDL minimization can be used only within the regression class with the same penalization type and not within all penalty types. Then MDLpenLR(y, γ γ γ) =n−k 2log2ˆτγ γ γ+1 2log2λ +k 2log2 ∥Xγ γ γˆ β β βγ γ γ∥2 2 n−log20(n−k 2)−log20(k 2) +n 2log2(nπ).(21) and MDLpenLR 8(y,γ γ γ) =n−k 2log2( ˆ 8ˆ β β βγ γ γ(y|x) n) +1 2log2λ+k 2log2 ∥Xγ γ γˆ β β βγ γ γ∥2 2 n −log20(n−k 2)−log20(k 2)+n 2log2(nπ).(22) It is well-known that Lasso, Ridge, and Elastic net regression can have various values for their regularization parameters. 8) MDL FOR POLYNOMIAL REGRESSION WITH MULTI-OBJECTIVE GOODNESS-OF-FIT Consider now set Mas a set of polynomial regression models of degree r≤r′. Denote ˆ θ θ θ=(ˆa0, . . . , ˆar) the set of coefficients in the polynomial of degree r. Since each ˆas,s= 0, . . . , ris a real number estimated from ndata points, each ˆas requires 1 2log2nbits to encode, the same the code for degree r. Thus L(ˆ θ θ θ)=L(ˆa0, . . . , ˆar)=r+1 2log2n+1 2log2n =r+2 2log2n.(23) The description of goodness-of-fit is L(y|ˆ θ θ θ)=n 2log2(RSSr n) (24) where RSSr=Pn i=1(yi−(ˆa0+ ˆa1xi+ · · · + (ˆarxr i))2.So the MDL for a polynomial of degree r≤r′is MDLpolREG(y,r)=r+2 2log2n+n 2log2(RSSr n) (25) VOLUME 13, 2025 19395 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling In our MDL, we propose to replace RSSr nin (25) by a multiobjective criterion 8ˆ θ θ θ(y|x), i.e. MDLpolREG 8(y,r)=r+2 2log2n+n 2log2( ˆ 8ˆ θ θ θ(y|x) n).(26) 9) MDL FOR A FEED-FORWARD NEURAL NETWORK WITH MULTI-OBJECTIVE GOODNESS-OF-FIT We generally consider a feed-forward network (FF-NN) with k≥1 hidden layers, each having hshidden units, s= 1, . . . , kand minput and poutput units. We propose a simple encoding for such models where the model description considers the encodings based on the encoding of the structure of the FF-NN and on the encoding of the learning part. a: THE ENCODING OF THE STRUCTURE The structure will be encoded as number of weights. In all hidden layers and in the output layer we considered ReLU activation function, and this is fixed for all FF-NN models. b: THE ENCODING OF THE LEARNING PART We encode the learning part of the FF-NN models so that we encode the learning rate lrt of the Adam optimizer, the batch size bs and the number of Adam hyperparameters Ahyp. Denote the vector of all parameters defining a FF-NN by θ θ θ. We do not encode the values of θ θ θexplicitely, but they are implicitely given by using Adam for their computation. Then MDLFF−NN (y,θ θ θ) =1 2log2(m×h1) +1 2log2(h1×h2)+ · · · + 1 2log2(hk×p) +1 2log2(lrt)+1 2log2(bs)+1 2log2(Ahyp) +n 2log2(RSSFF−NN n) (27) where RSSFF−NN is the residual sum of squares on the output of FF −NN and the values yand MDLFF−NN 8(y,θ θ θ) =1 2log2(m×h1)+1 2log2(h1×h2) + · · · + 1 2log2(hk×p) +1 2log2(lrt)+1 2log2(bs)+1 2log2(Ahyp) +n 2log2( ˆ 8ˆ θ θ θ(y|x) n).(28) where 8ˆ θ θ θ(y|x) is the multi-objective criterion applied on the output of FF −NN and the values of y. D. ANALYTIC HIERARCHY PROCESS (AHP) The Analytic Hierarchy Process (AHP) is a structured technique (Multiple Criteria Decision Analysis) for organizing and analyzing complex decisions [30]. It involves breaking down a problem into a hierarchy of subproblems that can be more easily comprehended and evaluated. The main steps in AHP are [31]: •To decompose the decision problem into a hierarchy. •To compare the elements at each hierarchy level to establish priorities. •To synthesize these comparisons to determine weights for each element. The consistency ratio (CR) [17] is calculated to ensure consistency in the comparisons: CR =CI RI (29) where CI is the consistency index, and RI is the random index. The consistency index (CI) measures the consistency of the pairwise comparisons. It is calculated as follows: CI =λmax −n n−1(30) where λmax is the largest eigenvalue of the comparison matrix, and nis the number of items being compared. The random index (RI) is the average consistency index of a randomly generated pairwise comparison matrix. The value of the RI depends on the number of items being compared and is used as a benchmark to assess the acceptability of the calculated CI. To implement AHP for model selection, we start by defining the criteria for model evaluation. For example, criteria such as Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Median Absolute Percentage Error (MdMAPE), Adjusted R-Squared (AdjR2), Prediction at 0.25 (Pred(0.25)), and Weighted Quantile Loss (wQL) can be used. Each model is evaluated based on these criteria through pairwise comparisons to determine their relative importance. The AHP process helps to synthesise these comparisons to assign a weight to each criterion, ultimately selecting the most suitable prediction model based on a comprehensive, structured evaluation. Each criterion has its own weight, which is set empirically or experimentally. IV. EXPERIMENTS This chapter outlines the experiments conducted using various regression model classes and two neural networks, namely multi-layered perceptron model classes to predict outcomes in the mentioned datasets. The experiments were divided into two main groups: Regression models and feed-forward neural networks. Each group utilized specific models’ families, evaluated based on their performance with the corresponding datasets. All experiments were implemented using Python and libraries pandas, numpy, sklearn, tensorflow and intertools. A. DATASETS The datasets employed in this research are widely acknowledged and are publicly accessible. The historical data utilized 19396 VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling TABLE 10. Regression results for AHP and MDL on GDP. performs best with the lowest value of 1066.64, indicating it has the least residual sum of squares. The Lasso and ElasticNet models using similar predictors also demonstrate competitive performance in this measure. The Polynomial Regression of Degree 3 with the predictors of Interest Rate, Industrial Production, Money Supply, and Personal Income is the best model when considering the AHP Score. The Polynomial Regression of the Degree 2 model stands out in terms of MDLAHP, and the Linear Regression model excels in MDLRSS. Thus, if all three criteria are taken into account, the Polynomial Regression of Degree 3 is the most optimal model due to its superior performance in AHP Score and competitive performance in MDLAHP and MDLRSS. Table 11 presents AHP,MDLAHP and MDLRSS for STOCK Dataset. The models tested include ElasticNet, Lasso, Linear Regression, Polynomial Regression (of degree 2 and 3), and Ridge Regression. Each model was evaluated with different predictors, specifically Open+High+Low and Date. The ElasticNet model, when using Open+High+Low as predictors, achieved an AHP Score of 0.52, an MDLAHP of −37733.67, and an MDLRSS of 67881.56. When using Date as the predictor, the AHP Score dropped to 0.04, while MDLAHP and MDLRSS were −57585.61 and 98004.58, respectively. The Lasso model with Low as the predictor had an AHP Score of 0.80, MDLAHP of −34271.85, and MDLRSS of 32672.06, whereas with Date as the predictor, the AHP Score was 0.04, MDLAHP was −57584.76, and MDLRSS was 98005.44. Linear Regression using Open+High+Low predictors achieved perfect AHP Scores of 1.00 with MDLAHP of −32535.83 and MDLRSS of 22562.16, while with Date as the predictor, the AHP Score was 0.04, MDLAHP was −57584.69, and MDLRSS was 98005.51. Polynomial Regression models (degrees 2 and 3) with Open+High+Low predictors also achieved perfect AHP Scores of 1.00, with MDLAHP and MDLRSS values of −32444.69 and 22851.12 for degree 2, and −32230.31 and 22619.67 for degree 3. Including the Date predictor alongside Open+High+Low in Polynomial Regression (degree 3) yielded similar results. The Ridge Regression model with Open+High+Low predictors had an AHP Score of 0.93, MDLAHP of −33062.29, and MDLRSS of 25693.66. Using Date as the sole predictor resulted in an AHP Score of 0.04, MDLAHP of −57584.69, and MDLRSS of 98005.51. B. FEED-FORWARD NEURAL NETWORKS-MULTI-LAYER PERCEPTRON Table 12 summarizes the performance of FF-NN models on the UCP dataset, evaluated through the AHP,MDLAHP, and MDLRSS. Two models, FF-NN I and FF-NN II, are compared using different combinations of predictors. The performance of FF-NN models on the UCP dataset was compared across various configurations of predictors. The results are summarised in Table 12. FF-NN I and FF-NN II models were evaluated with different predictors. For FF-NN I, when using the predictors UAW, UUCW, and ECF, the model achieved an AHP score of 0.71, with an MDLAHP of −31298.04 and an MDLRSS of 39384.06. However, when using TCF and ECF as predictors, the AHP score for FF-NN I dropped to 0.60, with MDLAHP and MDLRSS values of −31921.90 and 40464.73, respectively. In contrast, FF-NN II with the predictors UAW, UUCW, TCF, and ECF achieved the highest AHP score of 1.00, indicating a perfect performance with an MDLAHP of −30093.69 and an MDLRSS of 35846.81. When using only TCF as the predictor, FF-NN II had an AHP score of 0.62, and the MDLAHP and MDLRSS were −31764.88 and 40195.76, respectively. Comparing the models, FF-NN II consistently outperformed FF-NN I across all measures and predictor sets. This suggests that the additional complexity and parameters in FF-NN II provide a better fit for the UCP dataset. The combination of UAW, UUCW, TCF, and ECF yielded the best results for FF-NN II, achieving the highest AHP score and the lowest MDLRSS . This combination captures the relevant information more effectively than the other tested sets of predictors. The significant difference in performance measures between the two models and their predictor combinations Table 13 presents the performance VOLUME 13, 2025 19403 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling TABLE 11. AHP,MDLAHP and MDLRSS for regression models with stock dataset. TABLE 12. AHP, MDLAHP ,MDLRSS for FF-NN models with UCP dataset. of FF-NN models using different predictors combinations on the GLP dataset, evaluated using the AHP score, MDLAHP, and MDLRSS. The models compared are FF-NN I and FFNN II.For FF-NN I, when using the predictors AGE, DBP, SBP, TE, SPO2, HR, SHI, and DN, the model achieved an AHP score of 0.89, with an MDLAHP of −112604.29 and an MDLRSS of 44842.89. However, when using only DN as the predictor, the AHP score for FF-NN I dropped to 0.56, with MDLAHP and MDLRSS values of −117876.50 and 50788.08, respectively. In contrast, FF-NN II with the predictors AGE, DBP, SBP, TE, SPO2, HR, and SHI achieved an AHP score of 0.99, indicating near-perfect performance with an MDLAHP of −111319.67 and an MDLRSS of 42684.87. FF-NN II had an AHP score of 0.50 when using only DN as the predictor, and the MDLAHP and MDLRSS were −119184.98 and 50616.86, respectively. Additionally, when using all predictors (AGE, DBP, SBP, TE, SPO2, HR, SHI, and DN), FF-NN II achieved an AHP score of 0.99, with an MDLAHP of −111365.58 and an MDLRSS of 42619.76. Comparing the models, FF-NN II consistently outperformed FF-NN I across all measures and predictor sets. This suggests that the additional complexity and parameters in FF-NN II provide a better fit for the GLP dataset. The combination of AGE, DBP, SBP, TE, SPO2, HR, and SHI yielded the best results for FF-NN II, achieving the highest AHP score and the lowest MDLRSS . This combination captures the relevant health-related information more effectively than the other tested sets of predictors. The significant difference in performance measures between the two models and their predictor combinations highlights the importance of selecting appropriate predictors. The predictors AGE, DBP, SBP, TE, SPO2, HR, and SHI combined provide a robust model capable of accurately predicting the desired health outcomes in the GLP dataset. These results demonstrate the efficacy of using a more complex FF-NN model with a comprehensive set of predictors for superior performance in the GLP dataset. The next dataset GDP resulst are summarised in Table 14. The measures evaluated include the AHP Score, MDLAHP, and MDLRSS. The models were assessed based on different sets of predictors. For FF-NN I, when using the predictors Interest Rate, Unemployment Rate, Industrial Production, Money Supply, and Personal Income, the model achieved an AHP score of −2.86, with an MDLAHP of 12.86 and an MDLRSS of 1255.61. However, when using only Interest Rate as the predictor, the AHP score for FF-NN I remained at −2.86, with MDLAHP and MDLRSS values of 12.86 and 1255.71, respectively. In contrast, FF-NN II, with the predictors of Industrial Production and Personal Income, achieved the highest AHP 19404 VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling TABLE 13. AHP, MDLAHP ,MDLRSS for FF-NN models with GLP dataset. TABLE 14. AHP, MDLAHP ,MDLRSS for FF-NN models with GDP dataset. TABLE 15. AHP, MDLAHP ,MDLRSS for FF-NN models with STOCK dataset. TABLE 16. Selected FF-NN models per dataset and performance comparison. score of 1.00, indicating perfect performance with an MDLAHP of −294.06 and an MDLRSS of 849.81. When using Consumer Sentiment as the predictor, FF-NN II had an AHP score of 0.17, and the MDLAHP and MDLRSS were −415.77 and 1100.61, respectively. Comparing the models, FF-NN II consistently outperformed FF-NN I across all measures and predictor sets. This suggests that the additional complexity and parameters in FF-NN II provide a better fit for the GDP dataset. The combination of Industrial Production and Personal Income yielded the best results for FF-NN II, achieving the highest AHP score and the lowest MDLRSS . This combination captures the relevant economic information more effectively than the other tested sets of predictors. The significant difference in performance measures between the two models and their predictor combinations highlights the importance of selecting appropriate predictors. The Industrial Production and Personal Income predictors provide a robust model capable of accurately predicting the desired economic outcomes in the GDP dataset. These results demonstrate the efficacy of using a more complex FF-NN model with a comprehensive set of predictors for superior performance in the GDP dataset. For the last dataset (STOCK) the resulst are in Table 15. Resulst again constisting of scores for the AHP,MDLAHP, and MDLRSS For FF-NN I, when using the predictors High and Low, the model achieved an AHP score of 0.83, with an MDLAHP of −74826.91 and an MDLRSS of −2659.22. However, when using Open, High, and Low as predictors, the AHP score for FF-NN I dropped to 0.73, with MDLAHP and MDLRSS values of −75867.69 and 7134.96, respectively. In contrast, FF-NN II with the predictor Low achieved the highest AHP score of 1.00, indicating perfect performance with an MDLAHP of −73376.47 and an MDLRSS of −11413.85. When using Date and Open as predictors, FF-NN II had an AHP score of 0.83, and the MDLAHP and MDLRSS were −74826.41 and −2457.73, respectively. Comparing the models, FF-NN II consistently outperformed FF-NN I across all measures and predictor sets. This VOLUME 13, 2025 19405 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling TABLE 17. Selected regression models per dataset and performance comparison. suggests that the additional complexity and parameters in FF-NN II provide a better fit for the STOCK dataset. The predictor Low yielded the best results for FF-NN II, achieving the highest AHP score and the lowest MDLRSS . This combination captures the relevant stock price information more effectively than the other tested sets of predictors. The significant difference in performance measures between the two models and their predictor combinations highlights the importance of selecting appropriate predictors. The predictor Low provides a robust model that accurately predicts the desired stock price outcomes in the STOCK dataset. These results demonstrate the efficacy of using a more complex FF-NN model with a comprehensive set of predictors for superior performance in the STOCK dataset. C. DETAILED DISCUSSION The comparison between regression models and Multi-Layer Perceptron (FF-NN) models reveals several significant insights across different datasets, as highlighted in Tables 16 and 17. For the GDP dataset, the FF-NN II model achieved an MDLAHP of −415.77, substantially outperforming the Ridge regression model with Interest_Rate+Consumer_Sentiment predictors, which had an MDLAHP of −97.24. Similarly, the FF-NN II model with ‘‘Industrial_Production, Personal_Income’’ showed superior performance with an MDLAHP of −294.06, compared to the Linear Regression model using Interest_Rate+Industrial_Production +Personal_Income, which had an MDLAHP of 10.55. In the GLP dataset, the FF-NN II model with DN as the sole predictor demonstrated an MDLAHP of −119184.98, significantly better than the Ridge regression model with AGE, which had an MDLAHP of −57652.15. When multiple health indicators were used as predictors, FF-NN II again outperformed the Ridge regression model, achieving MDLAHP values of −111365.58 compared to −51369.50, respectively. For the STOCK dataset, FF-NN I, using Open, High, Low predictors, achieved an MDLAHP of −75867.69, much better than the Linear Regression’s −32535.83. The FF-NN II model using Low alone also performed exceptionally well with an MDLAHP of −73376.47, surpassing the ElasticNet regression model using ‘‘Date,’’ which had an MDLAHP of −57585.61. In the UCP dataset, FF-NN models consistently outperformed regression models. FF-NN I with TCF, ECF achieved an MDLAHP of −31921.90, better than ElasticNet with the same predictors, which had an MDLAHP of −14485.98. The FF-NN II model with UAW, UUCW, TCF, ECF showed an MDLAHP of −30093.69, outperforming ElasticNet and any other regression models tested. This analysis shows that FF-NN models generally outperform regression models across all datasets in terms of MDLAHP. This suggests that FF-NNs are more capable of capturing complex relationships within the data, which simple regression models might miss. However, while FF-NNs provide significant advantages due to their non-linearity and depth, they also come with higher computational costs and complexity, which can be a disadvantage regarding interpretability and ease of implementation. Across all datasets and models, MDLAHP consistently provides better performance measure than MDLRSS. For instance, in the GDP dataset, the MDLAHP for FF-NN II with −415.77, while MDLRSS is 1100.61, showing a contrast. This trend is observed across all datasets, underscoring that MDL with AHP is a better method for model selection. It better captures the trade-offs and multi-criteria evaluations inherent in complex model selection, which MDLRSS may oversimplify. However, one must consider that MDLAHP may also involve more subjective judgment in determining weights for different criteria, which can introduce bias. REFERENCES [1] F. Petropoulos, N. Kourentzes, K. Nikolopoulos, and E. Siemsen, ‘‘Judgmental selection of forecasting models,’’ J. Oper. Manage., vol. 60, no. 1, pp. 34–46, May 2018. [2] J. H. Friedman, ‘‘An overview of predictive learning and function approximation,’’ in From Statistics to Neural Networks: Theory and Pattern Recognition Applications (NATO ASI Series), vol. 136, V. Cherkassky, J. H. Friedman, and H. Wechsler, Eds. Berlin, Germany: Springer, 1994, doi: 10.1007/978-3-642-79119-2_1. [3] A. Boisbunon, ‘‘Model selection: A decision-theoretic approach,’’ M.S. thesis, Dept. IT Lab., Inf. Process. Syst., Université de Rouen, Rouen, France, 2013. [4] J. Rissanen, ‘‘Modeling by shortest data description,’’ Automatica, vol. 14, no. 5, pp. 465–471, Sep. 1978. [5] P. D. Grünwald, The Minimum Description Length Principle. Cambridge, MA, USA: MIT Press, 2007. [6] H. Akaike, ‘‘A new look at the statistical model identification,’’ IEEE Trans. Autom. Control, vol. AC-19, no. 6, pp. 716–723, Dec. 1974. [7] M. A. Villegas, D. J. Pedregal, and J. R. Trapero, ‘‘A support vector machine for model selection in demand forecasting applications,’’ Comput. Ind. Eng., vol. 121, pp. 1–7, Jul. 2018. [8] A. A. Ghobbar and C. H. Friend, ‘‘Evaluation of forecasting methods for intermittent parts demand in the field of aviation: A predictive model,’’ Comput. Oper. Res., vol. 30, no. 14, pp. 2097–2114, Dec. 2003. 19406 VOLUME 13, 2025 P. Silhavy et al.: MDL and Multi-Criteria Decision Analysis in Predictive Modeling [9] C.-O. Oh and B. J. Morzuch, ‘‘Evaluating time-series models to forecast the demand for tourism in Singapore: Comparing within-sample and postsample results,’’ J. Travel Res., vol. 43, no. 4, pp. 404–413, May 2005. [10] J. W. Taylor and P. E. McSharry, ‘‘Short-term load forecasting methods: An evaluation based on European data,’’ IEEE Trans. Power Syst., vol. 22, no. 4, pp. 2213–2219, Nov. 2007. [11] W. Han, X. Wang, F. Petropoulos, and J. Wang, ‘‘Brain imaging and forecasting: Insights from judgmental model selection,’’ Omega-Int. J. Manage. Sci., vol. 87, pp. 1–9, Sep. 2019. [12] A. Mardani, A. Jusoh, K. M. D. Nor, Z. Khalifah, N. Zakwan, and A. Valipour, ‘‘Multiple criteria decision-making techniques and their applications—A review of the literature from 2000 to 2014,’’ Econ. Res.-Ekonomska Istraž, vol. 28, no. 1, pp. 516–571, Jan. 2015, doi: 10.1080/1331677x.2015.1075139. [13] I. N. Durbach and T. J. Stewart, ‘‘Modeling uncertainty in multi-criteria decision analysis,’’ Eur. J. Oper. Res., vol. 223, no. 1, pp. 1–14, Nov. 2012, doi: 10.1016/j.ejor.2012.04.038. [14] Y. Badulescu, A.-P. Hameri, and N. Cheikhrouhou, ‘‘Evaluating demand forecasting models using multi-criteria decision-making approach,’’ J. Adv. Manage. Res., vol. 18, no. 5, pp. 661–683, Oct. 2021, doi: 10.1108/jamr05-2020-0080. [15] A. Bansal, B. Kumar, and R. Garg, ‘‘Multi-criteria decision making approach for the selection of software effort estimation model,’’ Manage. Sci. Lett., vol. 7, no. 6, pp. 285–296, 2017, doi: 10.5267/j.msl.2017.3.003. [16] L. G. Vargas and J. J. Dougherty, ‘‘The analytic hierarchy process and multicriterion decision making,’’ Amer. J. Math. Manage. Sci., vol. 2, no. 1, pp. 59–92, Jan. 1982, doi: 10.1080/01966324.1982.10737086. [17] T. L. Saaty, ‘‘How to make a decision—The analytic hierarchy process,’’ Eur. J. Oper. Res., vol. 48, no. 1, pp. 9–26, 1990. [18] F. Sarro, A. Petrozziello, and M. Harman, ‘‘Multi-objective software effort estimation,’’ in Proc. 38th Int. Conf. Softw. Eng., 2016, pp. 619–630, doi: 10.1145/2884781.2884830. [19] G. Schwarz, ‘‘Estimating the dimension of a model,’’ Ann. Statist., vol. 6, no. 2, pp. 461–464, Mar. 1978. [20] J. D. Angrist and J.-S. Pischke, Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton, NJ, USA: Princeton Univ. Press, 2009. [21] J. Rissanen, Stochastic Complexity in Statistical Inquiry, vol. 15. Singapore: World Scientific, 1998. [22] J. Rissanen, Information and Complexity in Statistical Modeling. Cham, Switzerland: Springer, 2007. [23] A. Barron, J. Rissanen, and B. Yu, ‘‘The minimum description length principle in coding and modeling,’’ IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2743–2760, Jun. 1998. [24] P. Grünwald and T. Roos, ‘‘Minimum description length revisited,’’ Int. J. Math. Ind., vol. 11, no. 1, Dec. 2019, Art. no. 1930001. [25] T. C. M. Lee, ‘‘An introduction to coding theory and the two-part minimum description length principle,’’ Int. Stat. Rev., vol. 69, no. 2, pp. 169–183, Aug. 2001. [26] C. E. Shannon, ‘‘A mathematical theory of communication,’’ Bell Syst. Tech. J., vol. 27, no. 3, pp. 379–423, Jul. 1948. [27] D. J. MacKay, Information Theory, Inference and Learning Algorithms. Cambridge, U.K.: Cambridge Univ. Press, 2003. [28] C. D. Giurcăneanu, S. A. Razavi, and A. Liski, ‘‘Variable selection in linear regression: Several approaches based on normalized maximum likelihood,’’ Signal Process., vol. 91, no. 8, pp. 1671–1692, Aug. 2011. [29] J. Rissanen, ‘‘MDL denoising,’’ IEEE Trans. Inf. Theory, vol. 46, no. 7, pp. 2537–2543, Jul. 2000. [30] T. L. Saaty, ‘‘What is the analytic hierarchy process?’’ in Mathematical Models for Decision Support. Cham, Switzerland: Springer, 1988, pp. 109–121. [31] M. Tavana, M. Soltanifar, and F. J. Santos-Arteaga, ‘‘Analytical hierarchy process: Revolution and evolution,’’ Ann. Oper. Res., vol. 326, no. 2, pp. 879–907, Jul. 2023. [32] R. Silhavy, P. Silhavy, and Z. Prokopova, ‘‘Analysis and selection of a regression model for the use case points method using a stepwise approach,’’ J. Syst. Softw., vol. 125, pp. 1–14, Mar. 2017. [33] Z. Prokopova, R. Silhavy, and P. Silhavy, ‘‘The effects of clustering to software size estimation for the use case points methods,’’ in Proc. Comput. Sci. On-Line Conf., in Advances in Intelligent Systems and Computing, vol. 575, Jan. 2017, pp. 479–490. [34] P. Silhavy, R. Silhavy, and Z. Prokopova, ‘‘Evaluation of data clustering for stepwise linear regression on use case points estimation,’’ in Proc. Comput. Sci. On-Line Conf., in Advances in Intelligent Systems Computing, vol. 575, 2017, pp. 491–496. [35] R. Silhavy, P. Silhavy, and Z. Prokopova, ‘‘Evaluating subset selection methods for use case points estimation,’’ Inf. Softw. Technol., vol. 97, pp. 1–9, May 2018. [36] D. Javale and S. Desai, ‘‘Dataset for people for their blood glucose level with their superficial body feature readings,’’ IEEE Dataport, Jul. 2021, doi: 10.21227/c4pp-6347. [37] Yahoo Finance. Stock Market Data. Accessed: Jul. 11, 2024. [Online]. Available: https://finance.yahoo.com [38] C. S. K. Dash, A. K. Behera, S. Dehuri, and A. Ghosh, ‘‘An outliers detection and elimination framework in classification task of data mining,’’ Decis. Anal. J., vol. 6, Mar. 2023, Art. no. 100164. [39] S. Louis. (2024). Gross Domestic Product. Accessed: Jul. 10, 2024. [Online]. Available: https://fred.stlouisfed.org/series/GDP [40] N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, ‘‘Dropout: A simple way to prevent neural networks from overfitting,’’ J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, Jan. 2014. [41] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016. [42] D. P. Kingma and J. Ba, ‘‘Adam: A method for stochastic optimization,’’ 2014, arXiv:1412.6980. [43] L. Prechelt, ‘‘Early stopping-but when?’’ in Neural Networks: Tricks of the Trade. Cham, Switzerland: Springer, 1998, pp. 55–69. PETR SILHAVY received the Ph.D. degree in engineering informatics from the Faculty of Applied Informatics, Tomas Bata University in Zlín, Zlín, Czech Republic, in 2009. He is currently an Associate Professor with the Faculty of Applied Informatics, Tomas Bata University in Zlín. He is a Senior Research and an Associate Professor of system engineering and informatics with a demonstrated history of working in research and higher education. He has expertise as a CTO and a Software Developer in database programming, database design, data management, and data science. His research interests include prediction and empirical methods for software engineering. KATEŘINA HLAVÁČKOVÁ-SCHINDLER received the M.Sc. degree (summa cum laude) in mathematics from Charles University, Prague, Czech Republic, the Ph.D. degree in computer science from Czech Academy of Sciences, and the Habilitation (Privatdoz) degree from University of Vienna, Vienna, Austria. She is currently a Senior Scientist with the Data Mining and Machine Learning Research Group, University of Vienna. She has more than 80 publications mostly on causal inference and causal discovery, machine learning, and artificial neural networks. RADEK SILHAVY received the Ph.D. degree in engineering informatics from the Faculty of Applied Informatics, Tomas Bata University in Zlín, Zlín, Czech Republic, in 2009. He is currently an Associate Professor and a Senior Researcher with the Faculty of Applied Informatics, Tomas Bata University in Zlín. He is an Associate Professor of system engineering and informatics with a demonstrated history of working in research, higher education, project management, and software analysis. His research interests include predictive analytics for software engineering, empirical methods in software engineering, or prediction models focused on cost, size, and effort estimations in system/software engineering. He is also involved in academic publishing as the editor-in-chief, an editor, and a reviewer. VOLUME 13, 2025 19407