Support vector machines in reliability calculations of engineering structures

Support vector machines in reliability calculations of engineering structures M. ˇ Somod´ ıkov´ a & D. Lehk´ y Institute of Structural Mechanics, Faculty of Civil Engineering, Brno University of Technology, Brno, Czech Republic ABSTRACT: In the paper, a metamodeling approach based on support vector regression is studied as a promising tool in the assessment of reliability level. The method consists of two steps: firstly, an approximation of the original limit state function is performed, and in the second step a failure probability or reliability index is calculated with a simpler, approximated function using traditional simulation techniques. Two problems with explicit limit state functions are used to study the effectivity of the method. In order to be as effective as possible with respect to computational effort, a stratified Latin Hypercube Sampling simulation method is utilized to properly select training set elements. The accuracy of the method is analyzed and compared with other surrogate modeling methods, namely the polynomialand artificial neural network-based response surface method, achieving comparable results. 1 INTRODUCTION In the field of structural engineering, assessing the reliability of a structure is crucial to ensuring its safety and performance throughout its service life. Determining the level of reliability is an important aspect in both the design of new structures and the verification of existing ones. When using a probabilistic approach in structural design and assessment, the reliability level associated with a particular limit state is quantified through reliability indicators such as failure probability pf, or reliability index β, using the predefined limit state function (LSF). The LSF can be defined as either an explicit function of random variables or obtained from non-linear finite element method analysis. In such cases, calculating the structural response of complex systems, such as bridges, is usually a time-consuming task, and therefore the use of approximation methods that reduce the computational effort to an acceptable level seems to be an appropriate solution. The basic principle of approximation methods (also referred to as metamodels or surrogate models) is to replace the original LSF (in its explicit or implicit form) with a simpler, approximated function whose evaluation is computationally less demanding. The failure probability is then calculated using traditional simulation techniques, e.g. crude Monte Carlo (MC), using the approximated function instead of the original one. Traditional approximation methods include the first-order reliability method (Hasofer & Lind 1974) and the second-order reliability method (Fiessler et al. 1979). There has also been a growing interest in applying advanced machine learning techniques to enhance the efficiency and accuracy of reliability predictions. With the recent development of metamodeling and the application of soft computing techniques, the response surface method (Myers 1971, Bucher & Bourgund 1990) has gained popularity as an effective approximation approach to solve reliability in complex engineering problems. Other methods that have become increasingly popular among researchers in recent decades include artificial neural networks (Lehk´ y & ˇ Somod´ ıkov´ a 2017), polynomial chaos expansion (Ghanem & Spanos 1991), Kriging metamodel (Kaymaz 2005) and also support vector machines (Li et al. 2006, Roy et al. 2019). In this contribution a support vector regression is studied as a promising tool in the assessment of reliability level. The efficiency and accuracy of the method are analyzed and compared with other approximation methods, namely the polynomialand artificial neural network-based response surface method. SEMC 2025: Author’s accepted manuscript 2 FORMULATION OF THE PROBLEM 2.1 Support vector regression (SVR) Support Vector Machines (SVMs), a powerful supervised machine learning algorithm first formulated and presented in Boser et al. (1992), is commonly used for linear or nonlinear classification, regression and even outlier detection tasks. In the context of structural reliability analysis, SVMs are increasingly being utilized to model complex relationships between input variables and the structural response or probability of failure. SVMs have shown considerable potential in structural reliability analysis due to their ability to handle high-dimensional data, nonlinear relationships, and inherent uncertainties (such as material properties, load conditions, and environmental factors). By mapping the input data into a higherdimensional feature space, SVMs can efficiently classify and predict the reliability of structural systems, offering a powerful tool for engineers and researchers. An SVR is a type of machine learning algorithm used for regression tasks. It is based on the principles of SVMs, which are used primarily for classification. However, while SVMs seeks to find a hyperplane that best separates data into different classes, SVR tries to find a function that approximates the relationship between inputs (features) and outputs (targets). Suppose that we are given training data {(x1,y1),(x2,y2),...(xn,yn)}, where each xi∈X is a p-dimensional vector of input features and Y with yi∈Ydenotes the vector of target values. In ε-SVR (Vapnik 2000), the main goal is to find a function f(X)that deviates by no more than εfrom the true target values yifor all training data, while also being as flat as possible. In other words, errors are acceptable as long as they do not exceed the precision ε(the so-called ε-margin), but any deviation beyond this threshold is not tolerated. This approach makes the model more robust to small errors. In its simplest form, let us consider a linear function that predicts the target variable yin the form: f(X) = wTx+b , (1) where wis the normal vector to the hyperplane and b the bias term. SVR formulates the problem as an optimization problem in which the goal is to minimize the complexity of the model (by minimizing w) while keeping the prediction errors as small as possible. One way to ensure this is to minimize the norm, i.e. ∥w∥2. The problem can be described as a convex optimization problem (Smola & Sch¨ olkopf 2004): minimize 1 2∥w∥2 subject to yi−wTxi−b≤ε wTxi+b−yi≤ε (2) For cases when the optimization problem is not feasible, i.e. the function f(X)cannot approximate all pairs (xi,yi)with εprecision, or if we want to allow some errors, the so-called slack variables ξi,ξ∗ iare introduced. The optimization problem is then modified to (Vapnik 2000): minimize 1 2∥w∥2+C l X i=1 ξi+ξ∗ i subject to    yi−wTxi−b≤ε+ξi wTxi+b−yi≤ε+ξ∗ i ξi,ξ∗ i≥0 (3) The regularization parameter C > 0controls the trade-off between the complexity of the model (represented by w) and the number of errors that exceed ε.εis a threshold value that defines a margin around the predicted value where errors are tolerated without penalty. If the difference between the true value yand the predicted value f(X)exceeds ε, a linear penalty proportional to the error beyond the margin is applied. The situation is depicted in Figure 1. Figure 1: The soft margin loss function for linear SVR The primal optimization problem in Equation 3 can be solved more easily in its dual formulation using Lagrange multipliers, as described in e.g. Fletcher (1989). Moreover, the dual formulation provides the key for extending SVMs to nonlinear functions, which is expected given the complexity of the phenomena being modeled. The so-called kernel trick is used in nonlinear SVR. Instead of solving the problem in the original input space, the kernel function allows the optimization to be done in a higher-dimensional feature space, where a linear solution can be found. Common kernel functions include the Radial Basis Function (RBF), polynomial kernels, and others. 2.2 Artificial neural network-based response surface method (ANN-RSM) As another surrogate modeling technique, artificial neural networks (ANNs) have been shown to be a very effective tool for modeling the functional relationship between model inputs and outputs due to their ability to adapt themselves to special environmental conditions by changing the connection SEMC 2025: Author’s accepted manuscript strengths or structure. An ANN is a signal processing system composed of simple processing elements, called artificial neurons, that are interconnected by direct links (weighted connections). The ANN serves as a surrogate model ˜g(X)for the approximation of an original LSF, with Xbeing the vector of inputs. p1 p2 pkb w1 w2 wk fy Figure 2: Example of a feed-forward ANN (left) and a single neuron model (right) In the method used in this contribution, a feedforward multi-layer network is used (Cichocki and Unbehauen 1993, Kara et al. 2020, Eser et al. 2021), with artificial neurons organized into different layers – an input layer, several hidden layers, and an output layer. The neurons in each layer are fully connected to the neurons in adjacent layers through unidirectional links, which are characterized by synaptic weights. These weights serve as signal multipliers on the corresponding interconnections. Connections within a layer are not allowed (see Fig. 2). In such a network type the input signal only moves in the forward direction, i.e. the data go from the input nodes through the hidden nodes (if any) to the output nodes. If the output vector of the whole neural network is required, output vectors have to be calculated layer by layer from the input layer to the output layer of the network. The output from a single neuron is calculated as (Fig. 2 right): y=f(x) = f X k (wk·pk+b)!,(4) where kindicates the input number, wkis the synaptic weight of the connecting path from the kth neuron of the previous layer, pkis the input signal coming from the kth neuron of the previous layer, bis the bias of the neuron and fis a transfer function of the neuron. Training the network (i.e. assessment of values of synaptic weights and biases) is an optimization task similar to finding a function in SVR. A set of example pairs of inputs and corresponding outputs (pi,yi), pi∈P,yi∈Yis introduced to the network. The aim of the subsequent optimization procedure is to find a neural network function fANN :P→Yin the allowed class of functions that matches the examples. Such an optimization consists in minimizing the criteria: E=1 2 N X i=1 K X k=1 y0 ik −y∗ ik2,(5) where Ndenotes the number of ordered input–output pairs in the training set, y∗ ik is the required new value of the kth output neuron at the ith input, and y0 ik is the actual output value of the kth output neuron at this input. There are various optimization methods by which the minimization of criteria Eis achieved, in the examples presented below in the text, a gradient descent method was used. 2.3 Polynomial-based response surface method (POLY-RSM) The method consists in approximating the LSF by a suitable function, usually of polynomial type. An efficient method for approximating complex systems was developed by Bucher et al. (1989). In general, the second-order polynomial function is most often used in its full form: f(X) = a+ n X i=1 bixi+ n X i=1 n X j=1 cijxixj,(6) where xi,i= 1,...,nare the input basic variables and parameters a,bi,ciare the unknown regression coefficients of the approximation function that are obtained by conducting a series of numerical experiments with input variables selected according to an experimental design. An iterative response surface approach based on a simplified polynomial function without mixed terms was presented in Bucher & Bourgund (1990) in the form: f(X) = a+ n X i=1 bixi+ n X i=1 cix2 i.(7) with the same notation as in Equation 6. The values of the interpolation points are situated around the mean values for the initial approximation. 3 NUMERICAL EXAMPLES Two examples are taken to check the accuracy and efficiency of the approximation based on SVR to calculate the reliability level. In order to observe an approximate effect, those examples are chosen to have an explicit limit state function. 3.1 Example 1 – a quadratic function A quadratic LSF (Guan & Melchers 2001) has been used as a test function for the case of the application of SVR in the calculation of reliability level. The function is defined as a 3D problem with 2 variables (features) in the form: g(X)=4−0.16(x1−1)2−x2(8) with x1and x2being the standard normal variables with zero mean and unit standard deviation. The “exact” value of the failure probability and the reliability index are considered to be pf= 8.15 ×10−4and SEMC 2025: Author’s accepted manuscript β= 3.15 (calculated as an average from 10 runs of 10 million MC simulations of the original LSF). Analyzes with n= 20,30,40,50,60 and 100 training data were performed to study the accuracy and capability of SVR to approximate the original LSF. The stratified Latin Hypercube Sampling method was used to generate input random variables. At first, the approximation with the basic random variables generated according to the stochastic model was used. In the next step, the standard deviation was set to twice, three times, and five times of the initial value in order to cover a wider range of the space of random variables, and hence to obtain a larger number of negative LSF values. When the surrogate model was fit, pfand βwere then calculated ten times using 1 million classical MC simulations generated based on the stochastic model. The whole procedure was repeated in five runs to account for the variability of the results. Note that the selection of kernel parameters has a great impact on the computational time and accuracy of the approximated function and would deserve in-depth research, which, however, is beyond the scope of this paper. In the examples presented, the selection is performed using the cross-validation technique (Chang & Lin 2011). The resulting βvalues in terms of the mean value and standard deviation are depicted in Figure 3 for different kernels. The comparison of pfand βwith the exact values is summarized in Table 1. The linear (SVR-lin), the 2nd degree (SVR-poly2) and the 3rd degree (SVR-poly3) polynomials, and radial basis function (SVR-RBF) kernels were used in the analyzes. Figure 3: Resulting reliability indices for the example with quadratic function Some conclusions and recommendations can be drawn from the obtained results: (i) Since the given LSF is a quadratic function, SVR-poly2 achieved very accurate results with minimal error for all cases; coefficient of variation (COV) of βwas in the range of 0.3–3.3 ˙ %. (ii) If the training set is generated with only 1 standard deviation (Fig. 3 upper left), the COVs are almost constant for all the kernels and are in the range of 0.7–3.5 %. SVR-RBF cannot fit the shape of the resulting surface in cases where all functional values Table 1: Mean values of failure probability pf(×10−4)and reliability index β(in brackets) for the quadratic function. nSVR-lin SVR-poly2 SVR-poly3 SVR-RBF 1 standard deviation: 20 1.44 (3.44) 3.41 (3.41) 71.60 (2.45) 0 (-) 30 1.77 (3.58) 3.99 (3.37) 72.07 (2.45) 0 (-) 40 1.95 (3.55) 5.19 (3.29) 61.11 (2.51) 0 (-) 50 2.03 (3.54) 4.40 (3.33) 71.87 (2.46) 0 (-) 60 1.89 (3.56) 5.40 (3.27) 51.26 (2.57) 0 (-) 100 1.84 (3.56) 5.16 (3.28) 30.37 (2.75) 0 (-) 2 standard deviations: 20 4.68 (3.38) 6.48 (3.22) 10.45 (3.15) 7.82 (3.18) 30 4.70 (3.34) 6.69 (3.21) 4.90 (3.33) 10.79 (3.08) 40 4.84 (3.53) 6.70 (3.21) 4.82 (3.31) 11.78 (3.06) 50 2.77 (3.47) 7.03 (3.19) 6.48 (3.22) 9.00 (3.12) 60 3.43 (3.40) 6.68 (3.21) 6.19 (3.23) 9.69 (3.10) 100 3.64 (3.38) 7.48 (3.18) 6.70 (3.21) 8.43 (3.14) 3 standard deviations: 20 7.24 (3.27) 6.72 (3.21) 1.82 (3.69) 8.54 (3.16) 30 12.02 (3.07) 7.18 (3.19) 3.79 (3.40) 10.29 (3.10) 40 13.28 (3.02) 7.29 (3.18) 5.71 (3.25) 9.72 (3.11) 50 7.68 (3.20) 7.23 (3.19) 5.95 (3.24) 9.74 (3.10) 60 10.62 (3.12) 7.57 (3.17) 6.29 (3.23) 7.36 (3.18) 100 13.58 (3.02) 7.71 (3.17) 6.54 (3.21) 7.80 (3.17) 5 standard deviations: 20 1274 (1.34) 6.97 (3.20) 2.13 (3.75) 14.48 (3.07) 30 395 (1.88) 8.00 (3.16) 5.21 (3.28) 10.42 (3.09) 40 290 (2.04) 7.59 (3.17) 6.38 (3.22) 8.37 (3.15) 50 329 (1.86) 7.75 (3.17) 6.67 (3.21) 9.38 (3.11) 60 321 (1.89) 7.82 (3.16) 6.64 (3.21) 10.91 (3.07) 100 386 (1.80) 7.83 (3.16) 6.90 (3.20) 9.59 (3.10) Exact values: pf= 8.15 ×10−4(β= 3.15) computed based on the generated inputs are only positive (see the null values of pfin Tab. 1). (iii) With a larger space covered by training data (generated with 2, 3 and 5 standard deviations), the trend of decreasing variance of βwith increasing nis quite evident. A sufficient number of training data can be considered as 40–50. The larger the range, the greater the impact of selecting the correct kernel on the results (see Fig. 3 bottom right, where SVR-lin achieved unreasonable results). (iv) As the number of training data increases, the accuracy of SVR-RBF also increases with a very small variability (COV = 0.7–8.5 %). Thus, RBF can be used as a universal kernel for cases where the shape of the region of interest is not known. For illustrative purposes, Figures 4, 5 and 6 show the training data (40 realizations generated with three times the original standard deviation) and the approximated functions for different kernels including the million random MC simulations (clusters of colored points in Figures 5 and 6). In all these figures, red points are situated in the region of failure. 3.2 Example 2 – a pitched-roof frame As the second example a simple pitched-roof frame (Fig. 7) was taken from the civil engineering reliability literature (Grigoriu 1982). The LSF was approximated using Lagrangian interpolation polynomials as: g(X) = ax3 2+bx2 2+cx2−x1+d(9) SEMC 2025: Author’s accepted manuscript Figure 4: Training data for the example with quadratic function Figure 5: SVR for the example with quadratic function – approximations with different kernels in 2D plot Figure 6: SVR for the example with quadratic function – approximations with different kernels in 3D plot with the parameters given as follows: a=−0.36355, b= 1.18046,c=−1.0892988,d= 4.2042064. Variables x1and x2represent the effect of the horizontal, H, and vertical, V, loads normalized by a limit plastic moment, mp, of the cross section (x1=Ha/mp, x2=V a/mp). Both are normally distributed with parameters defined in Table 2. Based on the findings of the previous example, training data for n= 20,30,40,50,60 were generated with five times the value of the standard deviation defined in the stochastic model (Tab. 2). An example of Figure 7: Diagram of the pitched-roof frame such a training set with n= 50 is shown in Figure 8. The values of pfand βwere then calculated using approximated functions as the average of ten repeated calculations with 1 million MC simulations generated based on the stochastic model with five different training data sets. Results of the SVR approximations are shown in Figures 9 and 10 for linear, the 2nd and the 3rd degree polynomials, and radial basis function kernels, as in Example 3.1. Table 2: Basic random variables of the pitched-roof frame. Variable Distribution Mean Standard deviation x1Normal 2.763 0.4 x2Normal 1.250 0.4 Figure 8: Training data for the example of the pitched-roof frame Figure 9: SVR for the example of the pitched-roof frame – approximations with different kernels in 2D plot Finally, the results of different approximation methods were compared with the exact values of the reliability indicators, which are considered to be pf= 2.21 ×10−3and corresponding β= 2.847 (the values were also calculated ten times using 1 million MC simulations resulting in average values of pf= 2.17 ×10−3and β= 2.850, which are almost identical to the exact ones). Figure 11 shows the results for POLY-RMS and ANN-RSM, which the authors previously presented in Lehk´ y & ˇ Somod´ ıkov´ a SEMC 2025: Author’s accepted manuscript Figure 10: SVR for the example of the pitched-roof frame – approximations with different kernels in 3D plot Figure 11: Comparison of βvalues obtained via different methods for the example of a pitched-roof frame (2017). SVR-poly3 and SVR-RBF are added as SVM representatives for comparison, achieving comparable results in terms of βmean value. However, the variability is higher (COV up to 7.7 % for SVR-poly3, 9.2 % for SVR-RBF compared to 1.2 % for ANNRSM and 4.9 % for POLY-RSM). 4 CONCLUSIONS SVR has proven to be very accurate and efficient in calculating the reliability level. However, a higher variability of results was observed. While SVR has been found to approximate the original LSF very well, especially when a wider space is covered by the training data, the POLY-RSM and ANN-RSM approximations are accurate only in the region of interest and can significantly deviate from the exact LSF in the rest of the space. In future work, the focus will be on updating the SVR function with training data generated from around the design point such that the original LSF is best approximated close to the failure region; see (Lehk´ y and ˇ Somod´ ıkov´ a 2017, Lehk´ y et al. 2022). ACKNOWLEDGEMENT The research was co-funded by the European Union under the project INODIN, No. CZ.02.01.01/00/23 020/0008487, and the project MMOSS No. 25-14337L supported by the Czech Science Foundation. REFERENCES Boser, B. E., I. M. Guyon, & V. N. Vapnik (1992). A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. Bucher, C., Y. Chen, & G. Sch¨ uller (1989). Time variant reliability analysis utilizing response surface approach. In P. ThoftChristensen (Ed.), Reliability and Optimization of Structural Systems, Volume 48, pp. 1–14. Berlin: Springer. Bucher, C. G. & U. Bourgund (1990). A fast and efficient response surface approach for structural reliability problems. Structural Safety 7(1), 57–66. Chang, C.-C. & C.-J. Lin (2011). Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2(3), 1–27. Cichocki, A. & R. Unbehauen (1993). Neural Networks for Optimization and Signal Processing. Stuttgart: Wiley, Teubner. Eser, A., A. A. Ayyıldız, M. Ayyıldız, & F. Kara (2021). Artificial intelligence-based surface roughness estimation modelling for milling of aa6061 alloy. Advances in Materials Science and Engineering 2021, 1–9. Fiessler, B., H.-J. Neumann, & R. Rackwitz (1979). Quadratic limit states in structural reliability. Journal of Engineering Mechanics ASCE 105(4), 661–676. Fletcher, R. (1989). Practical Methods of Optimization. New York: John Wiley and Sons. Ghanem, R. G. & P. D. Spanos (1991). Stochastic Finite Elements: A Spectral Approach. Springer. Grigoriu, M. (1982). Methods for approximate reliability analysis. Structural Safety 1(2), 155–165. Guan, X. L. & R. E. Melchers (2001). Effect of response surface parameter variation on structural reliability estimates. Structural Safety 23(4), 429–444. Hasofer, A. M. & N. C. Lind (1974). Exact and invariant secondmoment code format. Journal of the Engineering Mechanics Division 100(1), 111–121. Kara, F., M. Karabatak, M. Ayyıldız, & E. Nas (2020). Effect of machinability, microstructure and hardness of deep cryogenic treatment in hard turning of aisi d2 steel with ceramic cutting. Journal of Materials Research and Technology 9(1), 969–983. Kaymaz, I. (2005). Application of kriging method to structural reliability problems. Structural Safety XXVII(2), 133–151. Lehk´ y, D. & M. ˇ Somod´ ıkov´ a (2017). Reliability calculation of time-consuming problems using a small-sample artificial neural network-based response surface method. Neural Computing & Applications 28(6), 1249–1263. Lehk´ y, D., M. ˇ Somod´ ıkov´ a, & M. Lipowczan (2022). A utilization of the inverse response surface method for the reliabilitybased design of structures. Neural Computing and Applications 34(15), 12845–12859. Li, H., Z. L¨ u, & Z. Yue (2006). Support vector machine for structural reliability analysis. Journal of Applied Mathematics and Mechanics 27, 1295–1303. Myers, R. H. (1971). Response Surface Methodology. Allyn and Bacon, Inc. Roy, A., R. Manna, & S. Chakraborty (2019). Support vector regression based metamodeling for structural reliability analysis. Probabilistic Engineering Mechanics 55, 78–89. Smola, A. J. & B. Sch¨ olkopf (2004). A tutorial on support vector regression. Statistics and Computing 14(3), 199–222. Vapnik, V. N. (2000). The Nature of Statistical Learning Theory (2nd ed.). New York, NY: Springer. SEMC 2025: Author’s accepted manuscript