From fault detection to one-class severity discrimination of 3D printers with one-class support vector machine

From fault detection to one-class severity discrimination of 3D printers with one-class support vector machine Chuan Li a,1, Diego Cabrera a,b,∗,1, Fernando Sancho c, Mariela Cerrada b, René-Vinicio Sánchez b, Edgar Estupinan d a National Research Base of Intelligent Manufacturing Service, Chongqing Technology and Business University, Chongqing 400067, China b GIDTEC, Universidad Politécnica Salesiana, Ecuador c Department of Computer Science and Artificial Intelligence, Universidad de Sevilla, Spain d Department of Mechanical Engineering, University of Tarapaca, Arica, Chile Keywords: Fault detection Severity discrimination One-class support vector machine 3D printer Bidirectional generative adversarial network a b s t r a c t The lack of faulty condition data reduces the feasibility of supervised learning for fault detection or fault severity discrimination in new manufacturing technologies. To deal with this issue, one-class learning arises for building binary discriminative models using only healthy condition data. However, these models have not been extrapolated to severity discrimination. This paper proposes to extend OCSVM, which is typically used for fault detection, to 3D printer fault severity discrimination. First, a set of features is extracted from a set of normal signals. An optimized OCSVM model is obtained by tuning the kernel and model hyperparameters. The resulting models are evaluated for fault detection and fault severity discrimination using a proposed performance evaluation approach. Experimental comparisons for belt-based faults in 3D printers show that the distance to the hyperplane has the information to discriminate the severity level, and its use is feasible. The proposed hyperparameter optimization technique improves the OCSVM for fault detection and severity discrimination compared to some other methods. 1. Introduction Fault detection and severity assessment are two fundamental tasks of system health management. In this context, fault (anomaly) detection is the determination of whether a system is in normal working condition or some component contains a fault. Anomaly detection has been extensively studied in such fields as abnormal activity recognition [1], communication networks [2], malicious file detection [3], fault detection in power transmission lines and distribution systems [4], and industrial condition monitoring [5]. In the last field, although model-based approaches have been proposed [6], the current trend is to use data-driven techniques for reasons of flexibility and accuracy [7]. In data-driven approaches, a detection model is built using data collected from the monitored system. Informative features ✩Acknowledgments: The work was sponsored in part by GIDTEC Research Group of Universidad Politécnica Salesiana, the National Natural Science Foundation of China (51775112), the MoST Science and Technology Partnership Program (KY201802006), the Key Project of the Chongqing Natural Science Foundation (cstc2019jcyj-zdxmX0013), and the CTBU Open Project (KFJJ2019059) ∗Corresponding author at: GIDTEC, Universidad Politécnica Salesiana, Ecuador. E-mail address: [email protected] (D. Cabrera). 1The two authors contributed equally to this paper. are extracted from raw signals. A classifier is then created using supervised, semi-supervised [8], unsupervised, or one-class learning [9]. Although supervised learning is superior to oneclass learning in diverse applications [10,11], it only applies when labeled data in both normal and abnormal conditions can be obtained from the system, and it is usually expensive. Semisupervised and unsupervised learning typically require fewer labeled examples [12], but still need data in every system condition. Evolving approaches [13] address the issue of data availability in the training stage by recognizing new patterns in the testing stage and including them in the base of knowledge. These approaches require retraining of classification models after identifying a new cluster, which could be computationally expensive. One-class learning only uses data in normal condition to build the classifier, showing clear advantages in the absence of abnormal data and requiring less computation. The popular OCSVM [14] is a one-class learning model that has shown versatility and success in combination with feature extraction techniques for different applications [15,16]. It aims to build a decision hyperplane in a projected space of the input features. The hyperplane has samples in normal condition on one side, and abnormal ones on the other. The hyperplane is optimized using only data in normal condition. Nomenclature Symbols ∥x∥1L1 norm of x ∥x∥2L2 norm of x α,βLagrange multipliers ·Inner product γKernel hyperparameter RdSpace of input examples RnFeature space RTSignal space of length T ∇Gradient operator νOCSVM hyperparameter ⊘Element-wise division ρOffset of hyperplane ⊤Transpose operator ξiSlack variable of ith training example wVector normal to hyperplane Abbreviations 1DCNN One-dimensional convolutional neural network 1DDNN One-dimensional deconvolutional neural network ase Average severity error BiGAN Bidirectional generative adversarial network FNR False-negative rate, rate of normal condition examples classified as faulty condition examples FPR False-positive rate, rate of faulty condition examples classified as normal condition examples HVAC Heating, ventilation, and air-conditioning system OCSVM One-class support vector machine RBF Radial basis function OCSVM has the advantage that few hyperparameters (typically two for OCSVM with an RBF kernel) are required from the user. In industrial condition monitoring, grid searching combined with cross-validation is most often used for hyperparameter optimization. It has been successfully applied to supervised learning [17, 18], but cannot be used with one-class learning, considering the unavailability of faulty data. For example, grid searching has been used to optimize OCSVM hyperparameters in bearing fault detection [19,20] and HVAC anomaly detection [21], but such applications require a validation set with abnormal data, which violates the restriction of one-class learning. OCSVM hyperparameter configuration is most often accomplished by setting them to specific values or by rule of thumb. Such approaches have been applied to fault detection of complex industrial processes [22,23], bearing fault detection [24], simulated dynamical system anomaly detection [25], and HVAC system fault detection [26,27]. However, they suffer from lack of evidence that the hyperparameters are optimally configured, hence better performance could be possible. Considering the lack of anomalous data in OCSVM applications, heuristics-based hyperparameter optimization methods have been proposed. Specifically, [28] proposed the searching of γby optimizing the ratio between the variance and a metric of the kernel matrix. [29] presented a method for optimizing νand γby a trade-off between the number of support vectors and the maximization of the objective function. [30] introduced an heuristic based on a metric over the Gaussian space for optimizing γ. The previous works have shown a high performance in synthetic and applications different to severity discrimination, and their evaluation was performed only for RBF kernels. Therefore, other heuristics or the generalization of the above methods to other kernels are open study field. Fault severity discrimination is a more advanced task that requires the classification of a severity condition for a detected fault. It has been studied from a supervised learning perspective in applications of a wind turbine gearbox [31], helical gearbox [32,33], and bearings [34]. As before, data availability for all severity levels is assumed, but is infeasible in practice. From the above, we identify three issues. (i) Information about the fault severity that one-class models can contain has not been exploited. Specifically, information from the distance from a sample to the hyperplane in OCSVM could be used as a severity level discriminator for a set of features also learned using a one-class training set. (ii) OCSVM hyperparameter optimization has not been addressed beyond fault detection. Supervised approaches based on grid search are useless without an evaluation set (faulty data) because an evaluation metric for the one-class classifier without this set has not yet been defined. (iii) Unsupervised optimization approaches have been restricted to the RBF kernel in applications different to fault severity discrimination. This work introduces hyperparameter optimization approaches for γand νof OCSVM-based models in the context of 3D printer fault detection and fault severity discrimination. Our main contributions are summarized as follows: (i) a previously reported unsupervised optimization method for the RBF kernel hyperparameter is formally generalized and applied to other kernels. (ii) an unsupervised method is introduced to optimize the OCSVM hyperparameter; and (iii) a methodology to evaluate OCSVMs created with different kernels and optimization approaches in the context of fault severity discrimination is provided. The rest of the paper is organized as follows. Section 2intro- duces the formulation of OCSVM. Section 3presents the proposed OCSVM hyperparameter optimization, and the methodology is detailed in Section 4. In Section 5, the 3D printer test bed and experimental setup for comparisons are introduced. Results are shown in Section 6. We relate some conclusions in Section 7. 2. Preliminaries An OCSVM description as the core of a fault detection model is presented, with details on its optimization, the kernel trick, and some popular kernels. 2.1. OCSVM One-class support vector machines were first proposed [14] to address the one-class learning task using a similar framework to support vector machines [35] in binary classification. The fundamental idea is to build a hyperplane that optimally separates the available one-class dataset from the origin in a high-dimensional feature space. In this context, the optimum refers to the hyperplane that maximizes its margin (distance) to the origin, while keeping most of the one-class examples. In this way, a new example can be classified as positive (belonging to the available class) if it is over the hyperplane (far from the origin), and negative otherwise. 2 Formally, let x∈Rdbe an example to classify, and let φ: Rd→Rnbe a nonlinear mapping function from the input to the feature space. A decision function to classify xis defined by f(x)=sgn(w·φ(x)−ρ),(1) where sgn(z) is the sign function, which returns 1 when z≥ 0, and −1 otherwise, and wand ρare the parameters of the hyperplane in Rn, where w·y−ρ=0. Let {xi}m i=1be a one-class training set. Then the optimal hyperplane parameters are found by solving the following optimization problem: min w,ρ,ξ 1 2∥w∥2+1 νm m ∑ i=1 ξi−ρ s.t. w ·φ(xi)≥ρ−ξi,i=1, . . . , m ξi≥0, (2) where the slack variables ξ= [ξ1, . . . , ξm]allow for the presence of anomalous examples in the training set, and νlimits the fraction of training examples classified as anomalous. The method of Lagrange multipliers can be used to solve (2), enabling the transformation of the problem to the primal problem: min w,ρ,ξ max α,β L(w, ρ, ξ, α, β) s.t. αi≥0,i=1, . . . , m βi≥0,i=1, . . . , m, (3) where α= {α1, . . . , αm}and β= {β1, . . . , βm}are Lagrange multipliers, and L(·) is the Lagrangian, defined as L=1 2∥w∥2+1 νm m ∑ i=1 ξi−ρ− m ∑ i=1 αi[w·φ(xi)+ξi−ρ]− m ∑ i=1 βiξi.(4) Instead of solving (3), it is computationally convenient to reformulate the problem in its dual version, max α,β min w,ρ,ξ L(w, ρ, ξ, α, β) s.t. αi≥0,i=1, . . . , m βi≥0,i=1, . . . , m, (5) which has the same solution if the Karush–Kuhn–Tucker (KKT) conditions are met for the Lagrangian [36], which is the case with (4). Application of the KKT conditions, ∇w,ξ ,ρ L=0, to (5) results in the following equations: ∇wL=w− m ∑ i=1 αiφ(ξi)=0H⇒ w= m ∑ i=1 αiφ(ξi) (6) ∇ξL=1 νm−α−β=0H⇒ β=1 νm−α(7) ∇ρL= −1+ m ∑ i=1 αi=0H⇒ m ∑ i=1 αi=1.(8) Combining the constraints in (5) with (7) results in the restriction for α: 0≤αi≤1 νm.(9) Let L∗be the min Lfrom the substitution of (6),(7), and (8) in (4), L∗= −1 2 m ∑ i=1 m ∑ j=1 αiαjφ(xi)⊤φ(xj),(10) and combine that with (8) and (9), so the dual optimization problem becomes max αL∗(α) s.t.0≤αi≤1 νm,i=1, . . . , m m ∑ i=1 αi=1, (11) which is simpler than the primal optimization problem. 2.2. Kernel trick and decision function Let φ(xi)·φ(xj)=φ(xi)⊤φ(xj) be the inner product in the feature space of the projected xiand xj. The kernel trick defines a kernel function k(xi,xj)=φ(xi)·φ(xj) to avoid an explicit projection to the feature space. Considering k(·,·) in (11), the dual version is min α 1 2 m ∑ i=1 m ∑ j=1 αiαjk(xi,xj) s.t.0≤αi≤1 νm,i=1, . . . , m m ∑ i=1 αi=1, (12) where φ(·) is not required, and an algorithm [14] exists to solve the problem. The decision function in (1) can be reformulated to be independent from wand φ(·) by replacing (6) and applying the kernel trick to obtain f(x)=sgn( m ∑ i=1 αik(xi,x)−ρ),(13) where the elements xiwith αi>0 are called support vectors. It is easy to check that the hyperplane passes through the support vectors, or equivalently, ρ=∑ αi=0 αik(xi,xs),∀xs.(14) 2.3. Kernels A function can be used as a kernel only if it is positive-definite. Among the most used functions that fulfill this condition we can find: linear: k(xi,xj)=xi·xj(15) cosine: k(xi,xj)=xi·xj ∥xi∥ ∥xj∥(16) RBF: k(xi,xj)=e−γ∥xi−xj∥2(17) Laplacian: k(xi,xj)=e−γ∥xi−xj∥1(18) additive χ2:k(xi,xj)= − ∑(xi−xj)2⊘(xi+xj) (19) χ2:k(xi,xj)=e−γ∑(xi−xj)2⊘(xi+xj),(20) where γis the kernel hyperparameter. The use of additive χ2and χ2kernels is subject to xi≥0 and xj≥0. Hyperparameters νand γshould be determined based on the dataset properties before building the OCSVM model, although we usually rely on rule of thumb. 3 3. Hyperparameter optimization In the optimization of γand ν, we extend the approach of DFN (distances from training samples to their farthest neighbors and distances to their nearest neighbors) [37], which was initially proposed only for RBF kernels, for γoptimization in other kernels. A histogram-based approach is introduced to optimize ν. 3.1. γoptimization RBF kernel The DFN method [37] considers the optimal γvalue that maximizes the difference between the farthest element average distance and the nearest element average distance of the dataset projected by φ, i.e., the objective function is defined as g(γ)=1 m m ∑ i=1 max j∥φ(xi)−φ(xj)∥2−1 m m ∑ i=1 min j=i∥φ(xi)−φ(xj)∥2. (21) Thus g(γ) for the RBF kernel is: gRBF (γ)=2 m m ∑ i=1 e−γminj=i∥xi−xj∥2−e−γmaxj∥xi−xj∥2.(22) Expansion for Laplacian and χ2kernels In (21),∥φ(xi)−φ(xj)∥2=k(xi,xi)+k(xj,xj)−2k(xi,xj) from the expansion of the L2 norm and the kernel trick. Also, for Laplacian and χ2kernels, we have k(xi,xi)=k(xj,xj)=1, hence ∥φ(xi)−φ(xj)∥2=2−2k(xi,xj). Replacing the previous term in (21) and solving gives g(γ)=2 m m ∑ i=1 max j=ik(xi,xj)−2 m m ∑ i=1 min jk(xi,xj).(23) Finally, the g(γ) function for each γ-dependent kernel is obtained by replacing (18) and (20) for Laplacian and χ2kernels, respectively, i.e., g(γ)=2 m m ∑ i=1 e−γNear(xi)−2 m m ∑ i=1 e−γFar(xi),(24) where: Near(xi)=min j=i∥xi−xj∥1,for Laplacian (25) Far(xi)=max j∥xi−xj∥1,for Laplacian Near(xi)=min j=i∑(xi−xj)2⊘(xi+xj),for χ2(26) Far(xi)=max j∑(xi−xj)2⊘(xi+xj),for χ2. 3.2. νoptimization By definition, the hyperparameter νrepresents the percentage of training elements considered out of the class, and it is commonly set to a small value, assuming a few outliers in the oneclass set. The positive–negative distance approach [38] estimates this hyperparameter by iteratively incrementing it and choosing the value that maximizes the average in- to out-class distance. While this approach makes no assumptions about the data, it fails to estimate νwhen in- and out-class are not separable. Without loss of generality, we propose an approach to estimate νwithout assuming there are few out-class samples in the dataset. Consider that for a specific ν, the percentage of support vector candidates of the OCSVM model is 1−ν, where both correct and incorrect in-class examples are present. Because out-class examples tend to be grouped, the incorrect in-class examples are closest to the resulting hyperplane, and consequently, several become support vectors. Thus, although the correct out-class examples are successfully classified, they appear in the feature space through sampling the less probable values for the same inclass data distribution, i.e., they are in the tail of the in-class data distribution. However, proper configuration of νminimizes the number of out-class examples as support vectors. Then the correct outclass examples have a different distribution and are not in the distribution tail. Based on this observation, the optimization of ν is summarized in Alg. 1. Data: Dataset (D), kernel (k), inc,γ(if applicable) Result: Optimal ν ν=inc; while ν < 1do Create an OCSVM model for Dwith ν,γ, and k; Hin =Histogram of in-class hyperplane distances; Hout =Histogram of out-class hyperplane distances; if Hout is not tail of Hin then break; else ν=ν+inc; end end Return: νAlgorithm 1: νestimation. In Alg. 1,inc is the increment of νfor each iteration (usually, inc =0.05), and γis considered only if the kernel requires it. The ‘‘is not tail of’’ comparison verifies the continuity between the Hin and Hout histograms. For simplicity, this verification is performed by comparing the last bin of Hout ,Hlast out , with the first one of Hin, Hfirst in . The rule that Hout is not the tail of Hin if Hlast out >Hfirst in works for us, but more complex heuristics can also be evaluated. 4. Methodology OCSVM-based models are learned using only datasets from normal condition, and this is one of the most often used frameworks for fault detection. In this case, the binary decision function introduced in (13) allows us to determine whether a new example belongs to normal condition (+1) or faulty condition (−1). However, the information given by the position of the example in the feature space could also be exploited as a fault severity discriminator. It is useful to study the relationship between the perpendicular distance from an example to the hyperplane under different fault severity levels, and how different configurations of the OCSVM hyperparameters (optimized vs. not optimized) and kernels determine the model performance beyond fault detection. We introduce a methodology to compare OCSVM models in the context of fault detection and severity discrimination (see Fig. 1). The methodology can be summarized in the following steps. 1. Signal acquisition: We suppose that we can sample representatives (time-series) from a space Sof signals produced by a machine. These signals can be sampled from both known and unknown severity conditions. 2. BiGAN-based modeling: An unsupervised BiGAN-based feature extractor model [39] is built using only signals in normal condition in the training stage. In the testing stage, this model is used to extract features of signals in other severity conditions. 3. Min–max normalization: The normalization parameters are computed from normal signals and applied to normalize training and testing datasets. 4 Fig. 1. Methodology for OCSVM-based fault severity discrimination and model performance evaluation. 4. OCSVM-based modeling: For each hyperparameter configuration and kernel, an OCSVM model is built from normal signals. The resulting model is used to determine the distance of each new signal to the hyperplane in the feature space. 5. Histograms and performance comparison: The unavailability of samples from different severity levels in the training dataset does not allow to evaluate the model with the classical classification metrics. Therefore, a novel evaluation approach is proposed, i.e., by using samples of each severity condition from step 1, the overlapping area in the histogram of hyperplane distances is computed for each model. Each histogram estimates the probability distribution of the distance given a severity condition. Then the overlapping area represents the probability of confusing a pair of severity conditions. Therefore, the best models for fault severity discrimination have the smallest overlapping areas. 4.1. Signal acquisition Let C= {ci}K i=1be labels for the set of Kseverity levels. For every c∈C, let sc∈RTbe one time-series representative of the machinery dynamics under condition c, with Tsufficiently large. The goal is to build two sets, Dtrain and Dtest, to respectively train and test the models: •Dtrain = {sc1,i}m i=1is a set of mnormal condition (c1) signals, which are easily obtained from most types of machinery. This set will be used for BiGAN-based model building (step 2), and a transformed version will be used for the remaining training stages (steps 3 and 4). •Dtest = {sci,j}K,M i=1,j=1is a set of Msignals for every severity level, including c1. It will be used to test only the models created from Dtrain. Note that this set, with data from different severity conditions, is unknown in the training phase, and will be used only to evaluate the models. 4.2. BiGAN-based modeling Let X⊆Rdbe a feature space where the different time-series of Scan be represented. We aim to build an encoder function, Enc :S→X, that maps each element in Sinto a feature vector x∈X. One commonly used approach is to fix Enc by computing statistical condition indicators from different domains of scsuch as time, frequency, or time–frequency. Among the most common statistical indicators are RMS, standard deviation, kurtosis, and Shannon entropy [40]. The resulting vectors in each domain are concatenated to obtain a unique feature vector. The requirement of significant knowledge about the most informative statistical condition indicators regarding the severity level in the machinery is the main drawback of the previous approach, which makes it inapplicable to less-studied machinery such as 3D printers. Another approach is supervised deep learning of the Enc function. A signal set containing examples from every severity level is used to estimate an Enc function that can separate instances within each severity level. Although this is the optimal approach, it cannot be used because of the signal acquisition constraints in machinery. With this in mind, we propose to use a BiGAN to build Enc using only Dtrain [39]. This approach combines two 1DCNNs called Encoder (Enc) and Discriminator (Dis), and a 1DDNN called Generator (Gen). Generator is optimized to compete with Discriminator and try to fool it, i.e., Gen generates a synthetic signal that Dis may incorrectly recognize as real. Formally, Dis(s,x)={1,if s∈Dtrain and x=Enc(s) 0,otherwise.(27) Practically speaking, the previous models are parameterized by the θEnc,θGen, and θDis weight sets, which respectively define Enc, Gen, and Dis. The optimal parameter sets are subject to the solution of the following minimax optimization by gradient-based training algorithm: ˆ θDis,ˆ θGen,ˆ θEnc =arg min θGen,θEnc arg max θDis L(θDis, θGen, θEnc),(28) using the next loss function, L=∑ B log Dis(s,Enc(s))|s∼Dtrain +∑ B log (1 −Dis(G(x),x))|x∼PX, (29) where B≪mis the mini-batch size, and we take Bsamples from Dtrain in the first summation, and Bsamples from PX(a random noise distribution in X) in the second summation. Hence the optimal models are obtained. However, only Enc is necessary to realize the initial aim of this step. Finally, we obtain the datasets ˜ Dtrain =Enc(Dtrain) (30) ˜ Dtest =Enc(Dtest) (31) for subsequent steps. As we have provided an order in the original datasets, we define ˜ Di,j train =(Enc(si))jas the jth feature of the ith element. 4.3. Min–max normalization The sets ˜ Dtrain and ˜ Dtest ((30) and (31), respectively) are subject to unknown distributions based on the severity level of the original raw signals. Therefore, the scale range of extracted features is also unknown. 5 The scale of features plays a key role in the performance of various machine learning models, such as SVM [41]. Its application avoids the idea that features with a larger scale range are necessarily more important. Another advantage is to avoid numerical instability due to high values in the calculus. With this in mind, we normalize ˜ Dtrain and ˜ Dtest by a min–max approach, which conserves the data probability distribution and only rescales the features, specifically by linearly rescaling to the range [0,1]. Let ˜ xj=˜ D:,j train be the jth feature of the examples in the training dataset. The normalization parameters are computed as ˜ xj min =min(˜ xj) (32) ˜ xj max =max(˜ xj),(33) where the min and max functions respectively compute the minimum and maximum of the input vector. These parameters are used to rescale the features in ˜ Dtrain and ˜ Dtest according to the following equations: ¯ D:,j train =˜ D:,j train −˜ xj min ˜ xj max −˜ xj min (34) ¯ D:,j test =˜ D:,j test −˜ xj min ˜ xj max −˜ xj min ,(35) where ¯ Dtrain and ¯ Dtest are the normalized datasets for respectively training and testing the model. As the normalization parameters are obtained only for ˜ Dtrain, its range is guaranteed. However, the test dataset could have some features outside [0,1]that affect its use in OCSVM with additive χ2and χ2kernels, as previously stated. Then, for configurations with these kernels, elements from each dataset are squared after normalization, i.e., ¯ Dtrain and ¯ Dtest are replaced by ¯ D2 train and ¯ D2 test, respectively. 4.4. OCSVM-based modeling An OCSVM model is built for each evaluated kernel and hyperparameter configuration. We must decide whether they should be optimized or configured with default values. For γ-dependent kernels, we have four options (depending on whether γand ν take default values or are optimized), while only default values of νand optimized νare available for γ-independent models. When required, γoptimization is performed by the DFN method, which was extended to other kernels in Section 3.1. Then νcan be optimized by Alg. 1. Finally, OCSVM is obtained by solving the optimization problem (12) [14]. The procedure used to build each OCSVM model is summarized in Alg. 2. Data:¯ Dtrain, kernel, optimize γ?, optimize ν? Result: OCSVM model if does kernel require γ?then if optimize γ?then Optimize γ(Sec: 3.1); else Assign default value to γ; end end if optimize ν?then Optimize ν(Alg: 1); else Assign default value to ν; end Build OCSVM model; Return: OCSVM model; Algorithm 2: OCSVM model building procedure. After obtaining αand ρby respectively solving (12) and (14), we can apply OCSVM to new examples, and fault detection can be performed using (13). For fault severity discrimination, we propose that the perpendicular distances from a set of examples to the OCSVM hyperplane can discriminate between their fault severity levels. This distance is defined as dh(x)=∑ αi=0 αik(xi,x)−ρ, (36) where xis the position of the example for which is desired to calculate the distance from the hyperplane. Contrary to search severity patterns in a limited set of features obtained by BiGAN- based modeling, we characterize patterns with a single metric (L2norm) in a new, possibly infinite feature space, such as those obtained using RBF, Laplacian, or χ2kernels. 4.5. Histograms and performance evaluation To evaluate the fault severity discrimination performance of different model configurations, for each example in ¯ Dtest, we obtain the distances to the hyperplane, i.e., if xci,jis the jth feature vector under the ith severity level in ¯ Dtest, then the multiset distance is built from (DH)ci,j=dh(xci,j),∀xci,j∈¯ Dtest.(37) Let h∈Rbe a severity-dependent random variable defining the distance to the hyperplane (positive over the hyperplane, negative under it). Let p(h|c=ci),1≤i≤K, be the hyperplane distance distribution family conditioned in fault severity. We propose the severity pairwise error spe as the mixed area between the distributions of hunder the ciand cjseverity levels, i.e., spei,j=∫∞ −∞ min(p(h|ci),p(h|cj)) dh,i= j.(38) The p(h|c) distributions are analytically unknown, but an estimated severity pairwise error, ˆ spe, can be proposed as follows. With a stratified set of examples per severity level, the multiset DH can be represented by a matrix of dimension K×M K, where rows and columns respectively represent severity levels and evaluated test examples. Furthermore, since each example xin the dataset is a transformation applied to a randomly sampled time series s, the ith row of DH is the result of sampling the p(h|c=ci) distribution. Consequently, the estimated severity pairwise error can be defined by ˆ spei,j={∑Bins bin=1min(Histci(hbin),Histcj(hbin)),if i= j 0,otherwise ,(39) where Histci(hbin) is the bin-th value of the computed histogram of the ith row of DH, and the estimated severity pairwise error is aK×Ktriangular matrix whose (i,j) element shows the common area of two histograms representing a pair of fault severity levels. In the same way, the estimated error per severity level (slei), representing the performance of the model at each severity level, and average severity error (ase), as an overall performance metric, are respectively computed as slei=1 K−1∑ j=i spei,j(40) ase =1 K K ∑ i=1 slei.(41) 6 Fig. 2. 3D printer experimental setup. 5. Experiments The testbed for fault severity discrimination in 3D printers and the experiments to compare OCSVM models are detailed. 5.1. Testbed Data acquisition was performed on a 3D printer test platform developed for this work (Fig. 2). It was composed of an SLDBL600-6 3D printer with a belt-driven mechanism in a delta kinematic configuration, with three stepper motors moving the belts to control its three degrees of freedom, and a joint bearing in the terminal points of each arm to obtain free rotational movement. Magnetic field signals were selected as the information source for the severity discrimination task. They were acquired by a WIT MEMS BWT901 sensor with 14−bit resolution, sensitivity of 0.6 uT, and sampling frequency of 100 samples/s. The selected fault case was synchronous belt degradation. This type of fault appears due to an extended working period. It consists of the belt stretching due to looseness in the rigidity of the belt material. Thus additional vibrations begin to occur when the machine experiences high-velocity changes in areas with sharp corners. This fault reduces the quality of the final product. To identify the severity level of this fault is relevant to estimate the degradation of product quality and determine whether it is acceptable. To simulate degradation, the effective tension of one synchronous belt was decreased by 0 mm (optimal tension), 1 mm, 2 mm, and 3 mm, corresponding to severity levels c1,c2,c3, and c4, respectively. For these severity levels, three magnetic field signals were acquired for 324 s (32 400 samples at 100 samples/s). Each represented 20 circular patterns of 75 mm radius traveled by the printer head. Fig. 3 shows an extract of the signals for the four severity levels where distinguishable patterns cannot be observed for classification of different conditions. 5.2. Experimental setup The number of signals originally captured rendered us unable to tune the OCSVM parameters due to the curse of dimensionality. To address this issue, the signals were divided into sub-signals of 1620 samples, each containing one circular pattern. Therefore, with a sliding window of 10 samples, three groups of 3078 sub-signals were obtained. A three-fold cross-validation strategy was configured with these groups to minimize the risk of bias in the results. The comparison method was repeatedly applied with the training and testing sets Dtrain and Dtest respectively containing two and one of the previous groups. As stated before, only signals in normal condition were added to Dtrain. Let Tand lbe the input signal length and referred layer, respectively, in the 1DCNNs [42]. Then the number of layers (NL) and number of output channels in the lth layer (Kl) were set according to the equations NL = ⌊log2T⌋(42) Kl=2l,∀l=1, . . . , NL −1,(43) resulting in 10 layer networks. As required for adversarial training, one output was set for Dis. In the same way, 100 outputs were assigned to the Enc model, i.e., d=100. The 1DDNN architecture (Gen model) also has 10 layers to represent the inverse function on Enc. For Gen, the number of output channels of each l-layer (KGl) was KGl=2NL−l,∀l=1, . . . , NL.(44) The experiments were to compare OCSVM models with optimized and default hyperparameters, as well as various kernels. For the experiments with default values, γ=1/d=0.01, and ν=0.1, considering a fixed 10% of anomaly samples in the training set [43]. The γoptimization was performed by the DFN method extended in this work. The positive–negative distance approach and our proposal were compared regarding the ν optimization. 6. Results and analysis To analyze the characteristics of the input data for OCSVM models, Fig. 4 presents 2D projections of the BiGAN-based feature set with t-SNE and PCA techniques. It shows similar data distributions in both projections: two adjacent groups for c2, two adjacent groups for c3, and one group for c1adjacent to one c3 group, although a difference is noticeable between the closeness of c4with c1shown in PCA compared with t-SNE. The closeness of c3 and c4to c1highlights the complexity of the detection task with OCSVM models. In addition, multiple groups within the same severity condition, such as from c2and c3, represent a multivariable multimodal distribution, which greatly increases the difficulty of severity discrimination by assigning different mean distances to the same severity condition. In Fig. 4(b), it is appreciated that BiGAN-based feature extraction does not guarantee a separating distance between the groups according to the fault severity degree. Thus, the consequence is a non-monotonic dh function regarding to the severity degree, as is shown in Figs. 5 and 6. The comparisons between kernels, and of methods to optimize γand ν, are shown in Tables 1,2,3, and 4, where the column ‘‘optimize ν’’ shows the method used to optimize ν, and can take three values: 1. distances: optimization using positive–negative distance method; 2. histograms: optimization using the proposal shown in Section 3.2; 3. none: a default value of ν=0.1. 7 Fig. 3. Magnetic signals of healthy condition and fault in belts. Fig. 4. 2D projections of 100-dimensional 3D printer dataset. Examples in c1,c2,c3, and c4severity conditions are drawn respectively in violet, red, green, and blue. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.) 8 Table 1 Performance of the models for fault detection and fault severity discrimination using nonparametric kernels (independent of γ ). Kernel Optimize ν ν FNR (%) FPR (%) ase (%) Additive χ2Distances 0.95 87.20 0.00 0.56 Additive χ2Histograms 0.15 0.36 6.39 0.75 Additive χ2None 0.10 0.06 21.96 0.83 Cosine Distances 0.95 87.98 0.00 0.50 Cosine Histograms 0.20 0.71 2.86 0.69 Cosine None 0.10 0.13 27.30 0.86 Linear Distances 0.95 94.61 0.24 28.10 Linear Histograms 0.40 27.31 35.12 29.71 Linear None 0.10 3.41 42.65 32.35 The column ‘‘optimize γ’’ indicates whether γis optimized by the extended DFN method shown in Section 3.1. The FNR and FPR columns list the false negative (type II error) and false positive (type I error) rates, respectively, as evaluation metrics in the fault detection task performed by the OCSVM model. FPR is the ratio between the number of faulty samples classified as normal ones and the total number of faulty samples, and FNR is the ratio between the number of normal samples classified as faulty ones and the total number of normal samples. Finally, the proposed average severity error, ase, is presented as a performance metric of the OCSVM model in the severity discrimination task. 6.1. Nonparametric kernels Table 1 presents the fault detection (FNR and FPR) and fault severity discrimination (ase) performance for the nonparametric kernels. FNR ranks additive χ2, cosine, and linear kernel with a default value of νin the first, second, and third place, respectively. This ranking corresponds to the lower value for ν. Using FPR results in the same ranking order but using the positive–negative distance optimization method obtains the higher νvalue. However, the counterpart (FPR or FNR) was substantially increased in both cases without reaching an equilibrium. From these results, we confirm that these models are useless for the detection task. In addition, the models obtained with the histogram-based optimized νimprove both the FPR and FNR. This effect is noticeable for cosine and additive χ2kernels, with better performance for the first. Fig. 5 compares the distance distributions for different severity conditions using nonparametric kernels with the proposed histogram-based method. The overlapping of c1with distances lower than 0 produces an increment of FNR, and c2-c4distances bigger than 0 produce an increment of FPR. 6.2. Parametric kernels Models using the RBF kernel are compared in Table 2. As with nonparametric kernels, FNR is reduced with the lower fixed ν, and FPR is reduced using the positive–negative method. Both metrics decrease (i.e., performance improves) when the histogram-based method is applied. In addition, the optimization of γproduces a slight increase in FNR (slightly worse detection performance) and a consistent decrease of FPR (big performance improvement). In this sense, a large difference is appreciated between the fixed γ (0.01) and the optimal γ(0.292793). Something similar can be observed for fixed νand optimal ν. The optimization of γalso improves a model’s performance in the severity discrimination task, i.e., a reduced ase can be observed for each of the νoptimization methods. Summarizing the results in detection and severity discrimination, the evident winner is the model with optimized γand νoptimized with the histogram-based method. Fig. 5. Distance distribution of the best nonparametric kernels for each severity condition. Similar results are shown for the Laplacian and χ2kernels in Tables 3 and 4, respectively. The general results in detection and severity discrimination using χ2are slightly worse than those with RBF. The poorest performance is obtained with Laplacian comparing the parametric kernels. However, this highlights the improvement obtained with γoptimization, showing that the 9