SSMSPC: self-supervised multivariate statistical in-process control in discrete manufacturing processes

Biegel, Tobias; Helm, Patrick; Jourdan, Nicolas; Metternich, Joachim Article — Published Version SSMSPC: self-supervised multivariate statistical in-process control in discrete manufacturing processes Journal of Intelligent Manufacturing Provided in Cooperation with: Springer Nature Suggested Citation: Biegel, Tobias; Helm, Patrick; Jourdan, Nicolas; Metternich, Joachim (2023) : SSMSPC: self-supervised multivariate statistical in-process control in discrete manufacturing processes, Journal of Intelligent Manufacturing, ISSN 1572-8145, Springer US, New York, NY, Vol. 35, Iss. 6, pp. 2671-2698, https://doi.org/10.1007/s10845-023-02156-7 This Version is available at: https://hdl.handle.net/10419/317757 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/4.0/ Journal of Intelligent Manufacturing (2024) 35:2671–2698 https://doi.org/10.1007/s10845-023-02156-7 SSMSPC: self-supervised multivariate statistical in-process control in discrete manufacturing processes Tobias Biegel1·Patrick Helm1·Nicolas Jourdan1·Joachim Metternich1 Received: 10 January 2023 / Accepted: 27 May 2023 / Published online: 29 June 2023 © The Author(s) 2023 Abstract Self-supervised learning has demonstrated state-of-the-art performance on various anomaly detection tasks. Learning effective representations by solving a supervised pretext task with pseudo-labels generated from unlabeled data provides a promising concept for industrial downstream tasks such as process monitoring. In this paper, we present SSMSPC a novel approach for multivariate statistical in-process control (MSPC) based on self-supervised learning. Our motivation for SSMSPC is to leverage the potential of unsupervised representation learning by incorporating self-supervised learning into the general statistical process control (SPC) framework to develop a holistic approach for the detection and localization of anomalous process behavior in discrete manufacturing processes. We propose a pretext task called Location + Transformation prediction, where the objective is to classify both, the type and the location of a randomly applied augmentation on a given time series input. In the downstream task, we follow the one-class classification setting and apply the Hotelling’s T2statistic on the learned representations. We further propose an extension to the control chart view that combines metadata with the learned representations to visualize the anomalous time steps in the process data which supports a machine operator in the root cause analysis. We evaluate the effectiveness of SSMSPC with two real-world CNC-milling datasets and show that it outperforms state-of-the-art anomaly detection approaches, achieving 100% and 99.6% AUROC, respectively. Lastly, we deploy SSMSPC at a CNC-milling machine to demonstrate its practical applicability when used as a process monitoring tool in a running process. Keywords Self-supervised learning ·MSPC ·Anomaly Detection ·Discrete Manufacturing Introduction SPC is a well-known concept to monitor the condition of a process over time with the objective of detecting any anomalous process behavior that affects the performance of a process (Ferrer, 2007; Kourti & MacGregor, 1996; Zhang BTobias Biegel [email protected] Patrick Helm [email protected] Nicolas Jourdan [email protected] Joachim Metternich [email protected] 1Institute of Production Management, Technology and Machine Tools, Technical University of Darmstadt, Otto-Berndt-Straße 2, 64287 Darmstadt, Hesse, Germany et al., 2015). Within the discrete manufacturing domain, the practical application of SPC is typically based on univariate measurements of predefined quality characteristics of manufactured parts that are sampled in equidistant time intervals from the process (Montgomery, 2009). Literature agrees that this univariate post-process SPC-scheme is outdated, since it ignores the large amount of available process data in today’s data-rich manufacturing environments (Ferrer, 2014; Kourti & MacGregor, 1996; MacGregor, 1997; Woodall, 2017). Thus, in recent years, researchers in discrete manufacturing encouraged to shift this paradigm and transcend from univariate post-process SPC to MSPC by using sensor data collected from the machining process that are analyzed with machine learning (ML) methods to evaluate the process condition (Biegel et al., 2022a,2022b; Li et al., 2020;Qiu& Xie, 2021). Figure 1visualizes this paradigm-shift. The general SPC framework consists of two distinct phases. Phase I represents the offline monitoring phase, where the 123 2672 Journal of Intelligent Manufacturing (2024) 35:2671–2698 := System boundary Process Product Sampling Measuring quality characteristics control charting Post-process Process Multivariate sensor signals ML-based signal processing In-process control charting sample T2 upper control limit Diameter Shewart chart Hotelling’s T2chart sample ˆµ upper control limit lower control limit s1 s2 sm t ... x Diameter ( a ) Univariate post-process SPC ( b ) MSPC Fig. 1 Paradigm-shift in SPC. aUnivariate post-process SPC: Univariate measurements of predefined quality characteristics, i.e., quality data of manufactured parts are sampled in equidistant time intervals from the process to evaluate the process condition. bMSPC: Multivariate sensor signals, i.e., process data, are sampled continuously from the process and are analyzed with ML-based methods to evaluate the process condition control limits of the process are determined, based on historical normal process condition data. Phase II is the actual process monitoring phase, where new samples are drawn from the process and compared to the control limits to assess the process condition (Grasso et al., 2015;Qiu& Xie, 2021; Woodall, 2000; Woodall et al., 2004; Woodall & Montgomery, 1999). From an anomaly detection perspective, the SPC framework essentially describes a semi-supervised anomaly detection problem (Grasso et al., 2015; Wu et al., 2021; Xie & Peihua, 2022). Semi-supervised anomaly detection assumes that the training data incorporate only normal samples. The task is to model the normal behavior of the data and flag samples that strongly deviate from this state as anomalies (Chandola et al., 2009; Gu et al., 2019; Kumagai et al., 2019; Ruff et al., 2020; Shen et al., 2021; Tax & Duin, 2004; Ye et al., 2021). Due to this analogy, we treat MSPC in general and SSMSPC in particular as a semi-supervised anomaly detection problem. It is worth noting that some researchers refer to the semisupervised anomaly detection problem as unsupervised anomaly detection, see, e.g., Bergmann et al. (2019), Dehaene et al. (2020), and Zhang et al. (2019). However, unsupervised anomaly detection typically refers to a problem setting in which most but not all data in the training set are assumed to belong to the normal class (Bergman & Hoshen, 2020;Dai & Chen, 2022; Goyal et al., 2020; Pang et al., 2021). Research in anomaly detection is extensive, with many papers being published in recent years that focus on both shallow learning and deep learning approaches, see, e.g., Chalapathy and Chawla (2019), Chandola et al. (2009), Gupta et al. (2014), and Pang et al. (2021) for excellent reviews. Ruff et al. (2021), developed a unifying view for shallow and deep anomaly detection approaches in which they identify four main categories to which these methods can be assigned: (1) one-class classification, (2) probabilistic models, (3) reconstruction models and (4) distance-based methods. Recently, anomaly detection methods that rely on selfsupervised learning have shown outstanding performance on various benchmark tasks. Self-supervised learning is a form of unsupervised learning which aims to learn effective representations for real-world downstream tasks from unlabeled data by solving a supervised pretext task with automatically generated pseudo-labels, e.g., solving jigsaw puzzles or predicting image rotations (Doersch et al., 2015; Gidaris et al., 2018;Jing&Tian,2020; Noroozi & Favaro, 2016; Noroozi et al., 2018). Anomaly detection methods based on selfsupervised learning derive their anomaly score either directly from the pretext task or by using the learned representations 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2673 in the downstream anomaly detection task (Qiu et al., 2021; Sohn et al., 2021). Thus, defining suitable pretext tasks is a vital component in self-supervised learning approaches (Li et al., 2021). In this paper, we present SSMSPC, a novel approach for MSPC based on self-supervised learning to detect and localize anomalous process behavior in discrete manufacturing processes. We propose a pretext task that we refer to as Location + Transformation prediction. Given a time series input that has been augmented by one of kpredefined augmentation functions in one of pequally sized windows, the objective in this pretext task is to classify both, the augmentation and the corresponding window in a multi-task fashion. In the downstream task, we follow the conventional one-class classification setting and compute the Hotelling’s T2statistic as the anomaly score, based on the learned representations of the pretext task. The control limits are fitted with Kernel Density Estimation (KDE). In addition to that, we propose an extension to the traditional control chart view that combines metadata with the learned representations to (1) segment the process data into the individual process steps and (2) highlight the anomalous time steps, which supports a machine operator in the root cause analysis. To summarize, the contribution of this paper is threefold: •We propose SSMSPC, a novel approach for MSPC based on self-supervised learning to detect and localize anomalous process behavior in discrete manufacturing processes. •We present a pretext task called Location + Transformation prediction for learning effective representations, where the objective is to classify both, the type and the location of the augmentation based on a given randomly augmented time series input. •We introduce an extension to the conventional control chart view to facilitate the identification of the root cause by segmenting a raw time series signal into the individual process steps using metadata and highlighting the anomalous components. The remainder of this paper is structured as follows: “Related work” section presents the related work with respect to recent developments in self-supervised anomaly detection and applications of in-process monitoring in continuous processes as well as discrete manufacturing processes. “Problem statement” section provides a comprehensive description of the general problem statement that we consider for the application of SSMSPC. In “SSMSPC” section, we introduce the individual components of SSMSPC. This includes a detailed explanation of the applied framework, the proposed pretext task, the subsequent downstream task and the control chart extension. “Experiments” section presents the experiments based on two real-world CNC-milling datasets. We compare SSMSPC with state-of-the-art anomaly detection baselines and conduct a comprehensive ablation study. Our contribution ends with the conclusion and an outlook for future research. Related work Self-supervised anomaly detection The amount of research related to self-supervised anomaly detection has grown rapidly over recent years. Golan and El-Yaniv (2018) presented Geometric Transformations (GeoTrans) for anomaly detection in images. The authors designed a pretext task, where a multiclass neural classifier is trained to discriminate between geometric transformations that have been applied to normal images. The detection of anomalous images is accomplished by evaluating the softmax activations of the model when transformed images are used as an input. In a paper by Hendrycks et al. (2019), the authors combine rotation prediction (Gidaris et al., 2018) with geometric transformation prediction in their pretext task and are able to outperform a purely supervised approach based on outlier exposure (Hendrycks et al., 2019) to detect anomalies in images. Bergman and Hoshen (2020) present GOAD, a self-supervised anomaly detection approach for general data, i.e., images, tabular data etc. They extend the class of transformation functions in the pretext task to include random affine transformations to generalize to non-image data. Tack et al. (2020) propose contrasting shifted instances (CSI), which is based on the conventional contrastive learning scheme for learning visual representations. They introduce a new training method, in which a given sample is contrasted with distributionally-shifted augmentations of itself as well as other instances. By incorporating this into a new detection score, they achieve strong performance on stateof-the-art image anomaly detection tasks. Shen et al. (2020) propose THOC, a temporal hierarchical one-class network for time series anomaly detection. The authors use a dilated recurrent neural network with skip connections, and apply multiple hyperspheres obtained from a hierarchical clustering process to develop their one-class classification objective. They incorporate self-supervision by using a pretext task for multi-step-ahead prediction. Qiu et al. (2021) propose neural transformation learning for anomaly detection (NeuTraL AD) for data types beyondimages. Their approach consists of a fixed set of learnable transformations and an encoder, that are both trained jointly on a deterministic contrastive loss (DCL) that is also used to score new samples at test time. Sohn et al. (2021) present a two-stage framework for deep one-class classification. They learn self-supervised representations from one-class data and then build a conventional one-class classifier on top of the learned representations. 123 2674 Journal of Intelligent Manufacturing (2024) 35:2671–2698 The authors present a thorough analysis of different selfsupervised representation learning algorithms under their proposed framework. Li et al. (2021) introduce CutPaste, a simple augmentation strategy that cuts an image patch and pastes it at a random location. Their pretext task involves detecting whether an image has been augmented with CutPaste. They follow the two-stage framework proposed by Sohn et al. (2021). Fu and Xue (2022) introduce MAD, a selfsupervised learning task for time series anomaly detection. The objective of their pretext task is to predict the values of randomly masked samples of a time series input. In a paper by Shenkar et al. (2022), the authors present a novel contrastive learning approach for anomaly detection in tabular data. Given a data sample with masked features, the proposed learning approach is based on the assumption that the remaining features can be used to identify the masked ones. Wang et al. (2023) propose COCA, a negative-sample-free contrastive one-class anomaly detection method for time series data. The authors apply jittering and scaling augmentations to expand the number of training samples. They consider the representation in the latent space and the reconstructed representation of a Seq2Seq model as positive pairs. In-process monitoring applications Continuous processes Process monitoring based on MSPC and ML has seen wide application in industrial processes. Most of these applications originated in the context of continuous processes, such as chemical, petrochemical or polymer processes. The most basic approaches rely on Hotelling’s T2and Squared Prediction Error (SPE) monitoring statistics that are computed based on Principal Component Analysis (PCA) or Partial Least Squares (PLS) (Ge & Song, 2013;Qin,2003,2012; Wang et al., 2018). Throughout the years, many shallow learning approaches have been presented and extensively reviewed, see, e.g., Yin et al. (2014), Alauddin et al. (2018), and Qin and Chiang (2019). Recently, research with respect to deep learning applications has been very active, with numerous articles being published every year (Yu & Zhang, 2023). Yu et al. (2021) use a convolutional long short-term memory autoencoder (CLSTM-AE) for process monitoring and compute Hotelling’s T2and SPE monitoring statistics in the latent space and residual space, respectively. They validate their approach using two industrial benchmark processes, namely the Tenessee-Eastman process (TEP) and the continuous stirred tank reactor (CSTR) and find it to outperform conventional approaches such as PCA and LSTM-AE. In another paper by Yu and Zhang (2020), the authors propose a manifold regularized stacked AE (MRSAE) that relies on Hotelling’s T2and SPE as monitoring statistics. Their suggested approach outperforms other deep learning approaches on the TEP and the Fed-Batch fermentation penicillin process (FBFP) benchmark. In a paper by Cheng et al. (2019), the authors present a novel monitoring approach based on a variational recurrent AE (VRAE). They use the negative variational scores as the monitoring statistic and fit the control limit with KDE. The approach is validated using the TEP. Kong and Yan (2020) introduce the inner productbased stacked AE (IPSAE), which adds the inner product between the outputs of the neurons to the loss function for regularization purposes. They compute monitoring statistics in the feature space and the residual space and fit the control limit via KDE. For evaluation purposes, the TEP process is used and compared to PCA and stacked AE. In Tang et al. (2020), the authors combine Gaussian mixture models with a variational AE to monitor nonlinear processes with multiple operating modes. They construct latent variable and reconstruction variable indices as monitoring statistics. The authors validate their approach on the TEP and a hot strip mill process. Zhang et al. (2021) introduce a hybrid deep learning model based on a 1D-CNN and a stacked denoising AE. They demonstrate the effectiveness of their approach using the TEP, FBFP and a real-world industrial process for conveyor belts. Li et al. (2022) present a slow feature analysis-aided AE (SFA-AE) which leverages the extracted high-level features by the AE to learn deep slow variation patterns. With these patterns, they compute monitoring statistics based on Hotelling’s T2and SPE. In addition, they incorporate a self-attention mechanism to identify the anomalous process variables in a contribution plot. Liu et al. (2022) introduce a novel stacked multimanifold AE (S-MMAE) to predict and monitor key quality variables in industrial processes. The authors validate their approach using a real-world hydrocracking process. In Ai et al. (2023), the authors present KD-SCL, a novel industrial process monitoring framework based on knowledge distillation and contrastive learning. They rely on memory queue-based negative sample augmentation and hard negative sampling mechanisms to support the selection of negative samples for contrastive learning. The approach is validated using data from a lead-zinc flotation plant. Lu et al. (2023) introduce a cascaded bagging-PCA and CNN classification network. They define a self-supervised pretext task by trying to discriminate between the reconstructions of bagged and conventional PCA. The approach is validated using the TEP.Lietal.(2023) propose a self-supervised learning framework based on multisource heterogeneous contrastive learning. They employ a two-stage framework, in which the self-supervised feature learning phase is followed by a supervised fine-tuning step. The authors show the effectiveness of their approach using data collected from a heavy-plate production process. 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2675 Discrete manufacturing processes Regarding process monitoring in the context of discrete manufacturing processes, most of the existing research focuses on shallow or deep anomaly detection approaches that rely on some kind of reconstruction-based method, such as the AE. Biegel et al. (2022b) propose a novel approach that uses text data from machine operators to efficiently label the normal process condition data retrieved from a realworld CNC-milling process. Based on these data, they fit a simple PCA-based model and monitor the process with Hotelling’s T2and Squared Prediction Error (SPE) control charts. In another paper by Biegel et al. (2022a), the authors investigate the application of deep AE-based monitoring approaches and experiment with the reconstruction error and latent representation of the input data to compute Hotelling’s T2and SPE monitoring statistics. They use a realworld sheet metal forming process for their evaluation. In Lindemann et al. (2019), the authors present two data-driven self-learning approaches relying on k-means and LSTM-AE that are used to detect anomalies within a massive forming process. Lindemann et al. (2020) introduce a novel approach for anomaly detection in discrete manufacturing processes based on LSTM-AE that is evaluated on a multi-step forging process. Proteau et al. (2020) use a variational AE to monitor the condition of a CNC-milling process from the aerospace industry. Hahn and Mechefske (2021) present a disentangled variational AE with a temporal CNN to monitor tool wear in a self-supervised manner. They validate their approach using data from a CNC-milling process of small ball-valves. The authors treat the general AE scheme as a self-supervised learning objective. Ahmad et al. (2020) develop a hybrid model based on deep learning and SPC to monitor manufacturing processes in the presence of image or video data. They apply a fast region-based CNN to derive statistical features that are then plotted in an exponentially weighted moving average (EWMA) control chart. However, the authors validate their approach using only a simulated video. Lorenti et al. (2022) present CUAD-Mo, an approach for anomaly detection in machine operations that is based on Isolation Forest (IF) (Liu et al., 2008). The approach is validated in a CNC-milling process. Oshida et al. (2023)propose a stacked LSTM encoder-decoder model for anomaly detection. Their approach is evaluated on a real-world turning process for Inconel 718 to detect tool wear based on acoustic emission signals. In Sun et al. (2023), the authors introduce an AE-based semi-supervised anomaly detection method for cutting tools in machining processes. They validate the proposed method on an experimental and a public cutting tool wear dataset. The related work presented here demonstrates that the incorporation of self-supervised anomaly detection methods in process monitoring schemes for both continuous processes and discrete manufacturing processes is still rare and has not yet gained momentum. However, we expect that this line of research is going to grow rapidly in the near future. Problem statement In this paper, we address the semi-supervised anomaly detection problem in the context of discrete manufacturing processes. Specifically, we consider the problem of detecting and localizing anomalous process behavior using an ML model trained exclusively on data that correspond to a normal process condition. The reason for restricting ourselves to this problem setting is twofold: First, as pointed out in “Introduction” section, the assumptions made in the general SPC framework correspond to a semi-supervised anomaly detection problem. Second, especially in discrete manufacturing, anomalies typically account for rare data instances while normal data are easier to obtain and generally represent the majority of available data (Chalapathy & Chawla, 2019; Pang et al., 2021). When we speak of a discrete manufacturing process, we assume a machining process in which, (1) high-frequency process data, e.g., vibrations, and cutting forces are recorded throughout the processing cycle of a part in the form of a multivariate time series and (2) the process data are associated with the final quality characteristics of the produced part. Here, the second condition is of crucial importance to ensure that the monitoring scheme is in alignment with the economic objective of the process, which is to produce parts that meet the quality specifications. Thus, if an anomalous process condition is signaled, it should correspond to an increased likelihood of having produced a part that is out of specification. With the preceding explanations, we can formalize the problem statement as follows: Let D⊂Rn×mbe the set of all possible process data of a machining process, where ncorresponds to the number of time steps in a processing cycle and mrefers to the number of features, i.e., sensors used to record the process data. Each element x∈Dthus represents a manufactured part in the form of a multivariate time series. Let further X⊂Drepresent the set of all normal process condition data. Our overall objective is to provide a machine operator with the information whether the process data xfor a produced part correspond to a normal process condition. To achieve this, we wish to learn a mapping h:D→R, where larger values of h(x)indicate an increased probability that x/∈X, i.e., x is likely to represent an anomalous process condition. Based on h, we require a threshold mapping bucl :R→{0,1}such that 123 2676 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Fig. 2 Problem statement. We address the problem of anomaly detection and localization in the context of discrete manufacturing processes. aAn ML model is trained on lnormal process data {x1,...,xl}⊂X, with xi∈Rn×mto learn the mappings hand bucl. This corresponds to phase I in the SPC framework. bThe trained model is used to monitor new samples from the process. The anomaly scores h(x)are plotted in a control chart. If an anomalous process condition has been detected, i.e., (bucl ◦h)(x)=1, the model should locate the anomalous process condition in the time series to support a machine operator in the root cause analysis. This corresponds to phase II in the SPC framework (bucl ◦h)(x)=1,if h(x)>ucl 0,if h(x)≤ucl ,(1) where ucl is the corresponding threshold, i.e., the upper control limit. Thus, process samples exceeding the ucl are flagged as anomalies. Once an anomalous process condition has been detected, the ML model shall support a machine operator in the root cause analysis by locating the anomalous process condition in the time series. See Fig. 2for a visualization of the problem statement considered in this work. Note that SSMSPC can be applied to any discrete manufacturing process that is in accordance with this problem statement. SSMSPC In this section, we present SSMSPC. “Framework” section highlights the key components of the framework that constitutes our approach. “Pretext task” section dives into the details of the proposed self-supervised pretext task. “Downstream task” section explains how the learned representations of the pretext task are used to build a oneclass classifier based on Hotelling’s T2statistic (Hotelling, 1947) in the corresponding downstream task. Finally, in “Control chart extension” section, we demonstrate how the results of our approach can be visually interpreted by a machine operator to help identify the root cause for an anomalous process condition by highlighting the respective anomalous sections in the raw time series signal. Framework We follow the two-stage framework introduced by Sohn et al. (2021). The rationale for relying on this two-stage framework is twofold. First, self-supervised learning is inherently a two-stage process, where the first stage corresponds to solving the pretext task and the second stage embodies the downstream task (Jing & Tian, 2020; Noroozi et al., 2018). This framework thus represents a natural choice for selfsupervised anomaly detection methods. Second, the selected framework has demonstrated its effectiveness over end-toend approaches in previous works, see, e.g., Li et al. (2021). Figure 3illustrates the aforementioned framework. In the first stage, self-supervised learning is used to learn meaningful representations from normal process condition data with the help of a predefined pretext task. An encoder network fis used to transform the input data xto a latent representation f(x), that are then fed through a prediction head g, which is usually represented by a simple multi-layer perceptron (MLP) with a softmax output for the respective 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2677 Fig. 3 Two-stage framework based on Sohn et al. (2021). aIn the first stage, the model has to solve a predefined pretext task to learn effective representations. bThe second stage embodies the conventional one-class classification setting based on learned representations of the pretext task Fig. 4 Pretext task proposed in SSMSPC. aNormal process condition data are augmented in a randomly selected window, with a set of kpredefined augmentations T1,...,Tk. This increases the dataset size by a factor of k.bCWT is applied to the augmented dataset to compute the scalogram representation. cThe scalograms are used as an input for a LeNet-type encoder network that has two prediction heads attached to it. Given a scalogram as input, the task is to correctly predict the type and the location of the applied augmentation classification task. The model is trained end-to-end on the respective pretext task using backpropagation. The second stage corresponds to the downstream task where a one-class classifier is constructed on top of the learned representations of the pretext task. For this stage, it is common practice to discard the prediction head g, and only use the pretrained encoder network fas a feature extractor, since it has been shown that the learned representations of the layer right before gprovide better representations (Ermolov et al., 2021; Chen et al., 2020). It is worth mentioning that with respect to the general SPC framework both, pretext task and downstream task are incorporated in the offline monitoring phase, i.e., phase I, as they are used to fit the control limits for subsequent phase II monitoring. Pretext task In this work, we propose a pretext task that we refer to as Location + Transformation prediction, which is specifically designed to be applied in the setting described in “Problem statement” section. The core intuition behind our pretext task is (1) to learn which augmentation was applied to a given time series input, and (2) to learn where the augmentation occurred in the time series by predicting the respective augmentation window. By training a model with this pretext task, we show that the learned representations are highly effective for monitoring discrete manufacturing processes. Figure 4provides a visualization of the individual components that constitute the proposed Location + Transformation prediction pretext task. 123 2678 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Assuming a training set Xtrain ⊂Xwe apply a set T:= {T1,...,Tk|Ti:Rn×m→Rn×m}of kdifferent augmentation functions. Each Tiaugments the entire set of available normal process condition data Xtrain. For every x∈Xtrain the augmentation is applied in a randomly chosen window Wjfor all msensors using a set of ppredefined windows {W1,...,Wp}. The augmented time series data Ti(x) are consequently transformed using the continuous wavelet transformation (CWT) to retrieve the respective scalogram representation CWT ◦Ti(x)∈Rs×n×m, where srepresents the number of different scales of the chosen wavelet. Hence, by transforming the normal process condition data in this way, we retrieve a representation that can be interpreted as an m-channel image. Following this routine, the original dataset size is increased by a factor of k. Algorithm 1summarizes the general augmentation procedure. With the completion of the data preparation, the augmented dataset is used as an input for the subsequent Location + Transformation prediction task. The scalograms are fed into a simple LeNet-type encoder network f(Lecun et al., 1998) on which two prediction heads gtrans and gloc are attached for Transformation prediction and Location prediction, respectively. The prediction heads are represented by two equivalent MLPs differing only in the size of the softmax output layer. Here, gtrans outputs a softmax layer of size kto predict the applied augmentation, whereas gloc outputs a softmax layer of size p+1 to predict the window in which the augmentation occurred. Note that the output of gloc is p+1 in order to account for the case when no augmentation was applied, and thus no window was chosen. Algorithm 1 General augmentation procedure Require: T1,...,Tk,Xtrain,p 1: Xaugmented ←{} 2: compute window bounds for pwindows 3: for i∈{1,...,k}do 4: for x ∈Xtrain do 5: Randomly choose a window Wj 6: Augment xin window Wjwith Ti 7: Compute scalogram CWT ◦Ti(x) 8: Append CWT ◦Ti(x)to Xaugmented 9: end for 10: end for 11: Shuffle Xaugmented 12: return Xaugmented In the following, we provide further details for the individual components of the proposed pretext task. Data augmentations We propose a total of k=4 augmentations that have been proven most useful in learning good representations during our experiments. Fig. 5 Example of the CutPaste augmentation with p=5 windows. a A random-sized segment is cut from a randomly chosen window. bThe segment is pasted to another randomly chosen window Identity In alignment with other recently proposed research in the field of self-supervised anomaly detection such as Tack et al. (2020) and Golan and El-Yaniv (2018) the first augmentation that we propose is simply the identity, i.e., T1(x)=x. The reason for including the identity function is that this translates into including the normal process condition data Xtrain as one of the four classes. Since the model is exposed to the original data Xtrain during training, it needs to be able to distinguish the normal process condition from artificial anomalous data to perform well on the pretext task. CutPaste For the second augmentation class, we draw inspiration from the work of Li et al. (2021) and as such refer to it as CutPaste. However, in contrast to the authors that proposed CutPaste to augment images, we transfer the idea of CutPaste to the time series representation and adapt it for the application in Location + Transformation prediction. Figure 5provides an exemplary illustration of how a time series is augmented using CutPaste. First, a random cutting window Wcis chosen from {W1,...,Wp}. Within the bounds of Wc, two points are randomly selected, marking the start and end point of the cutting segment. Next, a pasting window Wjis randomly selected from the entire set of windows. The cutting segment is then pasted to a random location within Wj. Note that the described augmentation is done for each of the mfeatures, i.e., sensors of x. Algorithm 2provides further details on how the CutPaste augmentation works. MeanShift The next augmentation that we found useful to learn effective representations is referred to as MeanShift. For each sensor in x, this simple augmentation strategy shifts a pre-selected time series segment by the mean of the respective time series. Figure 6shows how the proposed MeanShift augmentation works. The first step consists of selecting a random window Wjfrom the set of windows {W1,...,Wp}. Similar to CutPaste, two points are then randomly chosen 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2685 apply a total of three convolutional modules using 8×(7×7) filters for the first module, 16 ×(7×7)filters for the second module and 32 ×(7×7)for the third. The convolutional modules are followed by two dense modules consisting of a dense layer, a batch normalization layer and leaky ReLU activation, respectively. Here, the first dense layer uses 32 neurons whereas the second uses 16. The prediction heads are mostly equivalent in their structure, i.e., each head consists of two dense layers with 16 and 8 neurons, respectively, followed by a batch normalization layer and leaky ReLU activation. The heads differ only in the last dense and softmax layer, where gloc uses 6 neurons and gtrans uses 4 neurons. Table 3provides further insights into the different layers used. Model training We train the model on the proposed Location + Transformation pretext task for 20 epochs with a batch size of 8 and a cosine learning rate schedule (Loshchilov & Hutter, 2017) with an initial learning rate of 10−4.Weuse a leakiness of β=10−1for the leaky ReLU activations. With respect to optimization, we apply the Adam optimizer (Kingma & Ba, 2015) and use weight decay with λ=10−3 for regularization. Downstream task: Bosch CNC-milling dataset With respect to the subsequent downstream task, we first apply CWT on each split of DSusing the same configuration as in the pretext task. We then resize the scalograms to 128×512 and apply min-max scaling based on the train split. Doing this, we obtain the required dataset for the downstream task DDownstream S⊂R128×512×3, where |DDownstream S|=143. Recall from “Downstream task” section, that we use the train split containing only normal process condition data for fitting the Hotelling’s T2one-class classifier. Model architecture As shown in Fig. 8, we discard the prediction heads gloc and gtrans and keep only the pretrained encoder fto extract the learned representations using the output of the last dense layer. We then attach a Hotelling’s T2layer to it. Table 4shows the model architecture for the downstream task. Model training For the one-class classification training phase, we pass the train data of DDownstream Sthrough the frozen encoder network fand fit the required parameters for Hotelling’s T2, i.e., ˆ µ∈R16×1and ˆ −1∈R16×16 based on the learned representations. We then use these parameters to compute the Hotelling’s T2values for the train data. Following this, we fit the upper control limit as described in “Downstream task” section using a significance value of α=0.01, to obtain a low false alarm rate. Thus, we take the 0.99 quantile of ˆ φT2as the upper control limit. Recall that this step completes the offline monitoring phase, i.e., phase I with respect to the conventional SPC framework. All hyperparameters, including those for pretext training, are selected Table 3 Model architecture used for the pretext task Layer type Output shape Parameters fInput Layer (None, 128, 512, 3) 0 Conv2D (None, 122, 506, 8) 1184 BatchNorm (None, 122, 506, 8) 32 LeakyReLU (None, 122, 506, 8) 0 MaxPool2D (None, 61, 253, 8) 0 Conv2D (None, 55, 247, 16) 6288 BatchNorm (None, 55, 247, 16) 64 LeakyReLU (None, 55, 247, 16) 0 MaxPool2D (None, 27, 123, 16) 0 Conv2D (None, 21, 117, 32) 25120 BatchNorm (None, 21, 117, 32) 128 LeakyReLU (None, 21, 117, 32) 0 MaxPool2D (None, 10, 58, 32) 0 Flatten (None, 18560) 0 Dense (None, 32) 593952 BatchNorm (None, 32) 128 LeakyReLU (None, 32) 0 Dense (None, 16) 528 BatchNorm (None, 16) 64 LeakyReLU (None, 16) 0 gloc Dense (None, 16) 272 BatchNorm (None, 16) 64 LeakyReLU (None, 16) 0 Dense (None, 8) 136 BatchNorm (None, 8) 32 LeakyReLU (None, 8) 0 Dense (None, 6) 54 Softmax (None, 6) 0 gtrans Dense (None, 16) 272 BatchNorm (None, 16) 64 LeakyReLU (None, 16) 0 Dense (None, 8) 136 BatchNorm (None, 8) 32 LeakyReLU (None, 8) 0 Dense (None, 4) 45 Softmax (None, 4) 0 using grid search based on the performance on the validation set in the downstream task. The final performance is reported based on the hold out test set. Baselines: Bosch CNC-milling dataset We compare the performance of SSMSPC with state-ofthe-art baselines from the realms of shallow, deep and self-supervised anomaly detection. We evaluate the baselines on (1) DDownstream S, i.e., CWT features and (2) statistical 123 2686 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Table 4 Model architecture used for the downstream task Layer type Output shape Parameters fInput Layer (None, 128, 512, 3) 0 Conv2D (None, 122, 506, 8) 1184 BatchNorm (None, 122, 506, 8) 32 LeakyReLU (None, 122, 506, 8) 0 MaxPool2D (None, 61, 253, 8) 0 Conv2D (None, 55, 247, 16) 6288 BatchNorm (None, 55, 247, 16) 64 LeakyReLU (None, 55, 247, 16) 0 MaxPool2D (None, 27, 123, 16) 0 Conv2D (None, 21, 117, 32) 25120 BatchNorm (None, 21, 117, 32) 128 LeakyReLU (None, 21, 117, 32) 0 MaxPool2D (None, 10, 58, 32) 0 Flatten (None, 18560) 0 Dense (None, 32) 593952 BatchNorm (None, 32) 128 LeakyReLU (None, 32) 0 Dense (None, 16) 528 T2Hotelling’s T2(None, 1,1) 0 features. For the statistical features, we compute root-mean square,peak-to-peak,interquartile range,mean,standard deviation,kurtosis,skewness and median absolute deviation for each sensor of the raw time series data in DS. Following this, we scale the data to have zero mean and unit variance based on the train set. Thus, for the evaluation with statistical features we obtain the following dataset Dstat S⊂R24, where |Dstat S|=143. Shallow baselines For the shallow baselines, we choose OneClass Support Vector Machine (OC-SVM) (Schölkopf et al., 1999), IF, Principal Component Analysis (PCA) and Kernel-PCA, as they are regularly used for the comparison of anomaly detection methods. Since the samples in DDownstream Srepresent image data, we first train a LeNet-type AE (LeNet-AE) on the train data and then use the encoder part to transform the scalograms into the latent representation. The shallow baselines are then applied on top of the extracted features. This scheme is commonly used in literature, see, e.g., Golan and El-Yaniv (2018). The encoder of the LeNet-AE has the same architecture as the LeNet-type encoder network that is used in SSMSPC. The decoder is constructed symmetrically to the encoder. We just replace convolutions with deconvolutions and max-pooling with upsampling. Regarding the evaluation on Dstat S, we apply the shallow baselines directly on top of the statistical features. We use the scikit-learn implementation for the presented baselines. Hyperparameters are selected using grid search Table 5 Hyperparameters for shallow baselines Hyperparameter DDownstream SDstat S OC-SVM kernel RBF RBF γ0.25 0.25 ν0.005 0.005 IF n_estimators 10 10 max_samples 32 80 PCA n_components 12 23 Kernel-PCA n_components 15 13 kernel cosine cosine based on the performance on the validation set. Table 5provides an overview of the selected hyperparameters. Deep baselines As representatives for deep anomaly detection baselines, we choose DAGMM (Zong et al., 2018), DeepSVDD (Ruff et al., 2018), PatchCore (Roth et al., 2022), and the LeNet-AE presented in “Baselines: Bosch CNC-milling dataset” section. DAGMM utilizes a deep AE and combines both, the latent representation and the reconstruction error of the input which is then fed to a Gaussian mixture model. The mixture model and the AE are trained in an end-to-end fashion. DAGMM is designed to work on tabular data and as such we evaluate it only on Dstat S. DeepSVDD uses the encoder of a pretrained AE as weight initialization to extract features from image data. The encoder is then trained end-to-end on the one-class classification objective to map the data into a minimum-volume hypersphere (Ruff et al., 2018). PatchCore is a recently published method that has achieved state-of-the-art results on the MVTec AD (Bergmann et al., 2019) industrial image anomaly detection benchmark. The intriguing property of PatchCore is that it does not comprise an actual training phase in the sense of gradient-based optimization, but it leverages the learned representations of mid-level feature representations of a ResNet (He et al., 2015) architecture trained on ImageNet (Krizhevsky et al., 2012). Specifically, given an image dataset for training that contains only normal data, PatchCore extracts locally aware patch features and stores them in a memory bank, by passing the training data through the network. With this memory bank in place, PatchCore computes the anomaly score of a given test image based on the maximum distance score between test patch-features in its patch collection and each corresponding nearest neighbor in the memory bank (Roth et al., 2022). For PatchCore, DAGMM and DeepSVDD, we use the official implementation that has been made publicly available by the authors with their respective publication. Table 6presents the selected hyperparameters. 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2687 Table 6 Hyperparameters for deep baselines Hyperparameter DDownstream SDstat S DAGMM batch_size – 8 learning_rate – 10−4 epochs – 1000 DeepSVDD batch_size 8 – learning_rate 10−3– epochs 50 – weight_decay 10−6– ν0.005 – PatchCore batch_size 2 – patch_size 3 – LeNet-AE batch_size 8 – learning_rate 10−4– epochs 100 – Self-supervised baselines Considering the self-supervised anomaly detection baselines, we choose RotNet (Gidaris et al., 2018), NeuTraL AD and GeoTrans. Regarding RotNet, we follow two different approaches. First, we treat it as a feature extractor and use the learned representations from the rotation prediction pretext task as an input for PCA, Kernel-PCA, OC-SVM and IF. Second, we follow the scheme presented by Hendrycks et al. (2019) and use the softmax probabilities of RotNet to determine if a sample is normal or anomalous. As with the deep baselines, we use the publicly available implementation of the respective self-supervised baseline to conduct the experiments. Table 7 displays the selected hyperparameters. Results: Bosch CNC-milling dataset In Table 8, we present the results of our experiments. We report mean and standard deviation of the Area under the Receiver Operating Characteristic (AUROC) curve over 10 different random seeds. The Receiver Operating Characteristic (ROC) curve of a classifier is a two-dimensional graph in which the recall is plotted over the false alarm rate for every possible decision threshold in the test set (Fawcett, 2006). The area under this curve represents the AUROC which is a threshold-independent evaluation measure that (1) provides an overall assessment of the capabilities of a classifier and (2) allows to compare the peformance of different classifiers (Ding et al., 2014). The AUROC value ranges from 0 to 1 (or 0% to 100%) and can be defined as the following mean computed across all anomalous and normal data pairs in the Table 7 Hyperparameters for self-supervised baselines Hyperparameter DDownstream S RotNet batch_size 8 learning_rate 10−1 epochs 100 weight_decay 5 ·10−4 input_size 128 ×128 + PCA n_components 5 +Kernel-PCA n_components 3 kernel cosine +OC-SVM kernel RBF γ0.5 ν0.005 +IF n_estimators 10 max_samples 64 NeuTraL AD batch_size 8 learning_rate 10−3 epochs 20 transformations 15 GeoTrans batch_size 8 learning_rate 10−3 epochs 3 input_size 128 ×128 test set (Aggarwal, 2017; Campos et al., 2016): AUROC := mean a∈A,n∈N⎧ ⎪ ⎨ ⎪ ⎩ 1,if h(a)>h(n) 1 2,if h(a)=h(n) 0,if h(a)<h(n) (13) where Nand Arepresent the respective sets of normal and anomalous data in the test set and h(·)corresponds to the anomaly score produced by the classifier. Intuitively, the AUROC can be interpreted as the probability that a classifier ranks a randomly chosen anomalous sample higher than a randomly chosen normal sample (Bradley, 1997; Fawcett, 2006). Due to the aforementioned properties, the AUROC has established itself as the standard measure for comparing the performance of anomaly detection models (Abati et al., 2019; Carrara et al., 2020; Cheng & Vasconcelos, 2021;Hu et al., 2020; Ruff et al., 2021; Shen et al., 2021; Wu et al., 2021). Focusing on the results of our experiment, we see that SSMSPC achieves a perfect score on the given task outperforming all baselines except LeNet-AE and PatchCore which have on-par performance. We can see that the shallow baselines perform well on the statistical features, especially OC-SVM and PCA with 91.5% and 89.7% AUROC, respectively. Perhaps surprisingly, the performance of the shallow baselines 123 2688 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Table 8 Results on the Bosch CNC-milling dataset Category Feature Method AUROC Shallow Statistical OC-SVM 91.5±4.4 PCA 89.7±4.7 Kernel-PCA 88.0±4.8 IF 79.1±8.7 CWT LeNet + PCA 75.9±10.3 LeNet + Kernel-PCA 78.3±8.7 LeNet + OC-SVM 81.3±9.8 LeNet + IF 54.3±10.8 Deep Statistical DAGMM 92.8±5.5 CWT DeepSVDD 99.9±0.2 LeNet-AE 100.0±0.0 PatchCore 100.0±0.0 Self-supervised CWT NeuTraL AD 97.9±3.6 GeoTrans 94.5±6.7 RotNet 56.0±9.9 RotNet + PCA 94.6±5.6 RotNet + Kernel-PCA 84.7±12.0 RotNet + OC-SVM 98.4±1.6 RotNet + IF 97.6±3.1 SSMSPC (ours) 100.0±0.0 We report the mean and standard deviation for AUROC over 10 different random seeds. State-of-the-art baselines are investigated using both, statistical features and CWT features. The best results are given in bold on the CWT features is much lower, with IF being only barely able to exceed the random guessing AUROC baseline of 50%. In terms of the deep baselines, we see an overall strong performance. Deep SVDD, LeNet-AE and PatchCore are very competitive and perform similar to SSMSPC with 99.9%, 100.0% and 100.0% AUROC, respectively. DAGMM shows the best performance among the baselines that are evaluated on statistical features with 92.8% AUROC. Regarding the self-supervised anomaly detection methods, we can see that RotNet, when evaluated with the scheme of Hendrycks et al. (2019), is performing close to the random guessing AUROC baseline. However, when used as a feature extractor we see that the performance strongly increases with RotNet + OC-SVM achieving 98.4% AUROC. GeoTrans and NeuTraL AD show a performance in the range of RotNet when used as a feature extractor, with 94.5% and 97.9% AUROC, respectively. Even though the general evaluation in this paper is based on AUROC, it is nonetheless necessary for practical applications to select a fixed threshold. Thus, we include thresholddependent performance metrics to provide further insights into the quality of the threshold selection scheme in SSMSPC. Table 9shows the performance of SSMSPC with respect to mean and standard deviation of Precision, Recall and F1-Score using KDE to fit the control limits of the process as described in “Downstream task” section. We present Table 9 Results for Precision, Recall and F1-score of SSMSPC for varying levels of αon the Bosch CNC-milling dataset αPrecision Recall F1-score 0.050 65.6±6.6 100.0±0.079.0±5.1 0.025 72.3±6.2 100.0±0.083.8±4.0 0.010 84.9±4.0 100.0±0.091.7±2.2 We report mean and standard deviation over 10 different random seeds the performance for varying choices of α. With α=0.01, SSMSPC achieves an average F1-Score of 91.7%, while identifying all the anomalies present in the hold out test set across all random seeds. Localizing anomalies Figure 11 presents qualitative results of the proposed control chart extension in SSMSPC based on anomalous samples taken from the validation set. We highlight the anomalous process modes identified by our model and compare it to the true labels as given in the dataset. As is shown, SSMSPC is able to identify the anomalous process modes and thus provides an effective support for a machine operator to find the root cause of the anomalous process condition. 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2689 Fig. 11 Qualitative visualization of the control chart extension proposed in SSMSPC. We present samples from the validation set and show in the tables below the true labels for each mode of the process (anomaly = 1, normal = 0). The highlighted regions are the anomalous components of the respective process steps that have been identified by the model Table 10 Effects of adding augmentations on the performance of the downstream task Method Augmentation AUROC SSMSPC base 100.0±0.0 +PointOutlier 98.7±1.6 +Trend 98.3±2.7 +Gaussian 96.0±3.9 We report mean and standard deviation for AUROC over 10 different random seeds. The best results are given in bold Ablation study We perform an ablation study to provide further insights into the individual components of SSMSPC. First, we analyze the effects of adding and removing augmentations for the pretext task. Second, we compare the performance of the proposed Location + Transformation prediction pretext task with Location prediction and Transformation prediction as individual pretext tasks. Third, we vary the number of windows that is used for the augmentations in the pretext task to see how this affects performance. Augmentations We define three additional augmentations, which we refer to as PointOutlier,Trend and Gaussian in order to assess whether additional augmentations have a positive effect on the performance in the downstream task. For the PointOutlier augmentation, we select two random points in a window and add the maximum value of the whole time series to these two points. Regarding the Trend augmentation, we emulate a growing shift in the standard deviation within a selected window. Gaussian selects a time series segment within a window and adds random numbers sampled from a Gaussian distribution with specified mean Table 11 Effects of removing augmentations on the performance in the downstream task Method Augmentation AUROC SSMSPC base 100.0±0.0 w/o MeanShift 99.5±0.7 w/o Identity 99.3±1.4 w/o CutPaste 99.1±1.4 w/o MissingSignal 98.7±1.2 We report mean and standard deviation for AUROC over 10 different random seeds. The best results are given in bold and standard deviation. Table 10 displays the results. We observe that the additional augmentations lead to a decrease in performance. The strongest decline can be seen with the Gaussian augmentation. In addition to adding new augmentations we also investigate the effects of removing the proposed augmentations one after another, to understand which of the augmentations is most important for the performance. The results are reported in Table 11. We observe that the MissingSignal augmentation is the most important component, leading to the strongest decline in performance upon omission, followed by CutPaste. The decline observed by removing MeanShift or Identity is less strong but nonetheless existing and thus justifies their inclusion into SSMSPC. Different pretext tasks To demonstrate the meaningfulness of the proposed Location + Transformation prediction pretext task and the corresponding loss function, we investigate the performance of Location prediction and Transformation prediction as individual pretext tasks. This is done by simply removing one of the prediction heads and then training SSMSPC on the remaining pretext task. All other components of SSMSPC are left unchanged. Table 12 shows the results. It 123 2690 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Table 12 Effects of using Transformation prediction and Location prediction as pretext tasks Method Pretext task AUROC SSMSPC Loc. + Trans. prediction 100.0±0.0 Transformation prediction 99.2±1.6 Location prediction 97.9±1.7 We report mean and standard deviation for AUROC over 10 different random seeds. The best results are given in bold Table 13 Effects of varying the number of windows used for generating the dataset for the pretext task Method #Windows AUROC SSMSPC 3 99.5±0.7 5100.0±0.0 798.9±1.5 We report mean and standard deviation for AUROC over 10 different random seeds. The best results are given in bold can be seen that the performance of SSMSPC decreases when trained solely on either Location prediction or Transformation prediction as pretext task. Hence, the representations learned by Location + Transformation prediction are more effective. Number of windows We investigate the effects of varying the number of windows on the performance of SSMSPC. Table 13 shows the results. We see that SSMSPC is fairly robust with respect to varying the number of windows, which is an important feature for practical purposes. CiP-DMD dataset The second experiment in this paper uses parts of the Center for industrial Productivity - Discrete Manufacturing Dataset (CiP-DMD), which is a novel benchmark dataset for discrete manufacturing processes that we are going to publish soon. The specific dataset that we consider originates from a CNC-milling process in which steel cylinder bottoms are produced for pneumatic cylinders. Since this dataset is not yet published, we provide a short introduction of the process and give some insights into the dataset structure. Process description The test bed used for the development of the dataset is a Deckel-Maho DMC-50H horizontal CNC-milling machine, see Fig. 12. The machining space of the DMC-50H is equipped with a rotating tower on which special fixtures are mounted that can hold several parts at once. Figure 13 provides a visualization for both sides of the rotating tower. Fig. 12 DMC-50H CNC-milling machine used as a test bed within the CiP-DMD dataset Fig. 13 aFront side and bback side of the rotating tower in the machining space of the DMC-50H test bed with special fixtures to hold raw material and semi-finished parts, respectively. The white circles indicate the positions where the parts are clamped At the beginning of a machining cycle, two distinct parts, i.e., a piece of raw material and a semi-finished part are clamped to the front and back side of the rotating tower, respectively. Upon completion of a machining cycle, the process outputs a semi-finished part and a finished part. Note that for the sake of simplicity regarding the creation of the dataset, we used only one part per tower side. Since the machine processes two distinct parts per cycle, we refer to the part at the front side of the rotating tower as part 1 (P1) and to the part at the back side as part 2 (P2). Figure 14 provides an illustration of the respective production stages for P1 and P2. Regarding the machining process itself, it is worth mentioning that the process for P1 consists of 9 distinct process modes whereas the process for P2 consists of 4. The total duration of one machining cycle amounts to roughly 5 minutes. More precisely, the process takes 199 seconds to complete P1 and another 113 seconds to complete P2. 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2691 Fig. 14 aP1 before and after processing. This corresponds to the conversion from raw material to a semi-finished part. bP2 before and after processing. This corresponds to the conversion from a semi-finished part to a finished part Fig. 15 Accelerometer used for data acquisition Dataset structure The DMC-50H test bed is equipped with a triaxial accelerometer having a sampling rate of 2.5 kHz that is mounted directly on the spindle. The sensor is connected to a data acquisition card that is plugged into an industrial PC which is located on top of the machine, see Fig. 15 for an illustration of the sensor positioning. In addition to the process data, we also recorded metadata from the machine control such as the NC lines of the machine program with a sampling rate of 5Hz. For the purpose of this experiment, we recorded a total of 776 machining cycles between November 2022 and March 2023. Specifically, we have 776 samples for P1 and 775 samples for P2. Note that the discrepancy in the total number of samples between P1 and P2 stems from the fact that we have one air cut for P2 which we did not include in the dataset. For P1, 736 samples correspond to normal processing cycles, whereas for P2 we have a total of 737 normal processing cycles. The remaining cycles correspond to anomalous process behavior. We incorporated two realistic process anomalies which need to be detected and located during the process. The first anomaly targets P1 and results from a piece of raw material that has been sawed off too short. As a consequence, the cutting tool does not remove any chips during the face milling process step. The result of this anomalous process is a semi-finished part with an unprocessed surface. The second anomaly targets P2 and corresponds to a falsely clamped Fig. 16 Anomalies for P1 and P2. aComparison of normal P1 (left) and anomalous P1 (right). bComparison of normal P2 (left) and anomalous P2 (right) Table 14 Overview of the data split size for DP1 and DP2 including the number of available normal and anomalous samples Split Normal Anomalous Total DP1 Train 465 −465 Validation 67 10 77 Test 204 30 234 DP2 Train 465 −465 Validation 68 9 77 Test 204 29 233 part. Due to this clamping error, the face milling step removes more chips from one side of the semi-finished part than from another. The finished cylinder bottom is thus crooked. Figure 16 illustrates the two anomalies. We refer to the dataset of P1 as DP1 and to the dataset of P2 as DP2. Given the processing time of 199 seconds for P1 and a sampling rate of 2.5 kHz, we have a total of 497500 samples for every axis of the sensor. Thus, DP1 ⊂R497500×3, with |DP1|=776. Conversely, for P2 with a processing time of 113 seconds, we have a total of 282500 samples per axis of the sensor, i.e., DP2 ⊂R282500×3, where |DP2|=775. We split the data using the same train, val and test split as we did in the first experiment, i.e., 60%, 10%, and 30%. Note that, as before, the training data comprise only normal process data. Table 14 shows the respective number of samples for DP1 and DP2. As in the previous experiment, we compare the performance of the selected baselines with statistical features and CWT features. The statistical features are computed according to the description in “Baselines: Bosch CNC-milling dataset” section with respect to each sensor in the raw time series data for DP1and DP2. Following this, we scale the data to have zero mean and unit variance based on the corresponding train split. Hence, we obtain Dstat P1,Dstat P2⊂R24. For the baselines that rely on CWT features, we follow the procedure presented in “Downstream task: Bosch CNC-milling dataset” section. Thus, for DP1and DP2,we compute the CWT using Morlet wavelets with s=128 scales on the raw time series data, resize the resulting scalograms 123 2692 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Table 15 Hyperparameters for shallow baselines Hyperparameter DDownstream P1Dstat P1 OC-SVM kernel RBF RBF γ2−30 2−30 ν0.01 0.005 IF n_estimators 300 200 max_samples 4 128 PCA n_components 3 6 Kernel-PCA n_components 9 23 kernel cosine cosine Hyperparameter DDownstream P2Dstat P2 OC-SVM kernel RBF RBF γ40.5 ν0.005 0.005 IF n_estimators 10 10 max_samples 256 64 PCA n_components 10 17 Kernel-PCA n_components 10 18 kernel cosine cosine to 128 ×512 and apply min-max scaling based on the train split. This leads to DDownstream P1,DDownstream P2⊂R128×512×3. Note that each of the preceding datasets contains a total of 776 samples for P1 and 775 samples for P2. Regarding the pretext datasets for SSMSPC, we use the setting as presented in “Pretext task: Bosch CNC-milling dataset” section, i.e., we augment the data using Identity, CutPaste,MeanShift and MissingSignal augmentations with a total of 5 windows. This is followed by CWT using Morlet wavelets with s=128 scales, resizing to 128×512 and subsequent min-max scaling based on the training data. Thus, we have DPretext P1,DPretext P2⊂R128×512×3, where |DPretext P1|= 1860 and |DPretext P2|=1860. Recall that we use the respective train split of DP1and DP2as the basis for the pretext dataset generation. Baselines: CiP-DMD dataset We use the baselines as presented in “Baselines: Bosch CNC-milling dataset” section. For the deep and some of the self-supervised baselines, we keep the hyperparameters from the first experiment, as we found them to provide the best performance in terms of the corresponding validation sets. The hyperparameters of the shallow and the RotNet self-supervised baselines are listed in Tables 15 and 16. Regarding the architecture of SSMSPC, we use the settings from Tables 3and 4, respectively. For the pretext tasks and the corresponding downstream tasks, we follow the procedure presented in “Pretext task” and “Downstream task” sections. In terms of the hyperparamTable 16 Hyperparameters for self-supervised baselines Hyperparameter DDownstream P1 RotNet +PCA n_components 5 +Kernel-PCA n_components 9 kernel cosine +OC-SVM kernel RBF γ2−30 ν0.005 +IF n_estimators 50 max_samples 64 Hyperparameter DDownstream P2 RotNet +PCA n_components 64 +Kernel-PCA n_components 10 kernel cosine +OC-SVM kernel RBF γ0.125 ν0.005 +IF n_estimators 50 max_samples 16 eters for SSMSPC, we increase the batch size to 32 and reduce the epochs to 10, leaving the remaining hyperparameters unchanged. Results: CiP-DMD dataset Table 17 shows the results for the second experiment. We follow the evaluation scheme presented in “Results: Bosch CNC-milling dataset” section, i.e., we evaluate the performance of each model with respect to AUROC. We included a “Total” column that contains the average performance of the corresponding baselines with respect to P1 and P2. Note that we use the average performance as the final evaluation measure, since it provides the best estimate for the overall monitoring performance. As we can see from the results, SSMSPC yields the highest AUROC score with respect to the overall performance and beats all other baselines on P1. In contrast to the first experiment, we make the surprising observation that the shallow baselines relying on statistical features demonstrate a very strong performance. Specifically, PCA outperforms state-of-the-art deep and self-supervised anomaly detection models such as, e.g., DeppSVDD, PatchCore and NeuTraL AD. When evaluated solely on P2, PCA even outperforms SSMSPC by a small margin. The shallow models relying on CWT features are worse, showing a drop in performance of at least 14.8% when compared to SSMSPC. 123 Journal of Intelligent Manufacturing (2024) 35:2671–2698 2693 Table 17 Results on the CiP-DMD dataset Category Feature Method AUROC Total P1 P2 Shallow Statistical OC-SVM 98.3±1.298.8±0.798.5 PCA 99.3±0.399.5±0.499.4 Kernel-PCA 98.7±0.798.4±1.298.5 IF 98.7±0.586.7±3.992.7 CWT LeNet + PCA 67.4±20.593.3±3.480.4 LeNet + Kernel-PCA 75.7±18.092.7±4.584.2 LeNet + OC-SVM 71.8±14.397.8±0.884.8 LeNet + IF 68.0±16.085.2±5.376.6 Deep Statistical DAGMM 99.4±0.395.3±3.097.4 CWT DeepSVDD 86.3±17.075.5±15.280.9 LeNet-AE 61.8±22.873.1±22.967.4 PatchCore 99.4±0.499.2±0.499.3 Self-supervised CWT NeuTraL AD 99.7±0.498.2±0.799.0 GeoTrans 97.6±1.659.5±18.478.6 RotNet 79.8±4.271.2±5.975.5 RotNet + PCA 99.5±0.497.7±1.498.6 RotNet + Kernel-PCA 97.9±1.393.4±2.995.7 RotNet + OC-SVM 97.9±1.697.8±0.897.9 RotNet + IF 98.2±1.181.6±7.589.9 SSMSPC (ours) 99.8±0.25 99.4±0.399.6 We report the mean and standard deviation for AUROC over 5 different random seeds for P1 and P2. State-ofthe-art baselines are investigated using both, statistical features and CWT features. The best results are given in bold In comparison to the first experiment, we see a strong deterioration for some of the deep baselines. Specifically, DeepSVDD and LeNet-AE show a weak performance. In fact, LeNet-AE yields the lowest overall score, being 32.3% worse than SSMSPC. Conversely, PatchCore and DAGMM show an overall strong performance. Regarding the self-supervised baselines, we see that NeuTraL AD shows the best results. The learned representations by RotNet when used as a feature extractor yield a strong performance similar to the first experiment. In Table 18,we display the Precision, Recall and F1-score of SSMSPC for varying values of αto demonstrate its effects on the monitoring performance. Summarizing, the results of the presented experiments provide some interesting insights. First, we can conclude that the learned representations of SSMSPC are indeed useful for solving tasks that are in alignment with the problem statement described in “Problem statement” section. Second, we can confirm that self-supervised learning is a very promising concept that helps to learn effective representations in situations where the amount of available data is scarce. Third, as we have seen in prior works such as Biegel et al. (2022a), it seems that in certain situations in discrete manufacturing the performance of conventional deep learning methods varies strongly and is often outperformed by basic shallow Table 18 Results for Precision, Recall and F1-score of SSMSPC for varying levels of αon the CiP-DMD dataset αPrecision Recall F1-score DP10.050 75.0±0.898.3±0.285.1±0.4 0.025 87.3±0.394.8±2.491.0±1.3 0.010 92.7±0.688.0±7.390.2±4.1 DP20.050 74.5±4.9 100.0±0.085.3±3.3 0.025 89.7±3.290.7±7.290.2±4.6 0.010 88.1±4.693.1±3.490.5±3.8 We report mean and standard deviation over 5 different random seeds learning approaches. Fourth, methods like PatchCore that do not encompass an actual training phase but leverage the learned representations from a pretraining phase with data from another domain show very promising performance for future applications in discrete manufacturing. Towards in-process control As a final demonstration of the practical applicability for real-world discrete manufacturing processes, we deployed SSMSPC at the DMC-50H machine that we used for the creation of the CiP-DMD dataset. Specifically, we incorporated 123 2694 Journal of Intelligent Manufacturing (2024) 35:2671–2698 Fig. 17 Web interface of the deployed SSMSPC system at the DMC50H test bed. aControl chart view which is used for anomaly detection. This view shows the control charts for P1 and P2 on a single page and is used by a machine operator to assess the process condition during the process. bExtended control chart view for P1 (top) and P2 (bottom). Once an anomalous process condition has been detected, the control chart extension can be used for anomaly localization purposes, to support a machine operator in the root-cause analysis SSMSPC into a web-based Graphical User Interface (GUI) that allows a machine operator to interact with the system, see Fig. 17. The web interface consists of two separate views. In the first view, two control charts are displayed for P1 and P2, respectively. These control charts are updated automatically, i.e., during the processing cycle of the machine. If the system detects an anomaly, the machine operator can access the control chart extension simply by clicking on the corresponding sample in the control chart. This constitutes the second view of the web interface. The control chart extension presents the process data of the current part. The time series is segmented into the corresponding processing steps using the NC lines that are recorded from the machine control and the anomalous time steps are highlighted. Note that the presented functionality is in alignment with the visualization of the problem statement in Fig. 2. The deployed system monitors the process data of every part, which is equivalent to a 100% check of the running process. As opposed to a conventional SPC scheme, in which manufactured parts are sampled in equidistant time intervals from the process, the presented system can be used as a trigger for measuring produced parts. Specifically, when the system detects an anomalous process condition, the machine operator can decide, whether it is necessary to measure the corresponding part, based on the extended control chart view. Conclusion In this paper we introduce SSMSPC, a novel approach for MSPC based on self-supervised learning to detect and localize anomalous process behavior in discrete manufacturing processes. We present a pretext task that we refer to as Location + Transformation prediction. Given a randomly augmented time series input, the objective in this pretext task is to train a model to classify both, the type and the location of the augmentation. In the subsequent downstream task, we apply Hotelling’s T2on top of the learned representations from the pretext task, following the one-class classification setting. We fit the control limits using KDE based on the computed T2values in the downstream task. In addition to the conventional control chart view, we propose a control chart extension that facilitates the identification of the root cause by segmenting a raw time series signal into the individual process steps using the NC lines of the machining program and highlighting the anomalous process components. We evaluate the performance of SSMSPC using two realworld CNC-milling datasets and compare it to state-ofthe-art baselines from the realms of shallow, deep and self-supervised anomaly detection. Our experiments show that SSMSPC learns effective representations achieving the highest score on the given tasks. In addition to that, we 123