Identifying repeating patterns in IEC 61499 systems using Feature-Based embeddings

Identifying repeating patterns in IEC 61499 systems using Feature-Based embeddings Markus Unterdechler, Antonio M. Guti´ errez, Lisa Sonnleithner, Rick Rabiser, Alois Zoitl CDL VaSiCS, LIT CPS Lab Johannes Kepler University Linz Linz, Austria {markus.unterdechler,antonio.gutierrez,lisa.sonnleithner,rick.rabiser,alois.zoitl}@jku.at Abstract— Cyber-Physical Production Systems (CPPSs) are highly variable systems of systems comprised of software and hardware interacting with each other and the environment. The increasing integration of technologies and devices has brought an unprecedented level of automation and customization. At the same time, it has also increased the efforts to maintain highly complex and heterogeneous systems. Although engineering practices support the reuse of common components to ease the development and maintenance of the systems in different projects, the identification of common components is still manually performed, which is a time-consuming, error-prone task. In this paper, a novel approach identifying repeating patterns in CPPSs based on artificial intelligence techniques is presented. This approach allows finding exact and similar components to support the CPPS design. Furthermore, it enables the maintenance of common components by reusing predefined types thereby reducing development effort. We implemented and evaluated our approach in an industry case study on developing CPPS control software with IEC 61499. Index Terms—cyber-physical systems, artificial intelligence, IEC 61499 I. INTRODUCTION AND MOTIVATION Cyber-Physical Production Systems (CPPSs), like metallurgical plants or manufacturing plants, are highly variable systems of systems that frequently evolve. They typically involve a large number of heterogeneous components (e.g., mechanical, electrical, mechatronics or software) that can be configured and combined in different ways. The heterogeneity results not only from hardware and software components but also from development processes, disciplines, methods, and tools. In contrast to typical software engineering life-cycle models, in control software development in CPPSs, different disciplines (e.g, mechanical, electrical, or software) are involved in different systems engineering steps. Decisions in one discipline can affect the others on a large scale both in the short and in the long term. While CPPSs are typically one-offs, the reuse of software components shows room for improvement [1]. The high variability in plant configurations is still mainly handled manually based on the know-how of experienced engineers, leading to high costs in project development [2]. A common clone-and-own approach with or without modification also presents challenges to software maintenance: they may require consistent application of the same or similar bug fixes or program changes to multiple copies. This problem is even worse for artifacts developed by engineers from different teams as they can do different designs for a similar component (e.g., different valves or cylinders) in the same project. The scale and complexity of the projects make it unpractical to manually detect all the instances of a similar component. A typical development lifecycle of a CPPS includes the analysis of reusable components to create a common workspace that can be used from project to project. This practice is depicted in Figure 1. Based on their domain knowledge and experience, companies identify a set of common artifacts that are developed and maintained centrally on a Core Platform. This platform includes Libraries for specific components and functionalities, Templates for pre-created project structures, and Solutions for well-known machines and plants. When a new project arrives, its requirements are analyzed to decide which components from the Core Platform to reuse. During the Project Generation, these components are cloned, extended, or adapted to the requirements of the project to obtain an initial set of assets. As control systems vary because of customers’ requirements or integration dependencies (thirdparty hardware and technologies), this Project Generation process has to accommodate the variability of the target control system. This can be an expensive task, which depends on the project teams’ expertise. Libraries Templates Solutions Project A Project Generation Commissioning Core Platform Project B Testing Engineering Commissioning Testing Engineering Project-specific development Round-trip Round-trip Fig. 1: Typical CPPS control software development process. This process helps to reduce maintenance and development costs. However, the identification of common components is typically a manual process based on the engineer’s experience making it tedious and error-prone. High engineering, testing, and commissioning efforts in the project phase mainly result from the need to manually adjust the generated control software to the specific machine or plant configuration. Potential deviations from the planning have to be minimized to ensure steep plant ramp up as well as high-quality products [3], [4]. And on the other hand, since there is no traceability between project components and common components, it is costly to make the round trip and consider improvements, changes, or additions from a specific project into the Core Platform based on a defined process; e.g. Change Control Board. The scale of the projects makes it difficult to detect identical components during the design phase and as far as we know there are no proposals to automatically detect variants of similar components for CPPSs. To overcome this difficulty, we consider techniques from artificial intelligence domain that have been successfully used to recognize patterns with large volumes of non-structured data [5]. Specifically, we consider embedding the structure from system components and using this embedding to classify the components according to their similarity. As several techniques and algorithms can be used for detecting and grouping components by similarity, we propose an architecture to apply different techniques to detect such similarities. We first implement such architecture for the processing of projects with the standard IEC 61499 [6]. This standard proposes an architecture for developing distributed control software with an event-driven execution pattern and in a platform-independent manner. Then we extract the features of the components in an IEC 61499 project to create embeddings and cluster these components based on their similarities. We apply this approach to an industry case study, pointing out similar components as result. This result provides useful information to improve the development practices, either by creating new common components or adapting existing components to increase the reuse of commonalities. The rest of the paper is structured as follows. Section II briefly describes the IEC 61499 standard and reviews existing related work to identify similarities, specifically in CPPSs. Section III describes the proposed architecture and its implementation for CPPSs defined with IEC 61499. In Section IV, we present the results of this proposal for an industry case study, and we discuss our findings. Finally, Section V presents the conclusions together with future research work. II. BACKGROUND A. IEC 61499 IEC 61499 is a domain-specific language for modeling distributed control software. It defines a hardware-independent application model that can be mapped to a system configuration and thus supports adapting the control hardware. It is based on the previous standard, IEC 61131. IEC 61131 is widely adopted by industrial automation and control applications. However, it is mainly used for centralized control systems and it hinders the creation of modular, distributed, Fig. 2: Composite FB with 5 FBs and 2 Events in IEC 61499. reusable, and flexible industrial automation applications [6], [7]. On the other hand, IEC 61499 was developed to bring these properties [6], [8], [9] in PLC-software. The smallest software component of IEC 61499 is a Function Block (FB). An IEC 61499 control application is a network of these FBs providing desired machine functionality. A FB communicate with other FBs with event or data connections. A FB can be implemented with different languages in IEC 61499, such as Execution Control Chart, ECC, to define a state machine based on the events, or Structured Text, ST. The FBs are instantiated from a common library with existing predefined FB types. The application itself is comprised of modular software components: basic FBs, composite FBs and subapplications. An example of composite FB is depicted in the Figure 2 1. Composite FBs and subapplications contain a FB network, i.e., one or multiples FBs which are interconnected. The interfaces of the internal FBs can be used for the interface of the composite FB/subapplication. FB in the FB network can also be composite FB/subapplications, creating a hierarchical structure. Any existing subapplication can be defined as a FB type to be shared as part of the common library, allowing it to be reused and maintained through composition. Eclipse 4diac IDE is an extensible, IEC 61499 compliant engineering environment for distributed control applications2. B. State of the art In this paper, we address the problem of identifying similar software components to increase their reusability and improve development and maintenance efforts. The identification of other problems during CPPS design time can also support the improvement of the CPPS design. A catalog of code smells in control software were proposed by Sonnleither et al. [10] to identify patterns that can indicate potential problems. The detection of clones is included in this catalog. In clone-and-own scenarios, textual software clone detection is used with certain metrics [11], [12] or by using data mining techniques to address the increasing time and resource complexity [13]. Ziadi et al. [14] showed how software abstraction 1Extracted from https://www.eclipse.org/4diac/en help.php?helppage= html/before4DIAC/iec61499.html 2https://www.eclipse.org/4diac/ through reverse engineering can help reduce induced noise resulting from different implementations of the same feature. Artificial Intelligence (AI) techniques have established themselves in many research fields to address difficult pattern recognition, learning, and other tasks under situations with large volumes of non-structured data [5]. There are already some research efforts to automate finding software clones and identifying similar code parts in textual programming languages. Specifically, there are some works addressing the problem of finding code clones with neural networks [15], [16], including a report from GitHub [17]. However, with visual programming languages like we have using 4diac IDE, we also have to deal with the extra dimension that comes from the graphical structure, which cannot be easily serialized. The existing approaches cannot simply be applied. But similar to textual programming languages, we also deal with different types of similarities. The similarity can be measured in different forms. We can have Token-by- Token similarity (exact clones) where the parts are compared one by one [18] or syntactical similarity where similarity is based on extracted syntax tree [19]. The next level of similarity are semantic similarities, where parts are looking different but provide the same functionality [20]. Especially semantic similarities are very hard to detect. There we can benefit from machine learning techniques [21]. A possible way to detect semantic similarities is to extract various features from the source code and use them for training a classifier [22]. Huge datasets are available for programming languages like Java that can be used for the training process. In IEC 61499 such datasets are not available, therefore such supervised learning methods cannot be used accordingly. Function block networks are graph-structured data. Analyzing graphs has become a popular area in the AI community these days as graph-structured data is the most natural and therefore the most widely available data structure. Graph Neural Networks (GNN) have achieved great success in the field of graph mining techniques [23]. It has been widely applied for various classification tasks or for learning node, edge or graph embeddings, but also for graph comparison and graph matching tasks [24]. This makes it a very essential and powerful tool, especially when dealing with large graphs. However, for graph-structured data the similarity can be calculated or estimated, such as by using graph distance measures. Exact distances like graph edit distance [25] or DeltaCon [26] are more computationally intensive than approximations like spectral distances, Larplacian distances, or feature-based distances [27]. One feature-based approach called NetSimile focuses on extracting local and egonet-based features and aggregating them into a feature vector [28]. It is a general approach that could be extended in almost any direction, like including domain-specific parameters (e.g., FB types, Edge- Types, FB-Complexity, . . . ). Feature-based approaches [29], [30] try to describe the graphical structure with a set of extracted metrics, which can then be used for further processing. However, a lot of information is lost through this extraction. Other approaches that process the graph in its structure can benefit from preserving the information [23], [31]. III. APPLYING ARTIFICIAL INTELLIGENCE TO IDENTIFY SIMILAR PATTERNS We envision that machine learning techniques can be used to automatically detect repeating patterns in CPPSs. To achieve that goal, we propose the architecture depicted in Figure 3. Basically, it consists of 4 phases (i) Candidate Selection, (ii) Similarity Embedding, (iii) Projection (optional) and (iv) Clustering. Given one CPPS or a set of CPPSs, first, we have to identify the candidates for the analysis in the Candidate Selection phase. Depending on the selection technique, we have to define the granularity of the extracted components and structure them as input for the analysis technique. For example, a complex system network can be recursively grouped to build a multilevel project hierarchy that suits the similarity analysis. Another approach would be to detect densely connected modules in the flat system using clustering methods. In this way, patterns could also be recognized across the actual project hierarchy. In the next step, the Similarity Embedding phase, we detect similarities in the selected candidate list. One-to-one processing of all candidate combinations cannot be implemented with complex comparison methods since the complexity grows exponentially with the number of candidates. We project selected information parts into a high-dimensional vector space that preserves similarities. In these embeddings, similarity is represented by the direction or distances of vectors. Proposed embedding approaches can be roughly divided into two general concepts, the Feature-Based and the Graph- Based concept. In the feature-based approach, various extracted information from the project parts is used to generate the embeddings. The considered features can capture absolute or structural metrics in an absolute or relative manner. Depending on what kind of similarity is to be detected, the importance of different extracted features can greatly vary. Either one has to manually define which features are of interest, or a neural network (NN) can be trained to create meaningful similarity embeddings. The autoencoder is a type of neural network that can learn efficent representations of unlabeled data [32]. There, an encoder part is trained to create lower dimensional embeddings, such that a decoder part can reconstruct the input data from them. Following that approach the encoder learns to preserve important information in the latent space. In the presence of a training dataset a NN could also be used to detect even more complex relationships like semantic similarities. In contrast to the feature-based approach, the graph-based approach is keeping the graphical structure while processing. A powerful way is to use machine learning methods like Graph Neural Networks (GNN). There a NN is trained to pass messages through the network to generate meaningful node embeddings. Followed by an Attention layer, we can then generate graph embeddings out of it. Embedding Similarity Embedding Feature-Based Graph-Based Feature Extraction GNN Attention Candidate Selection CPPS Projects Candidates NN Dimensionality Reduction Projection Clustering Similarity Clustering optional component alternative embedding implementation Legend: Fig. 3: Phases of Similarity Analysis: Selection, Embedding, Project and Clustering. Regardless of the approach implemented to create the embeddings, they must preserve similarity such that the distances between similar objects in the vector space are minimal. However, due to the so-called ”curse of dimensionality” [33], in high dimensional spaces, a loss of meaningful differentiation between similar and dissimilar objects is observed [34]. For this reason, an optional Projection phase can be used to map the high dimensional embeddings into lower space before clustering. Since dimensionality reduction potentially means loss of information, clever techniques have to be applied. A linear projection method called Principle Components Analysis (PCA) retain most of the useful information by spanning a new coordinate system on axes with the highest variance [35]. Other non-linear approaches like i.e. T-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation (UMAP) produce low dimensional projections by preserving local structure (t-SNE) or global and local structure (UMAP) [36]. Finally, similarity clusters should be detected in the Clustering phase. Clustering methods can be of different types. They can be categorized into hierarchical clustering, partitional clustering, density-based clustering, grid-based clustering and model-based clustering [37]. These methods vary in, i.e., the procedures used for measuring the similarity and the parameterization (i.e. thresholds, amount of clusters). As a result, we expect a list of similarity groups, where each group includes similar components. A. Feature-based extraction to find similarities in IEC 61499 As a proof of concept, we implemented a feature-based embedding with hierarchical clustering for IEC 61499 projects. As depicted in Figure 4, we considered IEC 61499 projects as input. As basic FBs are already instances of predefined common types, we consider FB networks (i.e. subapplications, composite FBs), as the analysis component. The FB network hierarchy is used to define the candidate selection method. FB networks can be untyped, i.e., unique instances just defined for a specific project, or typed FB networks, i.e., instances of a predefined common type. Both kinds are considered for the analysis. For the processing of the system, we export the IEC 61499 into a graph structure, where FB networks are flattened, FBs are the nodes and the data and event connections are the edges. In this export, we also need to consider which information (i.e. FB type, FB complexity) is required by the feature extraction to be included in the attributes of the nodes and edges. Next, we determine which features to consider in the embedding phase. Features can be of various kinds. The semantic of the similarity determines which kind to consider, for example, whether absolute metrics should be preferred over relative ones. In our case, we wanted to focus on implementationbased similarity. Therefore, we opted for a mixture of absolute and structural metrics combined with content-related features. The absolute metrics should bind the similar candidates in size. Content and structure-related metrics should then provide a measure of implementation-related similarity. We want to detect identical clones or project parts that are almost the same and share the same substructures. Parts that can be typed after small refactoring steps. We decided to extract the following features: •Function Blocks: The absolute amount of FBs contained in a FB network. •Connections: The absolute amount of Event and Data Connections. •FB Types: The normalized amount of FBs per type. •Vertex Degree: A collection of statistical indicators of the degree of neighborhoods (described below). The first two features define only one dimension each one. But the FB Types generates so many values as existing Type in the library (a typical library can include more than 40 Types). Also, the Vertex Degree feature include a tuple of statistical indicators about the FB neighborhood. We collect the degree of neighborhoods for each FB and then aggregate them into a feature vector using the median and the four moments of a distribution: mean, standard deviation, skewness, and kurtosis. In this way, the analysis focus on similarity in terms of size, content, and structure. The absolute amount of FBs and Connections between them refers to size similarity. It gives us two dimensions in the embedding space. To capture and measure content similarity we use the normalized amount of FBs per type. It has as many dimensions as different FB types are available. This way we can easily map each FB type to a specific dimension, sum the number of occurrences of each FB type within a FB network and normalize it. The Vertex Degree catches the structural composition. There are also a number of characteristics that could be considered as features. Function Blocks Connections FB Types Vertex Degree Extracted Features 61499 Project Project Hierarchy (SubApps) Function Block Network (Flattened) Hierarchical Clustering Embedding Space SubApp11 SubApp1 SubApp10 SubApp9 SubApp8 SubApp7 SubApp2 SubApp3 SubApp4 SubApp6 SubApp5 SubApp12 SubApp13 SubApp1 Fig. 4: Feature-based implementation for the propose architecture. Fig. 5: 2D UMAP Projection of extracted Features. The Data Connection types (integer, float, strings, etc.) have an impact on characterizing the FB content. Also, differentiating between Data Connections and Event Connections or the connection directions characterized the FB behaviour and could be considered as features. Also, the FB implementation details (such as state machines defined with ECC or Structured Text) can be extracted to be included as features. We chose the implemented features because we prioritized structure-metrics. The embedding is created for every project part in the candidate set, resulting in a feature matrix with dimension Candidates xFeatures. Through dimensionality reduction techniques, the clustering of high dimensional embeddings can be improved and noise can be reduced in the data. For our implementation we considered PCA, t-SNE and UMAP as optional projection methods. By projecting into 2-dimensional space, it is even possible to explore the embedding space visually, as shown in Figure 5, a 2D projection using UMAP. Using clustering methods, we then aim to form similarity groups in the spanned embedding space. The similarity is preserved in distances between the extracted feature vectors. We used a bottom-up hierarchical clustering approach that first assigns each sample to its own similarity group and then begins to merge clusters successively according to linkage criteria. As depicted in Figure 6, the result is a cluster hierar- Fig. 6: Example of the resulting hierarchical cluster tree. chy, which can be represented as a tree with a unique cluster containing all collected samples as the root. Each branch in the tree has a corresponding similarity value that divides its leaves. A threshold value then defines where the leaves should then be assigned to their similarity groups. As in the figure, the threshold can be represented as a horizontal line cutting the tree at a specific position. The number of intersection points corresponds to the number of resulting clusters, which are plotted in different colors. The four cut branches (orange, green, red and purple coloured) are identify as four groups of similar FB networks. One cut branch with one single leaf means a unique FB network (unique considering the similarity criteria) so it is not identified as a similarity group (Figure 6 does not contain any unique FB network). As a result, each detected similarity group contains the following information: •Similarity Value: The highest similarity value connecting contained tree leaves. The higher the value, the more distant and therefore dissimilar contained samples are. •Entities: The list of similar FB networks. The result of a good clustering mechanism will produce clusters with high intra-class similarity and low inter-class similarity. The resulting cluster structure is used as a result in itself, for inspection by a user. IV. EVALUATION To evaluate our approach, we considered an industry case example provided by a partner from the plant building domain. They provided a sample project which defines a single machine of a complete system. This project includes more than 1000 basic function blocks and about 150 subapplications. As an estimate, it represents less than 1% of the total number of devices in the system and has been designed by engineers with several years of experience in the area. Of the 150 subapplications, 136 subapplications are untyped ( 90%), i.e., are uniquely designed and implemented. For the evaluation, we used the feature-based implementation introduced in the previous Section and we considered the following configuration. In order to create the embeddings, we extracted the Features introduces in previous Section: (i) Function Blocks, (ii) Connections, (iii) FB Types, and (iv) Vertex Degree. Due to the limited size of the data, we consider the entire subapplication space without reduction by Projection. For the Clustering, we used a Threshold of 0.1. Although an exhaustive comparison of the results for all possible configurations is out of the scope of this contribution, we discuss in the next subsection the impact of the Threshold on the results. The result of the evaluation is a list of similarity groups. A similarity group is a set of subapplications that are considered to be identical or similar, based on the feature extraction and the clustering techniques. Note that the sets of subapplications do not overlap, i.e., a subapplication cannot be included in two different similarity groups. Each similarity group has a Similarity Value, which has to be less than the input Threshold. A. Results For the given configuration, the results are summarized in Table I. A total of 26 similarity groups (column Sim. groups) were identified, including a total of 74 subapplications (column Sim. subapps). We compared the results with a manual review of the subapplications to determine the accuracy of the technique. We considered the True Positive Rate (TPR) and the False Positive Rate (FPR). The TPR is the ratio of true positives (TP) regarding the total positives (true positives and false negatives, FN), that is: T P T P +F N . As there were no false negatives for the given configuration, the TPR = 1. The FPR is the ratio of false positives (FP) regarding the total negatives (false positives and true negatives, TN), i.e. F P F P +T N . As the number of false positives was 4 and the true negatives were 60, FPR ≈0.63. TABLE I: Evaluation of the similar subapplications. Sim. groups Sim. subapps TPR FPR 136 74 1 0.06 After reviewing the similar subapplications found, we categorized them into one of 4 different cases: 1) Two or more subapplications identical to an existing FB type. These subapplications should be created as instances of the FB type to improve their maintenance. 2) Two or more identical subapplications without matching an existing FB type. In this case, a new FB type must be created considering the functionality of the subapplications and naming convention. For example, we found a group with 2 different subapplications implementing identically a logic AND with 4 input data, depicted in Figure 8 (a) and (b), respectively. This 4-input AND should be implemented as a common FB type to avoid the manual implementation for each instance, which is tedious and error-prone for such basic functionality and to improve the maintenance. 3) Two or more different subapplications implementing similar functionality. For example, in the sample results, we found 2 complex subapplications (with about 10 FBs and dozens of connections) with just some slight differences in the number of external connections. We depicted one blurred version of these subapplications in Figure 7 as example of these complex structures. These subapplications need to be analyzed to decide about how to reuse or redesign them. 4) Two or more similar subapplications without functional relation. These are false positives. Fig. 7: Subapplication to be redesigned. (a) (b) Fig. 8: Two identical implementations for 4-input AND. Note: Some of the labels in the pictures have been changed or blurred because non-disclosure agreements. B. Discussion and threats to validity As a result of this study, we conclude that feature-based similarity detection provides useful feedback to find related components. This detection can be integrated into the development ecosystem to provide valuable information to the engineers during the design time. This information can lead to the reuse of common elements in the current project and, therefore, improve its maintenance. This information can also help to identify candidate components to extend the core platform and reduce the development efforts in similar future projects. Furthermore, the identification of matching patterns can be used as input for refactoring operations [38], such as substituting a group of nested FBs with an existing FB type and extending it to get the functionality needed by a subapplication. In the current work, we implemented one of the proposed techniques (feature-based) so the comparison with other techniques is considered for future work. Also, although a full comparison of the different feature extractions is beyond the scope of this paper, we analyzed the impact of the clustering Threshold to get an optimal value for the given project. Figure 9 depicts the evolution of the results with the Threshold. For a given Threshold (X-axis), a number of similarity groups are found (Y-axis) and each group contains the identified number of similar subapplications (Z-axis). That is, for a low Threshold (≈0), the results include 21 similarity groups that only include exact clones, that is, subapplications with the same connections, internal FBs, and FB types. The number of similar subapplications is 2, 3 or 4 per group. These results are of interest but could be achieved with deterministic approaches. As the Threshold increases, additional groups are found and some of the groups include also additional subapplications. As we pointed out in the previous subsection, these additional subapplications are similar but differ slightly and can also lead to a redesign of the subapplications to improve the reuse. However, with a higher Threshold, we also get false positives, i.e., FBs identified as clones that are not similar or intended to be reused as one single FB type. This behavior is especially significant for T hreshold > 0.2, when similarity groups that had a very high similarity for lower Thresholds now include several false positives (decreasing the Similarity Value). It is not depicted in the chart, but for the case study, the limit case is reached for T hreshold ≈5, when all the untyped subapplications are identified as similar in one single similarity group. For the results described in the previous subsection, we considered the T hreshold = 0.1, as all the similarity groups are detected (i.e. there are no false negatives) but we also found some false positives. Note: Because the sensibility of the Threshold does not increase linearly, the increase of the Threshold in the evaluation is also not linear. Regarding other threats to the validity, we have to point out that there is no systematic check of results by the industry partner but they were discussed in the working groups to confirm their feasibility. Also, we cannot reveal details of the analyzed project due to non-disclosure agreements. As we only used one industry system, the results might not be generalizable to other systems. But we can assume that at least is representative of the possible findings in similar systems in the plant building domain. V. CONCLUSIONS AND FUTURE WORK In this paper, we described how the management of reusable components in cyber-physical production systems is challenging in practice. Based on this challenge we described how to identify existing repeating patterns in a system based on the system network and artificial intelligence techniques. With our initial implementation, we analyze a case study project, getting significant results to support the improvement of the CPPS design. These results can be improved by the analysis of different techniques and configurations to increase the accuracy of the results. On the one side, we intend to validate this approach systematically with complete industry projects. On the other side, we consider analyzing the results to identify potential refactoring operations. The automation of such operations into existing IDEs (e.g., 4diac IDE) can help to improve the system quality and highly reduce the projectspecific efforts. ACKNOWLEDGEMENTS The financial support by the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development, and the Christian Doppler Research Association is gratefully acknowledged. We also explicitly want to thank our industry partner for their continuous support. We further thank Elene Kutsia for her feedback and support on this paper. REFERENCES [1] B. Vogel-Heuser, A. Fay, I. Schaefer, and M. Tichy, “Evolution of software in automated production systems: Challenges and research directions,” Journal of Systems and Software, vol. 110, pp. 54– 84, 2015. [Online]. Available: https://www.sciencedirect.com/science/ article/pii/S0164121215001818 [2] R. Rabiser and A. Zoitl, “Towards mastering variability in softwareintensive cyber-physical production systems,” Procedia Computer Science, vol. 180, pp. 50–59, 2021, proceedings of the 2nd International Conference on Industry 4.0 and Smart Manufacturing (ISM 2020). [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S1877050921001666 [3] A. Spanlang, B. Weiss, and W. Wukovits, “Quantifying uncertainty in metallurgical process operation and design,” K1-MET, Tech. Rep., 04 2018. [4] H. Meissner and J. C. Aurich, “Implications of cyber-physical production systems on integrated process planning and scheduling,” Procedia Manufacturing, vol. 28, pp. 167–173, 2019, 7th International conference on Changeable, Agile, Reconfigurable and Virtual Production (CARV2018). [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2351978918313696 [5] D. E. O’Leary, “Artificial intelligence and big data,” IEEE intelligent systems, vol. 28, no. 2, pp. 96–99, 2013. [6] A. Zoitl, T. Strasser, C. Sunder, and T. Baier, “Is IEC 61499 in harmony with IEC 61131-3?” IEEE Industrial Electronics Magazine, vol. 3, no. 4, pp. 49–55, 2009. [7] W. Dai and V. Vyatkin, “Redesign distributed IEC 61131-3 plc system in IEC 61499 function blocks,” in 2010 IEEE 15th Conference on Emerging Technologies Factory Automation (ETFA 2010), 2010, pp. 1–8. [8] W. W. Dai and V. Vyatkin, “A case study on migration from IEC 61131 plc to IEC 61499 function block control,” in 2009 7th IEEE International Conference on Industrial Informatics, 2009, pp. 79–84. [9] M. Wenger and A. Zoitl, “Re-use of IEC 61131-3 structured text for IEC 61499,” in 2012 IEEE International Conference on Industrial Technology, 2012, pp. 78–83. Fig. 9: Similarity groups found depending on the Threshold. [10] L. Sonnleithner, B. Wiesmayr, V. Ashiwal, and A. Zoitl, “IEC 61499 distributed design patterns,” in 26th IEEE International Conference on Emerging Technologies and Factory Automation, ETFA 2021, ser. ETFA ’21. IEEE, 2021. [11] T. Mende, R. Koschke, and F. Beckwermert, “An evaluation of code similarity identification for the grow-and-prune model,” Journal of Software Maintenance and Evolution: Research and Practice, vol. 21, no. 2, pp. 143–169, 2009. [12] R. Koschke, P. Frenzel, A. P. Breu, and K. Angstmann, “Extending the reflexion method for consolidating software variants into product lines,” Software Quality Journal, vol. 17, no. 4, pp. 331–366, 2009. [13] M. Jiang, J. Zhang, H. Zhao, and Y. Zhou, “Maintaining software product lines—an industrial practice,” in 2008 IEEE International Conference on Software Maintenance. IEEE, 2008, pp. 444–447. [14] T. Ziadi, L. Frias, M. A. A. da Silva, and M. Ziane, “Feature identification from the source code of product variants,” in 2012 16th European Conference on Software Maintenance and Reengineering. IEEE, 2012, pp. 417–422. [15] W. Wang, G. Li, B. Ma, X. Xia, and Z. Jin, “Detecting code clones with graph neural network and flow-augmented abstract syntax tree,” 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 261–271, 2020. [16] C. Huang, H. Zhou, C. Ye, and B. Li, “Code clone detection based on event embedding and event dependency,” 2021. [Online]. Available: https://arxiv.org/abs/2111.14183 [17] C. Sullivan, “Can we trust deep learning to detect duplicate code? GitHub did it,” https://hyperight.com/ can-we-trust-deep-learning-to-detect-duplicate-code-github-did-it/, 2020, [Online; accessed 28-April-2022]. [18] C. K. Roy and J. R. Cordy, “A survey on software clone detection research,” Queen’s School of Computing TR, vol. 541, no. 115, pp. 64– 68, 2007. [19] I. D. Baxter, A. Yahin, L. Moura, M. Sant’Anna, and L. Bier, “Clone detection using abstract syntax trees,” in Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272). IEEE, 1998, pp. 368–377. [20] J. Ghofrani, M. Mohseni, and A. Bozorgmehr, “A conceptual framework for clone detection using machine learning,” in 2017 IEEE 4th International Conference on Knowledge-Based Engineering and Innovation (KBEI). IEEE, 2017, pp. 0810–0817. [21] J. Svajlenko and C. K. Roy, “Efficiently measuring an accurate and generalized clone detection precision using clone clustering.” in SEKE, 2016, pp. 426–433. [22] A. Sheneamer and J. Kalita, “Semantic clone detection using machine learning,” in 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 2016, pp. 1024–1028. [23] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE transactions on neural networks and learning systems, vol. 32, no. 1, pp. 4–24, 2020. [24] P. Riba, A. Fischer, J. Llad´ os, and A. Forn´ es, “Learning graph edit distance by graph neural networks,” Pattern Recognition, vol. 120, p. 108132, 2021. [25] J. A. Bondy, U. S. R. Murty et al.,Graph theory with applications. Macmillan London, 1976, vol. 290. [26] D. Koutra, N. Shah, J. T. Vogelstein, B. Gallagher, and C. Faloutsos, “Deltacon: Principled massive-graph similarity function with attribution,” ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 10, no. 3, pp. 1–43, 2016. [27] P. Wills and F. G. Meyer, “Metrics for graph comparison: a practitioner’s guide,” Plos one, vol. 15, no. 2, p. e0228728, 2020. [28] M. Berlingerio, D. Koutra, T. Eliassi-Rad, and C. Faloutsos, “Netsimile: A scalable approach to size-independent network similarity,” arXiv preprint arXiv:1209.2684, 2012. [29] D. D. Lewis, “Feature selection and feature extraction for text categorization,” in Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992, 1992. [30] S. Khalid, T. Khalil, and S. Nasreen, “A survey of feature selection and feature extraction techniques in machine learning,” in 2014 Science and Information Conference, 2014, pp. 372–378. [31] J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, “Graph neural networks: A review of methods and applications,” AI Open, vol. 1, pp. 57–81, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2666651021000012 [32] W. Yu, G. Zeng, P. Luo, F. Zhuang, Q. He, and Z. Shi, “Embedding with autoencoder regularization,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2013, pp. 208–223. [33] P. Indyk and R. Motwani, “Approximate nearest neighbors: towards removing the curse of dimensionality,” in Proceedings of the thirtieth annual ACM symposium on Theory of computing, 1998, pp. 604–613. [34] I. Assent, “Clustering high dimensional data,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 2, no. 4, pp. 340–350, 2012. [35] M. Ringn´ er, “What is principal component analysis?” Nature biotechnology, vol. 26, no. 3, pp. 303–304, 2008. [36] Y. Wang, H. Huang, C. Rudin, and Y. Shaposhnik, “Understanding how dimension reduction tools work: an empirical approach to deciphering t-sne, umap, trimap, and pacmap for data visualization,” arXiv preprint arXiv:2012.04456, 2020. [37] T. S. Madhulatha, “An overview on clustering methods,” arXiv preprint arXiv:1205.1117, 2012. [38] M. Oberlehner, L. Sonnleithner, B. Wiesmayr, and A. Zoitl, “Catalog of refactoring operations for IEC 61499,” in 26th IEEE International Conference on Emerging Technologies and Factory Automation, ETFA 2021, Vasteras, Sweden, September 7-10, 2021. IEEE, 2021, pp. 1–4. [Online]. Available: https://doi.org/10.1109/ETFA45728.2021.9613398