AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS

AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS August 2025 AUTHOR(S): Bernard Tam The University of Sydney SUPERVISOR(S): Jean-Charles Tournier Fernando Varela Rodriguez CERN openlab Report 2025 ABSTRACT This project explores the development of an AI-enhanced operator assistant for UNICOS, CERN’s UNified Industrial Control System. While powerful, UNICOS presents a number of challenges, including the cognitive burden of decoding widgets, manual effort required for root cause analysis, and difficulties maintainers face in tracing datapoint elements (DPEs) across a complex codebase. In situations where timely responses are critical, these challenges can increase cognitive load and slow down diagnostics. To address these issues, a multi-agent system was designed and implemented. The solution is supported by a modular architecture comprising a UNICOS-side extension written in CTRL code, a Python-based multi-agent system deployed on a virtual machine, and a vector database storing both operator documentation and widget animation code. Three specialised agents were developed to work under a single supervisor agent: one to automate widget decoding, one to perform root cause analysis by traversing device hierarchies, and one to trace the DPEs responsible for widget animation. Preliminary evaluations suggest that the system is capable of decoding widgets, performing root cause analysis by leveraging live device data and documentation, and tracing DPEs across a complex codebase. Together, these capabilities reduce the manual workload of operators and maintainers, enhance situational awareness in operations, and accelerate responses to alarms and anomalies. Beyond these immediate gains, this work highlights the potential of introducing multi-modal reasoning and retrieval augmented generation (RAG) into the domain of industrial control. Ultimately, this work represents more than a proof of concept: it provides a basis for advancing intelligent operator interfaces at CERN. By combining modular design, extensibility, and practical AI integration, this project not only alleviates current operator pain points but also points toward broader opportunities for assistive AI in accelerator operations. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 1 CERN openlab Report 2025 TABLE OF CONTENTS 1 BACKGROUND 3 1.1 UNICOS........................................ 3 1.2 UNICOS Pain Points and AI Solutions . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 AdditionalBenefits.................................. 6 2 METHODOLOGY 7 2.1 SystemOverview ................................... 7 2.2 AIWidgetDecoding ................................. 8 2.2.1 ImageSegmentation ............................. 8 2.2.2 ImageUpscaling ............................... 8 2.2.3 DataExtraction................................ 9 2.2.4 Documentation Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 AIRootCauseAnalysis ............................... 11 2.3.1 UNICOS REST API Endpoints . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.2 Documentation Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4 AI Tracing of Widget Part Animation DPEs . . . . . . . . . . . . . . . . . . . . 13 2.4.1 CodeRetrieval ................................ 13 3 RESULTS AND DISCUSSION 15 3.1 AIWidgetDecoding ................................. 15 3.2 AIRootCauseAnalysis ............................... 16 3.3 AI Tracing of Widget Part Animation DPEs . . . . . . . . . . . . . . . . . . . . 17 4 CONCLUSION 18 5 FUTURE WORK 18 6 REFERENCES 19 AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 2 CERN openlab Report 2025 1 BACKGROUND 1.1 UNICOS UNified Industrial Control System (UNICOS) is a framework at CERN to build industrial control systems [3]. More than 200 systems at CERN run on UNICOS, including the technical infrastructure for the accelerator complex [4]. UNICOS development is carried out using WinCC OA, a Supervisory Control and Data Acquisition (SCADA) system developed by Siemens to monitor and control automated processes. Programs in WinCC OA are written in CTRL code [6], which is similar in syntax to C [1]. As a result, development work on UNICOS in this project will be carried out in CTRL code. Figure 1: An image of the interior of the CERN Control Centre on CERN’s Prévessin site Figure 1above shows the interior of the CERN Control Centre (CCC), with the monitors in the background seen to be running UNICOS. Figure 2below provides an example of a UNICOS panel, offering a clearer view of what is usually displayed on those monitors. Figure 2: A screenshot showing an example of a UNICOS panel AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 3 CERN openlab Report 2025 1.2 UNICOS Pain Points and AI Solutions Various pain points relating to UNICOS have been experienced by different types of CERN personnel, including UNICOS operators and maintainers [7]. The objective of this section is to explain each of the pain points in detail, describe their potential implications, and outline the solutions that have been developed in this work to address each pain point. Figure 3: A screenshot showing an example of a UNICOS widget The first pain point for UNICOS operators manifests in the decoding of widgets. Figure 3 above shows an example of a widget in UNICOS. Correspondingly, the documentation provided to UNICOS operators that explains the symbols on each widget is displayed in Figure 4 below. When decoding a widget or multiple widgets simultaneously, operators may have to reference the documentation frequently, potentially leading to high cognitive load and information overload. This would especially be the case if the widget in question contains an uncommon symbol, thereby requiring manual reference to the documentation. This issue is exacerbated in situations where multiple widgets are indicating a critical alarm, meaning that the full details of widgets must be decoded immediately and subsequent action taken to ensure a continued safe operation of systems. This work seeks to automate the widget decoding process with the help of AI, thereby reducing the cognitive load on UNICOS operators and contributing to a continued safe operation of systems. Figure 4: An image of the documentation provided to UNICOS operators to decode a widget [2] AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 4 CERN openlab Report 2025 Another pain point for UNICOS operators relates to the manual and time-consuming effort to diagnose the root cause of an issue. When a widget indicates an error or alarm, the root cause of the issue would need to be identified. It is important to note that, in UNICOS, devices are represented at a very atomic level. As such, a widget might not necessarily represent a device, but rather, a widget may represent a single signal coming from a device, with multiple widgets possibly representing multiple signals coming from a single device. This effectively results in a hierarchy of widgets that affect each other significantly. Therefore, root cause analysis would usually be carried out by traversing such a hierarchy of widgets - essentially inspecting widgets associated with the widget in question, including its master, parents, and children, recursively tracing the issue to its source. The window in Figure 5below shows the master, parents, and children of a certain widget in the UNICOS user interface. Root cause analysis is usually carried out by an operator, but since this process requires manual effort and is time-consuming, this work aims to leverage AI to automate root cause analysis. Figure 5: A screenshot showing the hierarchy of a widget in the UNICOS user interface UNICOS maintainers also face a pain point when debugging and determining exactly which datapoint elements (DPEs) are responsible for animating different parts of a widget. Signals from devices in the form of DPEs are each responsible for animating different parts of a widget. Logic, expressed as conditional statements in code, sits between such signals and the widgets, thereby determining how each widget is to be animated. When widgets are not animated as expected or exhibit anomalous behaviour, debugging must be carried out by maintainers to investigate and determine exactly which DPEs are responsible for animating each part of the widget. However, the animation logic has become difficult to trace and understand owing to the complex codebase structure and ‘spaghetti code’, leading to this manual process taking maintainers more than 30 minutes on average. As a result, this work streamlines the tracing of DPEs responsible for animating different parts of the widget by employing AI to automate this process. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 5 CERN openlab Report 2025 1.3 Additional Benefits In addition to addressing each of the pain points mentioned, this project is likely to come with additional benefits, including experience with multi-modal models, experimentation with retrieval augmented generation (RAG) in a complex context, creation of reusable architecture, and foundation for human-in-the-loop (HMI) interfaces. Due to the visual nature of the UNICOS interface and widgets, it is evident that computer vision (CV) models and large language models (LLMs) supporting image inputs will be used throughout the project. As such, the completion of this project is expected to offer a wealth of hands-on experience with multi-modal models that can take both text and images as input, a key capability for future AI applications in control systems. Furthermore, this project presents an opportunity to experiment with RAG in a complex context. At the time of writing, there have been no prior recorded attempts at integrating RAG with UNICOS. As such, this project would provide valuable insight into the performance and feasibility of implementing RAG in real-world UNICOS applications, and in a wider sense, industrial control applications. Given the opportunity to tackle challenges like partial documentation and complex codebases, the results produced by this project would provide a clear signal into whether RAG has a promising future in such use cases and contexts. The completion of this project would also produce reusable architecture. The modular architecture of the finished project could be reused or extended for other HMI features and expert assistance tools. Components in this modular architecture include the AI assistant’s user interface (UI), the UNICOS REST API endpoints, the multi-agent system, abstracted database modules, and more. Finally, this project could act as a foundation for HMI interfaces as it explores a new interaction model between UNICOS operators and the system. As such, it could serve as a prototype for future assistive HMI designs. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 6 CERN openlab Report 2025 2 METHODOLOGY 2.1 System Overview Figure 6: A system diagram providing a visual overview of the system developed Figure 6above provides a visual overview of the various components in the system developed. Overall, the system consists of two major parts. The first is the UNICOS side of the system, which is essentially an extension of existing UNICOS components using CTRL code. The new AI Assistant UI is implemented in this part of the system, along with the REST API endpoints that would be called by the agent performing root cause analysis to retrieve real-time widget information. This part is to be installed as a package to an existing project. The second part is the multi-agent system, which is a separate part of the system developed in Python and hosted on a regular virtual machine (VM). This part of the system exposes REST API endpoints for response generation when a user sends a query through the AI Assistant UI. It also hosts the entire logic behind the response generation, which includes 1 supervisor agent and 3 specialised worker agents. The supervisor agent delegates tasks to the most appropriate specialised worker agent, whereas the specialised worker agents comprise an AI Widget Decoding Agent, an AI Root Cause Analysis Agent, and an AI Widget Part Animation DPE Tracing Agent. The second part of the system also includes the vector database, which stores the documentation usually available to UNICOS operators for the purposes of decoding a widget and performing root cause analysis. The vector database will be queried by the AI Widget Decoding Agent and AI Root Cause Analysis agent during RAG, thereby providing agents with the same materials available to human operators when carrying out their duties. The vector database also stores the code related to the animation of widgets in a separate collection. Its contents will be queried by the AI Widget Part Animation DPE Tracing Agent in order to jump through the code and trace the DPEs, providing it with the very resources UNICOS maintainers would have access to when debugging and determining which DPEs are responsible for animating widgets. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 7 CERN openlab Report 2025 Language model modules also form part of the second part of the system, consisting of a chat completion module and an embedding generation module. These not only form the basis of the multi-agent system, but are highly abstracted and modular such that they support the easy swapping of underlying models to be used with the system, and are easily extensible for any models released in the future to a point where it is plug-and-play. Image manipulation modules also make up the second part of the system, handling image segmentation and upscaling tasks in the preprocessing stage for each user query to ensure that each widget image is properly formatted and visible before further analysis is carried out. 2.2 AI Widget Decoding 2.2.1 Image Segmentation Unlike for panels, UNICOS does not support directly taking a screenshot of an individual widget - this is because widgets are not panels, but rather elements within a panel. Given that UNICOS does have built-in support for taking a screenshot of the entire panel, this was decided as a starting point to segment the image. Given screenshots of entire panels, the initial thought was to train an object detection machine learning model to recognise widgets and subsequently extract them individually. However, this approach was found to require too much training data and annotation work, requiring a significant investment in human labour. In order to better focus on other parts of the project, it was decided that another approach would be optimal if it exists. It was then discovered that each widget within a panel comes with a distinct solid white boundary around it, with a significant contrast against a grey background. It was also discovered that a workaround could be used to temporarily hide all elements in a panel except the widget in question before taking a screenshot of the entire panel. Such discoveries were taken advantage of and the OpenCV2 Python library was used to segment the widget from the panel screenshot. The specific steps are as follows: 1. Hide all elements except the desired widget temporarily and take a screenshot of the entire panel 2. Convert the screenshot to greyscale 3. Use canny edge detection to easily detect the distinct solid white boundary around the widget in the screenshot 4. Extract all contents of the image within detected boundary 2.2.2 Image Upscaling A significant problem was encountered during data extraction where the LLM supporting image input was wrongly extracting data due to the low resolution of the widget image. This was especially the case when symbols in a widget overlapped with each other. As a solution, an open-source Enhanced Deep Residual Networks (EDSR) model was used to upscale each widget image by a factor of 4. This process was carried out after image segmentation and before data extraction, and solved the issue of inaccurate data being extracted. It should be noted that the EDSR model is extremely lightweight, fitting completely on a CPU setup with no GPU required. The EDSR model is available through the super-image Python library, making its integration into the existing code quick and hassle-free. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 8 CERN openlab Report 2025 3 RESULTS AND DISCUSSION 3.1 AI Widget Decoding Figure 7: A screenshot demonstrating AI Widget Decoding The screenshot in Figure 7above demonstrates AI Widget Decoding in action. The AI Assistant window, which was developed in this project as a new addition to UNICOS, is shown in the foreground. As seen within the window, the conversation history between the user and the system is shown, with the user sending a message requesting help in decoding the widget ‘FSVE_013’, and attaching the said widget. In response to the user’s query, the system is shown to respond with information about the widget in its decoded form. The system’s response suggests that the widget’s green body indicates that it is in Auto/Manual mode. It also suggests that the cyan ‘O’ in the top left corner indicates old data. The system finishes with stating that the white ‘M’ in the bottom right corner confirms that it is in manual mode. The descriptions produced by the system fully align with information found in the documentation provided to UNICOS operators, as seen in Figure 4. This successfully demonstrates the system’s ability to accurately decode the widget, correctly explaining the meaning of all 3 elements in the widget. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 15 CERN openlab Report 2025 3.2 AI Root Cause Analysis Figure 8: A screenshot demonstrating AI Root Cause Analysis The screenshot in Figure 8above demonstrates AI Root Cause Analysis in action. Similar to Figure 7that demonstrates AI Widget Decoding, the AI Assistant window is shown in the foreground. This same window is used by users to interact with the system for AI Root Cause Analysis. The window shows a different conversation history between the user and the system where the user makes a request to identify the root cause of issues with the widget ‘FSVE_015’ and attaches the said widget. The system is shown to have responded to the user’s query with an explanation of the potential root cause and suggestions to resolve the issue. In the system’s response, the root cause is suggested to be a communication problem with the frontend, referencing the frontend status code 10 and that such a code indicates a certain counter is not updated. The system’s response also states that the device status bits point to the device being on and running in manual mode. It finishes with suggesting a further investigation into the frontend and configuration of the device. In order to be aware that the frontend status code was 10, the system must have been able to access the data through the ‘Get Widget Frontend Status’ tool mentioned in Section 2.3.1 above, then access the documentation about frontend status codes to determine exactly what a value of 10 meant. Similarly, to retrieve the device status bits, the system must have been able to access the ‘Get Widget Device Status’ tool also mentioned in Section 2.3.1 above and subsequently access documentation about what each device status bit maps to. This successfully demonstrates the system’s ability to access live data from devices and understand documentation to make sense of the live data retrieved. The system’s output also highlights its ability to make sensible suggestions on how to proceed in investigating the issue. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 16 CERN openlab Report 2025 3.3 AI Tracing of Widget Part Animation DPEs Figure 9: A screenshot demonstrating AI Tracing of Widget Part Animation DPEs The screenshot in Figure 9above demonstrates AI Tracing of Widget Part Animation DPEs in action. This use case also makes use of the same AI Assistant window for users to interact with the system, just like both the previous use cases. As such, the same window is shown in the foreground. In the conversation history within the window, the user requests the system to assist in tracing the DPEs responsible for animating the widget ‘FSVE_014’ and attaches the said widget. The system then responds with the DPEs responsible for animating the widget and the method name containing the relevant code. The DPEs responsible for animating the widget are returned in the format of a numbered list, along with an explanation of which parts of the widget each DPE animates, including the warning text, body colour, and more. The response ends with the system directing the user towards a certain method in the code to further explore the exact effect of each DPE on the widget’s animation. To successfully trace the DPEs responsible for animating the widget, the system would have had to make use of the tools mentioned in Section 2.4.1 above to retrieve methods and sections of code by means of semantic search, file name search, method name search, or a combination thereof. By using such tools to navigate the codebase, jump between different methods, and trace the DPEs responsible for animating the widget, the system demonstrates its ability to find its way around complex codebase structures, understand what each section of code does, and purpose these skills towards a specific use case. The system’s response further illustrates its ability to explain what it has understood about the code to the user and point them in the right direction should they wish to dive deeper into the code themselves. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 17 CERN openlab Report 2025 4 CONCLUSION In this project, a multi-agent system was successfully developed to address three major pain points experienced by UNICOS operators and maintainers: widget decoding, root cause analysis, and the tracing of DPEs responsible for animating widgets. As demonstrated by the results of this project, the system establishes the capability of AI in substantially reducing cognitive load for operators, speeding up diagnostic processes, and supporting maintainers in navigating a complex codebase structure. Beyond tackling these immediate challenges, this work represents an important first step in providing AI-powered assistance to operators in the CCC. By incorporating multi-modal reasoning, retrieval augmented generation, and a modular architecture, the system establishes both a practical proof-of-concept and a methodological foundation for future research. Beyond its immediate benefits, this work also provides a basis for subsequent exploration into expanding the scope of intelligent support functionalities within UNICOS. 5 FUTURE WORK In terms of future work, an area for exploration could be the fine-tuning of models with CERNspecific data. In doing so, models would be trained to better comprehend CERN-specific vocabulary and terminology that would not otherwise be understood by foundation models. When such a fine-tuned model is used as the reasoning model powering the multi-agent system, it is expected to yield responses of a higher quality. Another possible direction of future work would be extending usage of the vector database storing the codebase to more contexts and purposes. The vector database has been populated with code in a way that takes extensibility and multiple use cases into account. As such, it would be sensible to take advantage of the already implemented functionalities that support searching the codebase by file name, method name, semantic similarity, or a combination thereof to extend beyond the tracing of widget animation DPEs. It would also be helpful to integrate CI/CD pipelines to update the vector database storing the codebase. This is because in case of any change to the code used in production for animating widgets, the updated code must automatically be synchronised into the vector database to reflect the latest changes. This would ensure that responses from the system are always up to date and reflect the current reality. AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 18 CERN openlab Report 2025 6 REFERENCES [1] ETM professional control GmbH. Control programming - Introduction to CTRL.url: https://www.winccoa.com/documentation/WinCCOA/3.18/en_US/Control_Grundlagen/ Control_Grundlagen-15.html (visited on 2025). [2] European Organization for Nuclear Research. UNICOS CPC Widget Help.url:https: //unicos.web.cern.ch/pages/documentation.html (visited on 2025). [3] Philippe Gayet, Renaud Barillere, et al. “UNICOS a framework to build industry like control systems: Principles & Methodology”. In: 10th ICALEPCS (2005). [4] Lukasz Goralczyk et al. “CERN SCADA Systems 2020 Large Upgrade Campaign Retrospective”. In: JACoW (2022), pp. 156–160. [5] NVIDIA Corporation. Llama 4 Models — NVIDIA NeMo Framework User Guide.url: https://docs.nvidia.com/nemo-framework/user-guide/latest/vlms/llama4.html (visited on 2025). [6] Riku-Pekka Silvola and Laura Sargsyan. “DevOps and CI/CD for WinCC Open Architecture Applications and Frameworks”. In: JACoW (2022), pp. 281–285. [7] Jean-Charles Tournier. UNICOS AI Operator Assistant.url:https : / / codimd.web. cern.ch/kbBPE8mjQBKd9tBmMJd22Q (visited on 06/2025). AI-ENHANCED OPERATOR ASSISTANCE FOR UNICOS APPLICATIONS 19