Extending Grafana for Visualizing Data from CERN's Large-Scale Industrial Control Systems

Extending Grafana for Visualizing Data from CERN’s Large-Scale Industrial Control Systems August 2025 AUTHOR: Lasse Bærland Strand Norwegian University of Science and Technology SUPERVISORS: Rafal Kulaga Martin Zemko CERN openlab Report 2025 PROJECT SPECIFICATION Dashboards play an important role in visualizing historical data from control systems, enabling transforming complex datasets into actionable insights. They enable operators, engineers and scientists to monitor system performance, identify patterns and detect anomalies with ease. At CERN, several custom web-based tools have been developed to enable visualization of historical data from WinCC OA-based control systems. Recently, there has been growing interest in complementing these custom solutions with Grafana - a popular solution for creating dynamic web dashboards with rich data visualization. Grafana provides built-in support for querying data from TimescaleDB/PostgreSQL - the database that will soon be used to store historical data from WinCC OA systems used at CERN. However, the complexity and large size of the signal metadata in these systems present unique challenges. To address this, custom Grafana extensions are essential to enable users to efficiently browse and select signals for display in dashboard panels. This project aims to develop proof-of-concept Grafana extensions that will enrich the platform’s functionality. These extensions will focus on providing high-performance, user-friendly widgets for browsing and selecting signal metadata, allowing users to create dynamic dashboards without the detailed knowledge of the underlying database schema. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 1 CERN openlab Report 2025 ABSTRACT The migration of CERN’s WinCC OA historical data to a TimescaleDB backend enables the use of Grafana. However, Grafana’s native data selection tools prove insufficient for our use cases, lacking the performance and filtering capabilities required to handle millions of signals. This report details the development of custom Grafana panel plugins to overcome these limitations. The primary solution is the Signal Selector, a high-performance plugin that enables users to efficiently browse and select signals without prior knowledge of the database schema. It features both a familiar Table View and a hierarchical Tree View, with support of filtering. A secondary plugin, the Band Chart Converter, was also developed to automate the complex and repetitive task of styling time-series graphs in dynamic dashboards. The resulting plugins have been demonstrated to major experiments (ATLAS, ALICE, CMS) with positive feedback, and are ready for evaluation by first users. This work removes a key technical barrier to using Grafana for control system data visualization at CERN. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 2 CERN openlab Report 2025 TABLE OF CONTENTS 1 Introduction 4 1.1 LimitationswithGrafana.............................. 4 1.2 GrafanaPlugins .................................. 5 2 Signal Selector 5 2.1 TableView ..................................... 6 2.2 TreeView...................................... 6 2.3 Filtering ....................................... 8 2.3.1 BuilderMode................................ 8 2.3.2 AdvancedMode .............................. 9 2.4 ConfigurationOptions ............................... 9 2.5 OptimizationMethods ............................... 10 2.5.1 Memoization ................................ 10 2.5.2 Virtualization ................................ 11 2.5.3 LazyLoading................................ 11 2.6 Codebase...................................... 11 2.6.1 DependencyGraph ............................ 12 2.6.2 AIUsage .................................. 13 2.7 Example Usage in Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3 Band Chart Converter 14 3.1 Objective ...................................... 15 3.2 Result........................................ 15 4 Impact 16 5 Future Development 16 6 References 17 EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 3 CERN openlab Report 2025 1 Introduction As detailed in the preceding project specification, the migration of WinCC OA historical data to a TimescaleDB backend provides a key opportunity to integrate Grafana into CERN’s control system infrastructure. This project was established to address the primary challenge associated with this integration: providing users with an efficient means to navigate and select signals at a scale far beyond what Grafana’s standard tools can support. While the initial goal was to produce a proof-of-concept, the work has since evolved into a solution that is ready for its preview release. 1.1 Limitations with Grafana The core value of introducing Grafana lies in its ability to facilitate dynamic dashboards, where user selections within one interface element can programmatically update the data displayed across multiple visualization panels. However, this potential is directly hindered by a significant technical limitation in Grafana’s native functionality. The standard interface for data selection is a simple variable dropdown menu, as illustrated in Figure 1. While this component may be adequate for applications with a modest number of signals, it presents several critical limitations that make it unsuitable for the scale of CERN’s operations: •Breaks at large scale: Attempting to load it with the millions of signals present in CERN’s WinCC OA systems would cause the dashboard to become unresponsive, rendering it fundamentally unusable. •Lack of Metadata Display: The dropdown list shows only the signal name, providing no other contextual metadata. This makes it extremely difficult for users to distinguish between signals or verify that they are selecting the correct one without pre-existing knowledge. •No Advanced Filtering: The component’s functionality is restricted to a basic text search. It offers no options for advanced filtering based on signal properties. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 4 CERN openlab Report 2025 Figure 1: Grafana’s native dropdown component. 1.2 Grafana Plugins A key architectural feature of Grafana is its extensibility, which is primarily achieved through a plugin system. Plugins are self-contained software components that can be installed to add new capabilities or integrations to the Grafana platform. This modular approach allows users and developers to adapt Grafana to a vast array of specific use cases, extending its functionality far beyond its default configuration [1]. The solution developed for this project takes the form of a panel plugin. In Grafana, a panel is the fundamental building block of a dashboard, typically used to display data in a visual format, such as a graph or a table. Creating a custom panel plugin allowed us to design a completely new interactive widget that could be added to any dashboard. 2 Signal Selector To overcome the limitations of Grafana’s native components, a custom panel plugin named the Signal Selector was developed using React and TypeScript. The plugin’s core function is to control a Grafana dashboard variable. When a user selects one or more signals within the plugin, their identifiers are written to the dashboard’s URL. This action updates the associated Grafana variable, which can then be referenced by any other panel on the dashboard. This mechanism allows the Signal Selector to act as a dynamic controller, enabling other panels to automatically reflect the user’s choices. The plugin provides an interface to browse the control system’s signal metadata, enabling users to locate and select signals without knowledge of the underlying database schema. It offers two views for this purpose: a Table View and a Tree View. The features of each view are detailed in the following sections. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 5 CERN openlab Report 2025 2.1 Table View The Table View displays signal metadata in a tabular format (Figure 2). Figure 2: The Signal Selector panel in Table View mode. The main functionalities of this view are: •Metadata Display: Presents the following columns of signal metadata: Element ID, Element Name,Comment,Alias,Valid From, and Valid To. The columns are sortable, and users can toggle their visibility. •Advanced Filtering: Provides a filtering system with two modes. The "Builder Mode" (shown in Figure 2) uses a graphical interface to construct queries. We also support an "Advanced Mode" that allows users to input raw SQL WHERE clauses directly. This filter component is shared with the Tree View. •Configurable Selection Output: The metadata column used to populate the Grafana variable is user-configurable. The user can choose to use Element ID,Element Name or Alias for its selection. 2.2 Tree View The Tree View offers a hierarchical representation of the signals, allowing users to navigate the system structure intuitively (Figure 3). EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 6 CERN openlab Report 2025 Figure 3: The Signal Selector panel in Tree View mode. A primary consideration for this view was ensuring a responsive and snappy user experience. Building a hierarchical view on-the-fly from raw system data would require computationally expensive recursive queries, resulting in poor performance. To circumvent this, we pre-compute the tree, as illustrated in Figure 4. Figure 4: Data flow for the pre-computation of the Tree View hierarchy. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 7 CERN openlab Report 2025 The process begins with the various tree structures defined within WinCC OA, as the hierarchical relationships for some trees are only available within the live system configuration and not in the archived data. A custom script is triggered to parse these source hierarchies. This script extracts all the parent-child relationships and transforms them into a single, optimized table which is then stored in the TimescaleDB database. Crucially, this script execution is completely decoupled from user activity in Grafana. It is run only when updates to the source hierarchies are needed. When a user interacts with the Tree View in the Grafana plugin, it does not query the complex source data. Instead, it sends simple, fast queries to this dedicated relational table. Expanding a node in the UI corresponds to a single, indexed database query to retrieve its immediate children. This architecture is the key to the Tree View’s performance, as it transforms what would be a complex, multi-level query into a simple, efficient lookup. 2.3 Filtering The Signal Selector includes a shared filtering system utilized by both the Table and Tree Views. The filter state is preserved when switching between these views. The system operates in two distinct modes: a Builder Mode for graphical query construction and an Advanced Mode for raw SQL input. 2.3.1 Builder Mode The default interface is the Builder Mode, which provides a user-friendly graphical interface for constructing queries without requiring knowledge of SQL. As shown in Figure 5, users can add multiple filter conditions. For each condition, they select a metadata string field (Element Name,Comment or Alias), choose a comparison operation (e.g., Contains,Ends With), and provide a value. All conditions can be joined with either a global AND or OR operator. This mode is designed for quick filtering on the primary string-based columns. Figure 5: The filtering interface in Builder Mode, showing three active conditions joined by an AND operator. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 8 CERN openlab Report 2025 3.1 Objective A common and effective method for visualizing analog signals with uncertainty or operational ranges is a "band chart". This style typically consists of an average value shown as a line, with a shaded region between its corresponding minimum and maximum values. While Grafana supports this visualization, its implementation relies on a system of "style overrides". These overrides are static rules that target specific, hard-coded signal names. This static approach presents a significant limitation in a dynamic dashboard environment. When a user selects a new set of signals using the Signal Selector, the signal names in the time-series panel change. As a result, the pre-configured style overrides no longer match the new signals, and the band chart visualization breaks. Manually re-applying the dozen or so necessary overrides for each new signal set is a time-consuming and error-prone process that discourages interactive data exploration. The number of overrides needed to get the band chart style is equal to 2N+ 2, where Nis the number of signals. The objective was therefore to automate this styling process, creating a tool that could apply a consistent band chart style to a time-series panel, regardless of the specific signals being displayed. 3.2 Result The solution is another panel plugin, the Band Chart Converter. This plugin acts as a controller that can apply a set of style overrides to a separate time-series panel with a single action. The workflow is straightforward. A user first adds the Band Chart Converter to their dashboard. To link it to a time-series graph, they set the converter’s data source option to "– Dashboard –", a special Grafana setting that allows a panel to use another panel’s data. From there, they select the target time-series panel from a list. Once connected, the converter plugin automatically recognizes the signals displayed in the target panel and is ready to apply the styling. As shown in Figure 10, with the connection established, the user simply clicks the "Apply Band Chart Style" button. This action programmatically generates and applies all the necessary style overrides to the target panel to create the band chart effect. If the user then selects some new signals with the Signal Selector, they only need to click the button again to instantly re-apply the correct styling to the new data. This eliminates the need for any manual configuration of overrides and makes the use of band charts viable in a fully dynamic dashboard. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 15 CERN openlab Report 2025 Figure 10: The target time-series panel (left) after the styles have been applied, and the Band Chart Converter plugin (right). 4 Impact The primary impact of the developed plugins is their role as an enabler for creating fully dynamic dashboards within CERN’s Grafana infrastructure. By providing a scalable and intuitive method for signal selection, the Signal Selector removes a critical barrier to the platform’s adoption, in turn making engineers and physicists more inclined to use Grafana. To gauge initial interest and gather feedback, the plugins were demonstrated to key stakeholders from three major CERN experiments: ATLAS, ALICE, and CMS. The response from these sessions has been highly positive, with all groups expressing significant interest in adopting the Grafana plugins. The ATLAS experiment alone has over 100 active Grafana users, many of whom are expected to interact with this plugin. Across all experiments, the total number of potential users is projected to be in the hundreds. 5 Future Development The next steps for this project are to deliver the plugins to the initial group of users and collect their feedback. Future development will be based on this input, following an iterative process where user suggestions and requests guide the further development. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 16 CERN openlab Report 2025 6 References [1] Grafana Labs. Anatomy of a plugin. Grafana Developer Documentation. Accessed: 202508-14. url:https : / / grafana . com / developers / plugin - tools / key - concepts / anatomy-of-a-plugin. [2] Piumi Liyana Gunawardhana. What is Memoization in React? Syncfusion. Accessed: 2025-08-14. June 2024. url:https://www.syncfusion.com/blogs/post/what-ismemoization-in-react. [3] Patterns. List Virtualization. Accessed: 2025-08-14. url:https://www.patterns.dev/ vanilla/virtual-lists/. [4] Cloudflare. What is lazy loading? Accessed: 2025-08-14. url:https://www.cloudflare. com/learning/performance/what-is-lazy-loading/. [5] GitHub. GitHub Copilot. Accessed: 2025-08-18. url:https://github.com/features/ copilot. [6] Anthropic. Introducing Claude 4. Accessed: 2025-08-18. May 2025. url:https://www. anthropic.com/news/claude-4. [7] Deepmind. Gemini 2.5: Our most intelligent AI model. Accessed: 2025-08-18. Mar. 2025. url:https://blog.google/technology/google-deepmind/gemini-model-thinkingupdates-march-2025/. EXTENDING GRAFANA FOR VISUALIZING DATA FROM CERN’S LARGE-SCALE INDUSTRIAL CONTROL SYSTEMS 17