LEAPS-INNOV D5.1 - Collated existing solutions and future requirements of facilities for beamline synchronization

Deliverable no. D05.1 Page 1 of 6 Project information Project full title LEAPS pilot to foster open innovation for accelerator-based light sources in Europe Project acronym LEAPS-INNOV Grant agreement no. 101004728 Instrument Research and Innovation Action (RIA) Duration 01/04/2021 – 31/03/2025 Website Deliverable information Deliverable no. D5.1 Deliverable title Collated existing solutions and future requirements of facilities for beamline synchronization Deliverable responsible ALBA-CELLS, DESY Related WorkPackage/Task WP5 “POSIT” / Task 5.3 “Synchronization” Type (e.g. Report; other) Report Author(s) Antonin Broquet, Ricardo Hino, Babak Kalantari, Oscar Matilla, Jean-Paul Ricaud, Oliver Seeck, Ervis Suljoti, Tim Wilksen, Shu Zhang, and all members of WP 5.3 Dissemination level Document Version 1.0 Date 21.03.2022 Download page Document information Version no. Date Author(s) Comment 1.0 07.03.2022 ALBA & DESY First draft 1.1 21.03.2022 LEAPS-INNOV WP 5.3 During WP 5.3 meeting 1.11 24.03.2022 LEAPS-INNOV WP 5.3 Corrections 1.12 24.03.2022 ALBA & DESY Small Corrections Deliverable no. D05.1 Page 2 of 6 [Table of Contents] Introduction…………………………………………………………………………………………………………………………… 2 Existing accelerator timing systems at LEAPS facilities…………………………………………………………… 2 Remarks on experiments with high synchronization……………………………………………………………… 3 Future requirements of a synchronization system for the beamlines …………………………………… 5 Introduction Future synchrotron radiation and X-ray laser experiments require complex synchronization of the different hardware components at the beamline and the accelerator. The reason is the need for fast in-situ and operando measurements to understand the kinematics and dynamics of nature on microand nano-second scales. These experiments are enabled by the latest development of ultra-fast detectors, actuators, the CW operation modes of lasers and fast switching external fields. Each single device of such experiments has to be exactly synchronized with the X-ray bunch sequence of the X-ray laser or the synchrotron radiation source. System-wide synchronization is a major challenge as each hardware offers its proprietary synchronization protocols and interfaces. In addition, a working solution installed at one particular LEAPS facility should be transferable to other LEAPS facilities without much adaptation work needed. To date, LEAPS facilities, are implementing their synchronization systems in the accelerator experimental stations on a case-by-case basis. To develop a LEAPS wide solution on fast synchronization, the current states of the different hardware and software solutions at different LEAPS facilities have been collected, including both the accelerator timing systems and the existing solutions in the experimental stations. From this compilation the requirements of a generic synchronization system have been discussed and are presented in this document. Existing accelerator timing systems at LEAPS facilities This summary is not intended to assess the performance of each existing accelerator timing systems. The aim is to compile all relevant information of the different timing systems since experimental stations will necessarily have to interface with accelerator timing system in order of to achieve nanosecond time scale synchronization with respect to the incoming beam. Deliverable no. D05.1 Page 3 of 6 Synchronization of X-ray experiments relies on the accelerator timing system as master. In all participating LEAPS facilities, the presented accelerator timing systems are purely event based. Different hardware solutions exist, based on products from different companies: • MRF (Micro-Research Finland) based protocol. Widely used in many facilities: ALBA, PSI, BESSY, Diamond. MRF has opened part of the source code. It reaches nanosecond resolution and tens of picoseconds rms jitter. Its advantages include the simplicity of its setup. On the weak side, it is not thought to realize complex synchronization pattern execution and has limited functionalities for diagnostics or to compensate e.g. temperature drifts on time scales below 8ns. In addition, it was not designed to allow twoway communication, although some efforts were made by particular facilities to implement particular solutions. • Greenfield (France). It is used at SOLEIL. It follows similar architecture as MRF system based in master/slave components. When upgrades are needed, modifications or updates must be performed by Greenfield. Strong and weak points are similar to the MRF based systems. • White Rabbit (CERN) is an open hardware/software. It is more generic and allows bidirectional communication. Recently, it became capable of synchronizing with RF and, therefore, able to be used in the synchrotron radiation sources. The system is complex and needs dedicated staff for development and implementation, however, there is the idea to create a White Rabbit starter kit to ease implementation. Several facilities evaluate White Rabbit as solutions for their upgraded facilities. • Other systems developed by the facility, e.g. delivering full bunch information: XFEL, PETRA IV, based on MicroTCA. These systems are more complex as they have to deal with particular needs of the facility and require a dedicated developer team. All those timing systems are capable of fulfilling future 4th generation sources requirements. In particular, the XFELs are prepared already for a number of fully synchronized experiments, however, synchrotron radiation experiments usually cannot be operated in a synchronized fashion by just extrapolating the techniques of an X-ray laser experiment. Different tailor-made solutions have been implemented for special types of fully synchronized experiments at synchrotron radiation sources. However, no generic solution exists. In summary, all LEAPS facilities have different hardware platforms. It is unrealistic to aim for a common and unified hardware platform at all LEAPS facilities for the accelerator timing system. These hardware platforms are critical for operation of the accelerators, an extensive commissioning of the accelerator would be required when replacing it and they are expensive systems. Once installed, the systems are kept for decades with minimum changes. Therefore, interfaces and protocols have to be defined to be able to fit user experiments to the different hardware platforms at the LEAPS facilities. Deliverable no. D05.1 Page 4 of 6 Remarks on experiments with high synchronization Synchronized methods are well established when using laser-pump-probe experiments. However, once properly installed at a LEAPS beamline, laser-pump-probe experiments cannot easily be moved to a different facility due to the different schemes of hardware, synchronization schemes, control systems and interfaces. Future and already existing experiments at LEAPS beamlines will comprise much more complex synchronization tasks between the machine timing system and the hardware and software at the beamline. Game changer is the availability of new MHz imaging detectors, fast actuators, X-ray pulse pickers and the capability of fast high-throughput networks to store streams of very large data sets. With these new developments users of LEAPS facilities will, and do already now, request experiments with complex synchronization patterns and high timing constraints. The following needs and opportunities have to be considered for successfully building future complex-synchronization experiments: • Laser pump-probe type of experiments are not a big issue. They have been done since more than two decades. However, standardization of the interfacing is lacking. • New developments to be integrated are the X-ray pulse pickers (e.g. from TXproducts) for X-ray bunch picking and shaping. • Fast detectors (e.g. XSpectrum, Rigaku) allow for acquisition frame rates close or equal to the X-ray bunch rate. Unfortunately, almost all detectors rely on internal clocks which makes synchronization difficult and requires exact triggering. • For each signal or event, it may be of great advantage to store an absolute time stamp with nanosecond precision to allow synchronization of the whole system. It might be even desirable to introduce a unique number derived from the cycle or pulse number attached to the distributed timing system information. This information might be used for tagging synchronization events as well as ADC data and more. • Some X-ray methods require the exact knowledge of the bunch charge (not necessarily on nsec speed) and preand post-triggers during the top-up injection procedure of the accelerator. • Synchronization of multiple motion controllers, encoders and other actuators must be done in respect to the bunch information. Even being orders of magnitude slower, unifying the systems would ease implementations and diagnostics. • When synchronizing hardware at high speeds, the internal data transfer time of each hardware device may be the limiting factor. In particular this is relevant if this time is not constant, e.g. for hardware working with interrupts, or for motion controller with feedback to encoders. Here, the absolute time stamp of each event or the unique number, as mentioned above, may help to partially mitigate this problem. • Whereas accelerators are fully prepared for synchronization, also on hardware level, this is different at the beamlines. Hardware can only be synchronized on different levels, e.g. Deliverable no. D05.1 Page 5 of 6 by gating, triggers, events, or by software. In some cases, the hardware is not designed to be synchronized at all. • In general, the setups at the different LEAPS beamlines are different, depending on the local hardand software, the standards of the sample environment, the hardand software of the devices from the users. Future requirements of a synchronization system for the beamlines The aim is to enable users to perform experiments with complex synchronization needs using a generic solution. Parts of this aim has already been addressed at different LEAPS facilities, but a generic solution is not available. Here, we summarize the needs in different aspects: Time requirements • Resolution – The variability of the portfolio of actuators and sensors is very broad and different solutions with different resolution may be needed to reduce the total cost of the system. o Mechanical actuators – In the order of few milliseconds. o Motion Control – In the order of tenths of microseconds. o Pump & probe experiments and other specific triggering needs – In the order of 10 picoseconds. In case specific long-term drift auto-compensation (for example due to temperature drift on the communication channel) would be desirable. • Precision o In general, the precision needed is one order of magnitude lower than the resolution in jitter rms (if we assume Gaussian source noise). Hardware Interfaces • The list of devices that the system has to monitor or trigger is long: o Mechanical actuators o Motion Controllers o Position Encoders o Detector or camera triggers o Pump lasers and Laser tweezers o External fields or scalars o Safety hardware o May be data transfer o Other • On the other hand, it is necessary that accelerator timing system events can be inserted. Thus, the time requirements listed above can be met. These events may come from: o White Rabbit system Deliverable no. D05.1 Page 6 of 6 o MRF system o Green Field system o PETRAIV and XFEL timing system System Services • Timestamp – Generic timestamp services for all the events must be provided. The resolution of the timestamp must be at least the actuators resolution. Optionally a unique number might be distributed along the events to tag data from ADC or synchronization events. • Diagnostics of links – Automatic status of agents and physical links would be desirable to improve diagnostic. High level interfaces • User Interfaces – High level software interfaces must be provided to allow non-expert users to interact with the system. This would ideally be achieved through automatic GUIs setup that would allow: o List the different devices of the system and their inherent connection. o Be able to program triggering patterns based in complex sequences and logics. o Monitor in real-time the execution of the pattern sequence (ideally with timestamp/unique number services) o Program alarms when time requirements are not met during execution. • Automated (programmable) Interfaces – Two different automated interfaces must be built: o Timing system must be able to be programmed to provide accurate synchronization information which is useful metadata of the experiment. This metadata, in most cases, will be stored together with experimental data. o Timing System must be able to be operated through automated scripts sequences. The objective is to provide useful tools for automated optimization processes in the future (for example using Machine Learning algorithms).