D4: Good practice guide for the calibration of digital substation instrumentation using PTPv2 timing

Confidentiality Status: PU - Public, fully open (remember to deposit public deliverables in a trusted repository) Deliverable Cover Sheet Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or EURAMET. Neither the European Union nor the granting authority can be held responsible for them. The project has received funding from the European Partnership on Metrology, co-financed from the European Union’s Horizon Europe Research and Innovation Programme and by the Participating States. 1 of 40 GA 21NRM02 - Digital-IT D4 - Good practice guide for the calibration of digital substation instrumentation using PTPv2 timing VTT, VSL, RISE, Łukasiewicz-ITR Due date of the deliverable: May 2025 Actual submission date of the deliverable: October 2025 2 of 40 List of Abbreviations 1PPS One Pulse Per Second - A timing signal that marks each second for synchronization 4B/5B 4-Bit/5-Bit Encoding - Converts 4-bit data into 5-bit symbols for reliable transmission API Application Programming Interface - Enables software systems to communicate and share functions BC Boundary Clock – A networking switch which operatd as a PTP master port for downstream devices COTS Commercial of the Shelf – easily available and replaceable hardware DUC Device Under Calibration - Equipment being tested for accuracy EURAMET European Association of National Metrology Institutes – Organization that develops and disseminates a measurement infrastructure for Europe GM Grandmaster - The primary time source in a PTP network GPS Global Positioning System - Satellite-based system for distributing location and precise time IEC International Electrotechnical Commission – A commission that develops global standards for electrical technologies IEEE Institute of Electrical and Electronics Engineers – An institute that sets standards and advances technology innovation IT Instrumet Transformer – Converts high voltage and high current signals in the electricity grid to measurable values LPIT Low-Power Instrument Transformer - Measures electrical signals, either small-signal voltage output or digital SV stream output MAC Media Access Control - The layer of protocols and methods that manages how devices access a shared network medium, preventing data collisions and ensuring orderly communication MII Media Independent Interface – Interface that connects MAC interface chip and its host processor MLT-3 Multi-Level Transmit-3 - Signal encoding using three voltage levels to reduce bandwidth NRZI Non-Return-to-Zero Inverted - Data encoding using signal transitions to represent bits NTP Network Time Protocol - Synchronizes clocks across computer networks P2P Peer-to-Peer – PTP delay method, where both link partners measure the link delay PC Personal Computer - General-purpose computing device for individual use PHC PTP Hardware Clock - Physical clock used for precise time sync in PTP systems PTP Precision Time Protocol - Synchronizes clocks in networks with high accuracy PTPv2 Precision Time Protocol Version 2 - Enhanced version of PTP with better scalability and precision REF Reference device - Provides a stable standard for time or measurement calibration RJ45 Registered Jack 45 - Standard connector for Ethernet cables Rx Receive - Process of accepting data or signals SAMU Stand-Alone Merging Unit - Digitizes signals from electrical equipment in substations SFP Small Form-factor Pluggable - Compact module for physical layer network interfaces SOF Start of Frame - Indicates the beginning of a data packet in communication protocols SV Sampled Value - Digitized measurements transmitted in real-time in electrical systems TAP Traffic Access Point - Allows monitoring or testing of network traffic 21NRM02 - Digital-IT 3 of 40 TC Transparent Clock – A networking switch which forwards PTP messages while correcting for internal delay TIC Time Interval Counter - Measures time between two events with high precision Tx Transmit - Process of sending data or signals UCA IUG Utility Communications Architecture International Users Group - Promotes standards in utility communications UTC Coordinated Universal Time - Global time standard used for synchronization UTP Unshielded Twisted Pair - Common network cable type without shielding to reduce interference 21NRM02 - Digital-IT 4 of 40 TABLE OF CONTENTS List of Abbreviations ........................................................................................................................ 2 1Summary ................................................................................................................................. 5 2Introduction .............................................................................................................................. 6 3Requirements for synchronicity ................................................................................................ 7 4Traceability for PTPv2 on a digital substation ........................................................................... 7 5Relevant aspects of PTPv2 ...................................................................................................... 8 6Absolute calibration method ..................................................................................................... 9 6.1 Setup and rationale ............................................................................................................ 9 6.2 PTP event message timing and clock models .................................................................... 9 6.3 Leveraging the Peer Delay mechanism ............................................................................ 12 6.4 Hardware and software .................................................................................................... 12 6.5 Measurement results from VTT’s digital substation calibration setup................................ 15 6.6 Measurement results of various non-PTP network components ....................................... 19 6.7 Digitizer triggering using PTP messages .......................................................................... 21 7Golden calibrator pair approach ............................................................................................. 23 7.1 Extended PTP time synchronization model ...................................................................... 23 7.2 PTP device time synchronization calibration model .......................................................... 24 7.3 PTP device calibration principle ....................................................................................... 26 7.4 New tools for transmission medium latency calibration .................................................... 26 7.5 PTP “Golden Calibrator” calibration results ...................................................................... 27 8Alternate method using time tagging at PHC, RISE ................................................................ 31 9Comparison of absolute and relative calibration methods....................................................... 36 10 Conclusion and input to standardisation ................................................................................. 39 11 REFERENCES ...................................................................................................................... 40 21NRM02 - Digital-IT 5 of 40 1 Summary The EURAMET Digital-IT project set out to develop a traceable method for calibrating stand-alone merging units and digital output LPITs under IEEE 1588 Precision Time Protocol synchronicity. No such service was to date available from any national metrology institute globally. The problem has so far been the somewhat crossdisciplinary nature of the problem, the electricity grid metrologists are not timing experts, and time and frequency researchers mostly focus on time scales and frequency transfer. The grey area between the disciplines had so far been unexplored. But timing is becoming increasingly important in operating electricity grids and the need for calibrating PTP timing in the context of digital substations has become important Three complementary methods were developed within the project duration. Two methods rely on determining the PTP clock time based on the traffic it sends and receives. A third method relies on a pair of similar clocks being used back-to-back, where either clock can operate as a master for its pair. The latter method was compared against one of the traffic monitoring methods and it was found that they agree within the uncertainty budgets. A goal for uncertainty was set to be in the 100-nanosecond range and all three methods succeeded in this. The goal was set an order of magnitude lower than what was foreseen as the absolute maximum tolerable uncertainty according to the requirements set in IEC 61869-13 standard. Besides calibration of master clocks, methodology for determining timing errors for other relevant equipment is presented based on one of the traffic monitoring methods. As a “prenormative” project in the 2021 EURAMET project call, one of the outputs of Digital-IT is to develop best practices for industry and input to future amendment of relevant standards. This guide thus ends with a conclusions section including suggestions for developing standardization in testing equipment under PTP synchronicity. The findings indicated that a necessary step in traceable calibration is to isolate the test setup timing errors from the result. This can be achieved by a traceable calibration of the master clock and by ensuring a sufficiently low asymmetry in the physical transfer medium between the master clock and device under calibration. 21NRM02 - Digital-IT 6 of 40 2 Introduction The digital substation environment in accordance with IEC 61850 family of standards is rapidly gaining in popularity in substations connecting parts of the transmission and distribution grids. Within a substation, the process bus is transmitting data to communicate specific events as Sampled Values (SV) streams of time domain measurements of grid signals in accordance with IEC 61850-9-2 [1]. Phase of the signals encoded into the SV streams is related to a local time source, often aligned with the coordinated universal time (UTC). A preferred time source in the UCA IUG implementation guideline [2] for IEC 61850-9-2, the first, and widely accepted, document to suggest a conformal set of SV transmission parameters, is the one-pulse-per-second signal (1PPS) delivered via means of a dedicated optical or electrical connection. The original standard IEC 61850-9-2 was first released in 2004 (superseded by the 2011 release and 2020 amendment), with the implementation guideline published in the same year. Since then, the IEEE 1588-2008 Precision Time Protocol (PTPv2) [3] has gradually been adopted as the preferred method for time synchronization. Originally an industry preference over 1PPS, PTPv2 is now considered as the primary means of time synchronization in IEC standard for digital interface for instrument transformers IEC 61869-9 [4]. The practical reasons for adopting PTPv2 are clear. It requires no additional cabling, since the already present process bus can be used for delivering the synchronization. And some applications, such as synchrophasors require absolute UTC time, which cannot be delivered using 1PPS. While PTPv2 may have desirable properties for many applications on a digital substation, its use also significantly complicates the verification of the phase displacement performance of measurement instruments. While many national metrology institutes in the EU and worldwide have developed methods for calibrating merging units and other devices, which adopt 1PPS as a means of synchronicity, traceable methods for calibrating PTPv2 synchronized devises have not been proposed yet. Two major complications for calibrating a PTPv2 device clock can be readily identified. First, the timing information is embedded into a reasonably complicated exchange of messages between a master clock and a slave clock. And second, a slave clock’s timing is unique, meaning that no two devices can be expected to have the same concept of time even if they receive their synchronicity from the same master clock. The latter property means that the classical approach of a reference measurement is not valid anymore. One of the goals of the EURAMET 21NRM02 project “Metrology for digital substation instrumentation” was to develop methodology for calibration of merging unit phase displacement under PTPv2 synchronicity. The consortium consists of experts from both smart grids and time and frequency communities as an answer to the cross-disciplinary nature of the problem. The planned output from the project in this regard was to communicate the findings and developed methods to relevant standardization bodies, especially but not limited to IEC TC 38. This good practice guide serves as the means to this end. 21NRM02 - Digital-IT 7 of 40 3 Requirements for synchronicity The most stringent timing requirements for a phase measurement referred to an absolute time source come from the measurement class stand-alone merging unit /SAMU) specifications, which are defined in the IEC 61869-13 standard. To support connecting the most accurate class 0.1 and 0.2S instrument transformers and LPITs to the process bus, an accuracy class 0.05 for SAMU is introduced. A maximum tolerated phase error for the class for both current and voltage is 2.5 minutes, which corresponds to a timing error of 2.3 microseconds if no other error sources exist. When calibrating devices, it is often desirable for the reference setup to have an uncertainty an order of magnitude smaller than what is the desired final uncertainty of calibration. This means that the requirement for timing accuracy provided for the calibrated device should be as low as 230 nanoseconds or even lower if the uncertainty budget contains other sources with significant magnitude. Furthermore, devices (often test bridges) for calibrating SAMUs and instrument transformers under 1PPS synchronicity are already on the market and it is expected that such devices will soon support also PTP synchronicity. This introduces even more stringent requirements for synchronicity. Since the traceability chain is an additional step longer, the uncertainty in a test bridge calibration should be low enough to enable its use for calibrating class 0.05 SAMUs. For these reasons, the goal for uncertainty in PTP time synchronicity calibration was set as low as 100 nanoseconds in the Digital-IT project. 4 Traceability for PTPv2 on a digital substation A drawing illustrating the principle of traceable phase displacement calibration is shown in Figure 1. A time source is introduced, which provides both 1PPS and PTP timing referenced to its timing plane. A physical link is established between the time source and both the device under calibration (DUC) and the reference measurement setup. The link may introduce errors if not implemented and calibrated correctly. Figure 1. Principle of time synchronicity when using 1PPS and PTP in a calibration setup In the case of 1PPS synchronicity, the time source reference plane may be considered common for both the device under calibration and the reference setup. In this case, the timing difference introduced by different lengths of 1PPS signal should be accounted for if different length cables are used. In a correctly working PTP master-to-slave link, the delay is compensated by the delay mechanism. Therefore, it is important that the time source, i.e. the master clock delay mechanism is implemented correctly, and the physical link introduces no additional error. The former can be guaranteed by calibrating the master clock with any preferred method and the latter by guaranteeing sufficient symmetry in the PTP transport path. If both requirements are satisfied, any error introduced between the master and slave clocks is due to the slave clock, the DUC in this case. Additionally, the master clock’s internal time may be different from a 1PPS rising edge, even if supplied by the master itself. Thus, the internal time must be known with respect to any preferred timing reference plane. When the master clock’s delay mechanism and internal time are calibrated, and the PTP transport path has no appreciable symmetry, timing related error to DUC SV data phase displacement is due to the DUC only and may be calibrated. To address master clock calibration, three complementary methods have been developed within the Digital-IT project. VTT have developed a method, which can be used for determining the error of the time stamps generated by a PTP clock and deriving clock inaccuracies from the results. The method is based on probing on-the-wire Layer 1 signals between two clocks in a master to slave communication link. The method 21NRM02 - Digital-IT 8 of 40 presented in detail in section 0 is called the absolute calibration method. RISE have developed a similar approach, relying on Linux APIs, which support PTP hardware clocks (PHC) on Ethernet interfaces, and both allow time stamping of Ethernet frames as well as external events of 1PPS representations of a traceable timescale. RISE’s method is described in detail in section 0. In VSL’s method two PTP clocks in the masterslave hierarchy are taken as a pair of PTP clocks for calibration, and the calibration is done by setting up a PTP communication path between the PTP clocks pair accordingly. By assessing relevant PTP parameters in the link as well as the phase comparison on the 1PPS signals, total internal latencies and relative PTP time offset of the two PTP clocks in pair are calibrated. VSL’s approach is called the relative calibration method and is described in section 7 A comparison of the absolute and relative methods is presented in section 0. 5 Relevant aspects of PTPv2 Shortly after the release of the 2008 version of the precision time protocol, IEEE Power and Energy Society released a standardized set of rules in IEEE C37.238-2011 [5] for using PTPv2 for synchronizing devices. This document is commonly known as the “Power profile” for PTPv2. A common set of rules for implementing PTPv2 and the mandatory requirement for its support in IEC TC 38 release of IEC 61869-9 [4], which standardized the sampled values output of IEDs for various applications meant that the precision time protocol became the primary means for time synchronization. The IEEE Power Profile further evolved into a dual logo release of the IEC/IEEE 61850-9-3:2016 known as the “Utility Profile” and to specific requirements for the North American market defined in IEEE C37.238-2017 [6], [7], [8]. What is most important in the standard profiles for this work are the definitions of some key aspects of using PTPv2 on a substation. While the method developed at VSL is not dependent on the specifics of a PTP setup, certain aspects are leveraged in other solutions developed in the Digital-IT project. Most importantly, the following specifications are of significance: 1. Switches are allowed to be configured as either transparent or boundary clocks. While both types have their pros and cons, a boundary clock is preferred in a calibration setup. Using a boundary clock greatly simplifies the Sync mechanism messages delivered to a slave-only device, since no residence time correction is required and a high level of Rx/Tx path symmetry between the slave and master ports can be guaranteed. 2. One-step and two-step devices are allowed. Two-step clocks are often preferred, since its use simplifies the process updating relevant fields in the PTP messages used in sync and delay mechanisms. 3. Peer delay mechanism for path delay measurement is the only allowed method. The associated Pdelay_Req, Pdelay_Resp, andPdelay_Resp_Follow_Up messages available from both link partners carry all the necessary information to assess all critical parameters of the PTP ports. Furthermore, when using two-step devices, the rules state that requestReceiptTimestamp and responseOriginTimestamp fields should be populated in the respective response messages. This makes calculating the ingress and egress timestamp errors straightforward. 4. Layer 2 transport with a predefined Ethertype field. Locating and decoding PTP traffic on the wire is simpler, when the messages can be identified as PTPv2 traffic based on the Ethertype field in the Layer 2 frame. in Layer 3 traffic the Ethertype field would be used for indicating transmission of IPv4 packets. 21NRM02 - Digital-IT 9 of 40 6 Absolute calibration method 6.1 Setup and rationale VTT’s setup for calibrating devices, which produce sampled values data and rely in PTPv2 for time synchronicity is shown in Figure 2. The figure shows relevant devices, which deliver synchronicity between the devices. The Grandmaster (GM) clock is synchronized using GPS and along with PTPv2 sync, has also an output for a 1PPS signal used by the reference device (REF). The boundary clock, which is synchronized by the Grandmaster in turn synchronizes its downstream slave lock, the device under calibration (DUC). The GM and the boundary clock switch are set up in accordance with the Utility Profile. The SV data produced by the devices is transferred to a PC (not drawn) though the same switch. Voltage and current test signal connections are omitted in the figure for simplicity. In this regard, the setup works as any other setup for comparing readings from a device under calibration to a calibrated reference device: the same signals are measured by both devices. Figure 2. Timing and data connections of the measurement setup for calibrating devices that produce SV data. In Figure 2 the thick lines show the definition of timing reference planes of the relevant devices in the setup. For the device under calibration, this is the boundary between the upstream master clock and the network, i.e. the physical network connector. For the reference device, the timing reference plane is the 1PPS input connector, against which its phase displacement is calibrated. To calibrate the phase displacement of the device under calibration, one needs to know what the difference in the synchronicity received by both devices is. The rationale for defining this is that the error of the DUC upstream master clock (the boundary clock switch) must be known so that it can be accounted for. This includes determining the internal time of the master clock as well as its ingress/egress asymmetry. Additionally, the network cabling between the switch and the DUC must not introduce additional asymmetry. Any phase displacement in slave device SV stream can then be attributed to the slave only, while the upstream master port error can be accounted for in the result. 6.2 PTP event message timing and clock models The error of a PTP clock can be quantified by determining how its internal time stamping is performing with respect to the delays it needs to compensate for. Looking into the original PTPv2 standard IEEE 1588-2008 [3], three clauses are of significance: Clause 6.6.5 Generation of message timestamps: A timestamp event is generated at the time of transmission and reception of any event message. The timestamp event occurs when the message’s timestamp point crosses the boundary between the node and the network. Therefore, determining the time of transmission or reception of messages with respect to an external time reference plane and the time the PTPv2 instance claims as the transmission or reception time, enables determining the time stamping error1 with respect to the timing reference plane. Clause 7.3.4.1 Event message timestamp point: Unless otherwise specified in a transport-specific annex to this standard, the message timestamp point for an event message shall be the beginning of the first symbol after the Start of Frame (SOF) delimiter. This means that the timestamp point in the PTPv2 messages is the first bit of the destination MAC address at the beginning of a Layer 2 frame. Figure 3 illustrates this location in a 100Base-X ethernet connection, where 1 Since the reference plane is somewhat arbitrarily defined as the reference device 1PPS input, the use of “difference” instead of “error” of any PTPv2 time stamp would be more appropriate. 21NRM02 - Digital-IT 16 of 40 Figure 11. Measurement setup for calibrating a boundary clock’s master port and slave clock’s internal time and residual asymmetry. Table 2. Clock egress and ingress errors, internal times, and residual asymmetries. Property Master (switch) Slave (DUC) Egress error (96.2 ± 1.7) ns (-11.5 ± 0.1) µs Ingress error (97.7 ± 3.8) ns (11.6 ± 0.2) µs Internal time vs. 1PPS (97.0 ± 4.2) ns (48 ± 228) ns Residual asymmetry (-0.7 ± 4.2) ns (-11.5 ± 0.2) µs The data in Table 2 shows that the switch master port internal error is quite high, although it’s only one hop away from the grandmaster, which supplies also the 1PPS signal. Slave clock time reflects this, although uncertainty is quite high due to large deviation2 between individual measurements. Grandmaster error vs. its own 1PPS output Master port error may be further investigated by measuring the time inside its upstream master. The setup to do this is shown in Figure 12. The switch slave port is synchronized by the Grandmaster clock in the setup. The clocks are connected using a 100Base-TX link. A passive probe in Figure 7 (a) is connected between the devices and data is analyzed. Unfortunately, the Rx traffic into the Grandmaster is impossible to decode from the digitizer capture due to excessive ringing and the MLT-3 encoded signal settling at constantly varying levels between 1’s in the data. Thus, Grandmaster internal time and asymmetry are not possible to be measured. However, egress error of the Grandmaster is available and can be determined from the Sync and Follow_Up message pair. Time instance te is the Sync message SOF delimiter egress time and te’ is the preciseOriginTimestamp in the Follow_Up message. Figure 13 shows the egress error data collected in the measurement. Interestingly, a quantization of 10 ms for Sync message egress time can be observed in the data. The 1.6 ns quantization in error values is due to sampling frequency of 625 MHz in the digitizer used for capturing the physical layer signals. While not exactly comparable to the previous measurement of the switch master port internal time, the Grandmaster egress error of (97.9 ± 0.8) ns appears to account for most of the time error in the upstream devices. 2 The likely cause for this is a software-only implementation of PTP in the slave, with no hardware-assisted timestamping. In this case, any variable delay in servicing interrupts will cause jitter in time stamping PTP event message egress and ingress times. 21NRM02 - Digital-IT 17 of 40 Figure 12. Measurement setup for testing grandmaster’s timing error against its own 1PPS output. Figure 13. Grandmaster egress error vs. 1PPS against Sync message egress time. DUC phase displacement calibration under PTPv2 synchronicity After calibrating the upstream master port in Figure 12, it’s error may be subtracted from the calibration result. The DUC was calibrated using an SV data rate of 4000 Hz using the setup in Figure 2. As per the example in section 6.2 on page 9, the master clock error tm is compensated by subtracting its influence from the reading ɛ’φ,DUC using the formula 𝜀𝜑,𝐷𝑈𝐶=𝜀′𝜑,𝐷𝑈𝐶−2𝜋𝑓𝑡𝑚 , where ɛφ,DUC is the corrected calibration result and f is the calibration frequency. The phase calibration results, including various uncertainty contributions are given in Table 3. It can be seen that for the smallest uncertainties, the PTP vs. 1PPS calibration uncertainty becomes a clear contributor to the overall calibration uncertainty. The roughly 100 ns error in switch master port time in VTT’s setup becomes a relevant factor when trying to achieve the best possible uncertainty. An error of 100 ns would result in an error of 1.8 mdeg, which is almost the same as the lowest achieved uncertainty, when using input values of 1 A, 5 V, and 100 V. This underlines the need for calibrating the timing network prior to calibrating merging units or other SV enabled equipment. 21NRM02 - Digital-IT 18 of 40 Table 3. Calibration result of a Stand-alone merging unit using VTT’s calibration setup. Test conditions Error of input [mdeg] Uncertainty contributors [mdeg] Uncertainty Current inputs IA IB IC IN Deviation PTP v. 1PPS Setup [mdeg] k = 2 1 A, 50 Hz 70 73 45 62 1.1 0.3 0.3 2 0.05 A, 50 Hz 73 71 49 95 10 0.3 0.3 21 0.025 A, 50 Hz 88 137 99 -4 36 0.3 0.3 72 Voltage inputs VA VB VC VN 100 V, 50 Hz 10.9 17.7 21.9 20.1 0.3 0.3 0.2 1.0 5 V, 50 Hz 815 19 17 0.9 0.3 0.2 2 The timing network influence may be further investigated by introducing an artificial error to the time the DUC receives. One way to achieve this is to add asymmetry in the link between the upstream PTP master clock and the DUC. This is shown in Figure 14, where additional fiber is added either in the master clock egress or ingress path. Using the definitions in Figure 5, the sign of the resulting asymmetry is positive when the extra fiber is added to the master egress (Tx) port. A positive asymmetry is drawn as a red fiber spool in the Figure 14 and a negative one as a blue spool. The SV stream of the DUC is recorded and phase displacement of one of the current signals is calculated for each asymmetry configuration. The measured input signal in the test is phase locked to 1PPS and no reference measurement is performed. Thus, the phase displacement result is the observed difference from the no-asymmetry condition. The measurement results are given in Table 4. The uncertainties are mostly dominated by deviation of readings. Figure 14. Introducing errors by adding asymmetries in the timing network Table 4. Measured values of DUC slave clock time and relative phase displacement vs. link asymmetry. Asymmetry DUC slave clock time [ns] DUC SV stream phase displacement [ns] value [ns] uncertainty [ns] k = 2 value [mdeg] uncertainty [mdeg] k = 2 value [ns] uncertainty [ns] k = 2 315 -373 320 4.83 1.21 268 67 215 -185 130 4.70 3.54 261 197 100 -204 646 3.77 3.56 210 163 042 335 02.76 0153 -100 49 370 -1.96 3.56 -109 198 -215 311 128 -6.36 3.35 -353 186 -315 295 126 -6.71 1.48 -373 82 21NRM02 - Digital-IT 19 of 40 Figure 15 shows plots of DUC internal time and SV stream phase displacement vs. link asymmetry. In each case a first order polynomial fitted to the measurement data using weighted fitting, where weights are the reciprocals of squared measurement uncertainties. Unsurprisingly, a positive asymmetry causes a negative time error of the same magnitude in the slave clock, which in turn causes the slave device to timestamp SV data too early, causing a positive (leading) phase error. The slope of the phase error vs. asymmetry plot is 0.0188 m°/ns, reasonably close to a theoretical ratio of 2𝜋∗50𝐻𝑧∗180° 𝜋=0.018𝑚° 𝑛𝑠. Figure 15. DUC internal time and phase displacement measurement results vs. link asymmetry. 6.6 Measurement results of various non-PTP network components Other networking components, which are not used in VTT’s calibration setup were tested in the project for their residence time. The tests include PTP and non-PTP compliant devices: 1. Transparent clock residence time calibration. Using the absolute probing method, it is possible to calibrate a transparent clock for the error it introduces due to miscalculating the data in the correction field associated with PTPv2 event messages passing through it. Another switch is added to the measurement setup to facilitate the test due to lack of TC support in the original switch. 2. Non-PTP compliant networking components’ residence time. It may be desirable to know the residence time of PTP or any other message in networking equipment. Such devices include active network taps and media converters, which were tested using the probing setup. Transparent clock residence time calibration The setup can be used for determining an error introduced by a transparent clock miscalculating the residence time, which it is supposed to report as an updated value of the correction field in an event message, or in a follow-up message if a two-step mechanism for relaying event data is used. In this case the Sync and Follow_Up message pair is used. A second switch configured as a transparent clock is inserted in between the Grandmaster and boundary clocks and 100Base-TX TAPs are installed in both connections as in Figure 16 below. 1PPS signal is used for triggering the measurement but is not necessary for determining errors in the data due to the differential nature of the test. 21NRM02 - Digital-IT 20 of 40 Figure 16. Measurement setup for testing a transparent clock for its residence time correction. The ingress and egress time of a Sync message to and from the transparent clock is read using the digitizer and the corresponding Follow_Up message at the ingress and egress ports is decoded to read the change in correction field value. The residence time correction error ∆tTC is then calculated from ∆𝑡𝑇𝐶=𝑡𝑐𝑜𝑟𝑟−𝑡𝑇𝐶,𝑒−𝑡𝑇𝐶,𝑖, where tcorr is the updated correction field value read from the Follow_Up message, and tTC,e and tTC,i are respectively the Sync message egress and ingress times. The measured error is (-11.8 ± 2.4) ns as shown in Figure 17, where the measured average residence time of (1.05 ± 0.04) ms is subtracted from individual measured residence times, thus producing a plot centered around zero on the horizontal axis. The measurement is done using one concurrent SV stream in the switch. The switch seems to perform well considering that IEC 61850-9-3 allows for a transparent clock to introduce a maximum error of 100 ns. A higher traffic load may still have an impact on the time stamping quality. Figure 17. Transparent clock residence time correction error. Non-PTP compliant networking components’ residence time Similarly to transparent clocks, residence time of various other network components may be determined. With non-PTP compliant devices that do not update the correction field in relevant PTP messages, the desirable function is to have as little of delay jitter as possible and no appreciable asymmetry. This way, the delay may be subtracted from any tests done using a probing method and the PTP delay mechanism has a chance to compensate for network latency correctly. Any packet may be used for this purpose, but since VTT’s ethernet probe is sensitive to PTP messages only, filtering out the rest, a stream of Sync and Follow_Up messages from a master to slave is used with the tested component inserted in between. An active traffic access point in non-aggregate mode and a 100Base-TX/100Base-FX media converter were tested for all relevant delays. The measurement setup is shown in Figure 18 with the DUT inserted in between the two clocks. The results are shown respectively for the TAP and media converter in Table5 and Table 6. Uncertainty numbers include both the type B uncertainty 21NRM02 - Digital-IT 21 of 40 of the measurement setup and the standard deviation of the measurement data. Neither tested device introduces appreciable asymmetry into the link. The active network TAP has a reasonably high residence time and some deviation, but no asymmetry can be seen in the data. Figure 18. Test setup of non-PTP compliant networking devices Table 5. Residence time of an active network traffic access point from port to port Ingress port Egress port Residence time [ns] Port A Port B 7954 ± 35 Port B Port A 7959 ± 34 PortA Monitor Port 1 7955 ± 31 Port B Monitor Port 2 7951 ± 34 Table 6. Residence time of a media converter Ingress port Egress port Residence time [ns] 100Base-TX 100Base-FX 42 ± 1 100Base-FX 100Base-TX 57 ± 1 6.7 Digitizer triggering using PTP messages The method of acquiring data contained in Ethernet frames of the PTP protocol and determining the timestamp for the SOF bit, described in section 7.4, is based on the use of a high-frequency (625 MHz) sampling device. For the typical PTP time update interval of 1 second, a one-second data buffer must also be maintained in the digitizer to ensure full capture of the PTP frame exchange transaction. If this is not possible due to hardware limitations of the digitizer, the method can be extended by using a trigger device to activate the digitizer at the moment PTP frames appear on the Ethernet link. This minimizes the number of samples that need to be recorded. Additionally, an extra time interval counter (TIC) is required to calculate the time of the SOF bit occurrence relative to the 1PPS signal. An example setup is shown in Figure 19. The trigger device may monitor network traffic via an additional TAP or via a mirror port on a switch. 21NRM02 - Digital-IT 22 of 40 Figure 19. Example of setup with additional triggering device. The trigger device can be built using e.g. the DP83640 Ethernet PHY integrated circuit and a host processor. This device can generate a signal on its output when an incoming PTP Ethernet frame is detected. The delay of this signal relative to the SOF bit is 7.5 µs with a standard deviation of 50 ns. Interrupt clearing by the host processor is performed via software by writing to a register in the chip and takes a maximum of 40 ms. The chip also supports hardware generation of a pulse precisely when the SOF bit appears. This allows for measuring the time difference between the trigger signal (indicating receipt of the PTP frame) and the SOF bit signal. 21NRM02 - Digital-IT 23 of 40 7 Golden calibrator pair approach 7.1 Extended PTP time synchronization model The PTP synchronization model shown in [3] only concludes the ideal scenario in time synchronization, where the delays in the bi-directional link are symmetric and the clocks in master-slave hierarchy are synchronized. However, link asymmetry and clock offset are almost inevitable in realization. Therefore, the ideal time synchronization model needs to be extended to a general model which includes the real scenario of link asymmetry and clock offset. Figure 20. P2P time synchronization model Figure 20 describes the extended timing exchange mechanism based on Peer-to-Peer (P2P) delay mechanism, which is used by Power Utility Profile IEC 61850-9-3. In this time synchronization model, Node-A and Node-B are synchronized via P2P mechanism, where Node-A serves as the master clock and Node-B is the slave clock. A reference time is used as the common reference of Node-A and Node-B clock. Assume Node-A clock follows the reference time, marked as 𝑡𝑀=t, and Node-B clock has a frequency and phase offset compared to Node-A, marked as 𝑡𝑆=𝜔𝑠t+𝜑0𝑠. The moment that 𝑡1 is sent is labelled as 𝑡𝑅𝑒𝑞 in the reference time, and the moment that 𝑡3 is sent is labelled as 𝑡𝑅𝑒𝑠𝑝 in the reference time. The propagation time of the timing message from Node-A to Node-B is 𝜏𝑀𝑆 measured by the reference time, and the propagation time from Node-B to Node-A is 𝜏𝑆𝑀 when measured by the reference time. Then we can deduced the relation between the local time of Node-A and Node-B and the reference time as: 󰇫𝑡1=𝑡𝑅𝑒𝑞 𝑡2=𝜔𝑠𝑡𝑅𝑒𝑞+𝜏𝑀𝑆+𝜑0𝑠 (1) 𝑡3=𝜔𝑠𝑡𝑅𝑒𝑠𝑝+𝜑0𝑠 𝑡4=𝑡𝑅𝑒𝑠𝑝+𝜏𝑆𝑀 (2) Accordingly, in the syntonization case where 𝜔𝑠≅1, the message transmission time interval object 〈𝑚𝑒𝑎𝑛𝑃𝑎𝑡ℎ𝐷𝑒𝑙𝑎𝑦〉 and the clock time error object 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉 can be derived as: (𝑡2−𝑡1)=(𝜔𝑠−1)𝑡𝑅𝑒𝑞+𝜔𝑠𝜏𝑀𝑆+𝜑0𝑠 =𝜏𝑀𝑆+𝜑0𝑠 (𝐿𝑒𝑡𝜔𝑠≅1) (3) (𝑡4−𝑡3)=−(𝜔𝑠−1)𝑡𝑅𝑒𝑠𝑝+𝜏𝑆𝑀−𝜑0𝑠 =𝜏𝑆𝑀−𝜑0𝑠 (𝐿𝑒𝑡𝜔𝑠≅1) (4) 〈𝑚𝑒𝑎𝑛𝑃𝑎𝑡ℎ𝐷𝑒𝑙𝑎𝑦〉=(𝑡2−𝑡1)+(𝑡4−𝑡3) 2 Node-A Requestor time Node-B Responder time t1 t2 t3 t4 Reference Time tτMS τSM tReq tResp tM=t tS=ωst+φ0s 21NRM02 - Digital-IT 24 of 40 =(𝜔𝑠−1)𝑡𝑅𝑒𝑞−𝑡𝑅𝑒𝑠𝑝+(𝜔𝑠𝜏𝑀𝑆+𝜑0𝑠)+(𝜏𝑆𝑀−𝜑0𝑠) 2 ≅(𝜏𝑀𝑆+𝜑0𝑠)+(𝜏𝑆𝑀−𝜑0𝑠) 2(𝐿𝑒𝑡𝜔𝑠=1) (5) 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉=(𝑡2−𝑡1)−(𝑡4−𝑡3) 2 =𝜑0𝑠+(𝜔𝑠−1)𝑡𝑅𝑒𝑞+𝑡𝑅𝑒𝑠𝑝+(𝜔𝑠𝜏𝑀𝑆+𝜑0𝑠)−(𝜏𝑆𝑀−𝜑0𝑠) 2 ≅(𝜏𝑀𝑆+𝜑0𝑠)−(𝜏𝑆𝑀−𝜑0𝑠) 2(𝐿𝑒𝑡𝜔𝑠=1) (6) According to PTPv2 protocol [3], Node B will be synchronized to Node A by forcing 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉=0 with the assumption of 𝜏𝑀𝑆=𝜏𝑆𝑀. In an ideal symmetric link where 𝜏𝑀𝑆=𝜏𝑆𝑀,𝜑0𝑠=0 when forcing 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉=0. While in an asymmetric link where 𝜏𝑀𝑆≠𝜏𝑆𝑀, forcing 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉=0 leads to 𝜑0𝑠=−𝜏𝑀𝑆−𝜏𝑆𝑀 2≠0. In summary, link asymmetry will cause an extra phase offset between a master clock and a slave clock under PTPv2 protocol. 7.2 PTP device time synchronization calibration model The calibration model for PTP devices is shown in Figure 21. PTP devices are calibrated in pairs, where the pair of PTP devices is in master-slave hierarchy. Taken the internal structure and function of a PTP device into consideration, a PTP device is divided into two units: Timing-message unit and Clock-servo unit. The timing-message unit implements the ingress and egress of PTP messages between the PTP port to the internal time-stamp clock, and the clock-servo unit corresponds to the RF signal conversion from the time-stamp clock to the pulse port. Here the time-stamp clock is defined as the internal clock of a PTP device which marks timestamps for PTP timing packets. Its clock time is referred to the moment when a timestamp is generated. Furthermore, ingress and egress latencies in Timing-message unit as well as the electrical latencies in Clockservo unit are defined specifically as follows: Δ𝑇𝑋𝑀𝑃𝑇𝑃,Δ𝑇𝑋𝑆𝑃𝑇𝑃: PTP timing packet propagation delay from the moment when it is timestamped to the moment when the transmitted packet reaches to the PTP port in a master or slave device, respectively. Δ𝑅𝑋𝑀𝑃𝑇𝑃,Δ𝑅𝑋𝑆𝑃𝑇𝑃: PTP timing packet propagation delay from the moment when it reaches the PTP port to the moment it is timestamped in the clock-servo unit in a master or slave device, respectively. Δ𝑃𝑈𝐿𝑆𝐸𝑀𝑃𝑇𝑃2𝑂𝑈𝑇,Δ𝑃𝑈𝐿𝑆𝐸𝑆𝑃𝑇𝑃2𝑂𝑈𝑇: electrical delay between the output of the time-stamp clock and the pulse output port in a master or slave device, respectively. Δ𝑃𝑈𝐿𝑆𝐸𝑀𝐼𝑁2𝑂𝑈𝑇: electrical delay from reference PPS input port to the output of the time-stamp clock in a master device. Figure 21. PTP devices calibration model In the calibration model of Figure 21, the syntonization case is assumed, that the slave clock is syntonized with the master clock, leaving a constant phase offset of 𝜑0𝑠 with respect to the master clock. [PTPv2 packets] PTP-MASTER device PTP Port (MASTER) PULSE OUT PTP-SLAVE device PTP Port (SLAVE) PULSE OUT tPULSEM tPULSES Timing-message unit ΔTXMPTP ΔRXMPTP Clock-servo unit ΔPULSEMIN2PTP Forward Link Reverse Link δSM ΔRXSPTP ΔTXSPTP Clock-servo unit ΔPULSESPTP2OUT Timing-message unit PPS IN tREF-PPS [ t1 ] [ t4 ] ΔPULSEMPTP2OUT [ t2 ] [ t3 ] δMS Pulse Port (MASTER) Pulse Port (SLAVE) t M =t time-stamp clock t S =t+ φ 0 S time-stamp clock 21NRM02 - Digital-IT 25 of 40 Apply the calibration model to the extended PTP synchronization model described by equation (5) and (6), the relation among each items can be deduced as follows: In a direct PTP link between a master PTP device and a slave PTP device, define the total PTP device latencies in forward link and reverse link as ∆𝑋𝑀𝑆𝑃𝑇𝑃≝Δ𝑇𝑋𝑀𝑃𝑇𝑃+ΔRXS𝑃𝑇𝑃,∆𝑋𝑆𝑀𝑃𝑇𝑃≝Δ𝑇𝑋𝑆𝑃𝑇𝑃+ΔRXM𝑃𝑇𝑃, respectively. And the total PTP link latencies in bi-direction are defined as 𝜏𝑀𝑆=∆𝑋𝑀𝑆𝑃𝑇𝑃+𝛿𝑀𝑆,𝜏𝑆𝑀= ∆𝑋𝑆𝑀𝑃𝑇𝑃+𝛿𝑆𝑀, where 𝛿𝑀𝑆 and 𝛿𝑆𝑀 are the latencies from the transmission medium of the PTP link, for example, optical fibres or UTP cables, via forward and reverse links respectively. Accordingly, in the case of a direct connection between a pair of PTP devices, equation (3) and (4) can be explained as: (𝑡2−𝑡1)=∆𝑋𝑀𝑆𝑃𝑇𝑃+𝛿𝑀𝑆+𝜑0𝑠 (𝑡4−𝑡3)=∆𝑋𝑆𝑀𝑃𝑇𝑃+𝛿𝑆𝑀−𝜑0𝑠 (7) Similarly, equation (5) and (6) can be elaborated in this scenario as: 〈𝑚𝑒𝑎𝑛𝑃𝑎𝑡ℎ𝐷𝑒𝑙𝑎𝑦〉𝑚𝑒𝑎𝑠=(𝑡2−𝑡1)+(𝑡4−𝑡3) 2 =(∆𝑋𝑀𝑆𝑃𝑇𝑃+𝜑0𝑠)+(∆𝑋𝑆𝑀𝑃𝑇𝑃−𝜑0𝑠) 2+𝛿𝑀𝑆+𝛿𝑆𝑀 2(8) 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉𝑚𝑒𝑎𝑠=(𝑡2−𝑡1)−(𝑡4−𝑡3) 2 =(∆𝑋𝑀𝑆𝑃𝑇𝑃+𝜑0𝑠)−(∆𝑋𝑆𝑀𝑃𝑇𝑃−𝜑0𝑠) 2+𝛿𝑀𝑆−𝛿𝑆𝑀 2(9) where the subscript 〈⋅〉𝑚𝑒𝑎𝑠 labels the measurement results. Therefore, the time synchronization criteria of PTPv2 protocol can be elaborated in the proposed PTP calibration model as: let 〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉𝑚𝑒𝑎𝑠=0 by assuming ∆𝑋𝑀𝑆𝑃𝑇𝑃+𝛿𝑀𝑆=∆𝑋𝑆𝑀𝑃𝑇𝑃+𝛿𝑆𝑀. On the other hand, the asymmetry in the PTP link leads to an extra phase offset of the slave clock. To investigate the source of the phase offset, 𝜑0𝑠 can be further categorized into two parts: Δ𝑋𝜑, the offset due to the asymmetry from PTP device and Δ𝜑𝑙𝑖𝑛𝑘, the offset results from the asymmetry from PTP transmission link. Define 𝜑0𝑠≜Δ𝑋𝜑+Δ𝜑𝑙𝑖𝑛𝑘, and then equation (9) can be decomposed into: Δ𝑋𝜑=〈𝑜𝑓𝑓𝑠𝑒𝑡𝐹𝑟𝑜𝑚𝑀𝑎𝑠𝑡𝑒𝑟〉𝑚𝑒𝑎𝑠−∆𝑋𝑀𝑆𝑃𝑇𝑃−∆𝑋𝑆𝑀𝑃𝑇𝑃 2 Δ𝜑𝑙𝑖𝑛𝑘=−𝛿𝑀𝑆−𝛿𝑆𝑀 2(10) where Δδ𝑙𝑖𝑛𝑘 is the link-dependent phase offset of the slave clock, and Δ𝑋𝜑 is device-dependent phase offset of the slave clock. In addition to PTP messages, pulse signal measurements has the relation of: 𝑡𝑃𝑈𝐿𝑆𝐸𝑀=𝑡+Δ𝑃𝑈𝐿𝑆𝐸𝑀𝑃𝑇𝑃2𝑂𝑈𝑇 𝑡𝑃𝑈𝐿𝑆𝐸𝑆=𝑡+𝜑0𝑠+Δ𝑃𝑈𝐿𝑆𝐸𝑆𝑃𝑇𝑃2𝑂𝑈𝑇 𝑡𝑅𝐸𝐹−𝑃𝑃𝑆=𝑡−Δ𝑃𝑈𝐿𝑆𝐸𝑀𝐼𝑁2𝑃𝑇𝑃 (11) Then the phase difference between two pulse signals, which could be measured by a Time Interval Counter (TIC), for example, can be expressed as: 〈Δ𝑇𝐼𝐶𝑀𝑆〉𝑚𝑒𝑎𝑠=𝑡𝑃𝑈𝐿𝑆𝐸𝑀−𝑡𝑃𝑈𝐿𝑆𝐸𝑆 =(Δ𝑃𝑈𝐿𝑆𝐸𝑀𝑃𝑇𝑃2𝑂𝑈𝑇−Δ𝑃𝑈𝐿𝑆𝐸𝑆𝑃𝑇𝑃2𝑂𝑈𝑇)−𝜑0𝑠 ≝Δ𝑃𝑈𝐿𝑆𝐸𝑀𝑆𝑃𝑇𝑃2𝑂𝑈𝑇−𝜑0𝑠 (12) 〈Δ𝑇𝐼𝐶𝑅𝐸𝐹2𝑀〉𝑚𝑒𝑎𝑠=𝑡𝑅𝐸𝐹−𝑃𝑃𝑆−𝑡𝑃𝑈𝐿𝑆𝐸𝑀 =−Δ𝑃𝑈𝐿𝑆𝐸𝑀𝐼𝑁2𝑃𝑇𝑃−Δ𝑃𝑈𝐿𝑆𝐸𝑀𝑃𝑇𝑃2𝑂𝑈𝑇 ≝Δ𝑃𝑈𝐿𝑆𝐸𝑀𝐼𝑁2𝑂𝑈𝑇 (13) 21NRM02 - Digital-IT 32 of 40 on (HWTSTAMP_TX_ON) Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) all (HWTSTAMP_FILTER_ALL) ptpv1-l4-sync (HWTSTAMP_FILTER_PTP_V1_L4_SYNC) ptpv1-l4-delay-req (HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ) ptpv2-l4-event (HWTSTAMP_FILTER_PTP_V2_L4_EVENT) … ptpv2-event (HWTSTAMP_FILTER_PTP_V2_EVENT) ptpv2-sync (HWTSTAMP_FILTER_PTP_V2_SYNC) ptpv2-delay-req (HWTSTAMP_FILTER_PTP_V2_DELAY_REQ) For an interface x, the PHC is found in the device tree /dev/ptpx and exposes its capabilities through the PTP hardware clock infrastructure for Linux [9]. Several user tools exist that allow to inspect and interact with the clocks in the system, such as phc_ctl from the linuxptp software suite [10]: phc_ctl eno1 caps phc_ctl[5797288.125]: capabilities: 62499999 maximum frequency adjustment (ppb) 0 programable alarms 2 external time stamp channels 2 programmable periodic signals 4 configurable input/output pins has pulse per second support has cross timestamping support doesn't have adjust phase support The above example shows the PHC capabilities of a motherboard-integrated Intel I225-LM. The PHC has external time stamp channels and allows to output periodic signals on the configurable pins. Those pins may not always be accessible on the hardware, especially if the solution is integrated as in the example above. If accessible, the PHC can measure external pulses, such as a reference 1 PPS signals. Similarly, can 1PPS be generated that represent the PHC phase and establish traceability to an external timescale. This situation is depicted in the left part of Figure 28. The reference delay and the total internal delay 1PPSin-1PPSout, i.e. between input reference pulse and its time stamping representation, must be established by using a counter and evaluating the time stamps created by the external events of the time reference. On typical I225/6 expansion cards the SPD pin header can be wired to establish a reference plane for such calibration. Typical values for the internal delay are about 45 ns and need to be estimated for every setup. The inability to determine the time stamp delay with respect to an external reference poses one of the largest uncertainties with this method. If the input and output delays are considered symmetrical, the uncertainty for interfacing external reference is thus in the order of 22.5 ns. As at least two interfaces are needed to capture all relevant packets, the respective PHCs need to be synchronized, or the phase differences needs to be measured and corrected for. If synchronization is preferred a PHC can be synced to an external 1PPS reference by steering the PHC using the EXTTS measurements. Depending on the quality of the reference and the NIC local oscillatory, synchronizing can be done with a jitter of a few nanoseconds. ts2phc -f ts2phc.conf ts2phc[92120.552]: /dev/ptp0 offset 199 s0 freq +7328 ts2phc[92121.552]: /dev/ptp0 offset 199 s2 freq +7328 ts2phc[92122.552]: /dev/ptp0 offset 206 s2 freq +7534 ts2phc[92123.552]: /dev/ptp0 offset 0 s2 freq +7390 ts2phc[92124.552]: /dev/ptp0 offset -62 s2 freq +7328 ts2phc[92125.552]: /dev/ptp0 offset -61 s2 freq +7310 ts2phc[92126.552]: /dev/ptp0 offset -44 s2 freq +7309 ts2phc[92127.552]: /dev/ptp0 offset -24 s2 freq +7316 ts2phc[92128.552]: /dev/ptp0 offset -6 s2 freq +7326 ts2phc[92129.552]: /dev/ptp0 offset -4 s2 freq +7327 ts2phc[92130.552]: /dev/ptp0 offset -3 s2 freq +7326 ts2phc[92131.552]: /dev/ptp0 offset -1 s2 freq +7328 ts2phc[92132.552]: /dev/ptp0 offset -1 s2 freq +7327 ts2phc[92133.552]: /dev/ptp0 offset 0 s2 freq +7328 21NRM02 - Digital-IT 33 of 40 Common practice is to use several interfaces with EXTTS time input tied to the same reference creating a common clock. The use of shared PHCs on network cards with several interfaces (e.g. Intel E810-xxvda2) is a convenient method yielding low uncertainties of time tagging the tapped communication. Figure 28 however suggests using a single interface for synchronization and separate interfaces for capturing data. This is a more realistic setup for use with COTS server hardware that integrate interfaces, or the use of compact units with a single PCIe expansion slot for field calibrations. In such a case the PHC synchronization within the same system requires internal measurements. Traditional methods apply repetitive polls of the PHCs versus the system clock to establish statistics that minimize the measurement error, which can be substantial and may not be appropriate for the purpose of PTP calibrations. PCIe Precision Time Measurements is an PCIe standard (PTM [13]) increasingly found on modern Intel based system which implements an efficient time measurement across the PCI Express bus. Using PTM, PHC and system clocks can be measured with low jitter (e.g. Intel i226, Xeon gen4+) and synchronization can be established across all the PHCs on a system. Typical timing resolution is a few nanoseconds limiting the measurements and a consequent synchronization for which the phc2sys tool can be used. Steady state synchronization jitter is usually below 10 ns. phc2sys -s CLOCK_REALTIME -c enp1s0 -O 0 -m -w phc2sys[155.859]: enp1s0 sys offset 5246 s2 freq +11979 delay 0 phc2sys[156.859]: enp1s0 sys offset 2977 s2 freq +11283 delay 0 phc2sys[157.860]: enp1s0 sys offset 1371 s2 freq +10570 delay 0 phc2sys[158.860]: enp1s0 sys offset 505 s2 freq +10116 delay 0 phc2sys[159.861]: enp1s0 sys offset 70 s2 freq +9832 delay 0 phc2sys[160.861]: enp1s0 sys offset -51 s2 freq +9732 delay 0 phc2sys[161.862]: enp1s0 sys offset -101 s2 freq +9667 delay 0 phc2sys[162.862]: enp1s0 sys offset -76 s2 freq +9662 delay 0 phc2sys[163.863]: enp1s0 sys offset -47 s2 freq +9668 delay 0 phc2sys[164.863]: enp1s0 sys offset -15 s2 freq +9686 delay 0 phc2sys[165.864]: enp1s0 sys offset -30 s2 freq +9666 delay 0 phc2sys[166.864]: enp1s0 sys offset 2 s2 freq +9689 delay 0 phc2sys[167.865]: enp1s0 sys offset -9 s2 freq +9679 delay 0 phc2sys[168.865]: enp1s0 sys offset 8 s2 freq +9693 delay 0 phc2sys[169.866]: enp1s0 sys offset -1 s2 freq +9687 delay 0 It is however not always necessary nor desirable to synchronize clocks. Good practice in time and frequency metrology prefers to measure and track clock differences instead of steering clocks. Any clock preserves its inherent stability if left free running. However, the clocks implemented on network devices often have questionable properties that not always can be exploited, thus rapid steering is a valid option. Table 11 lists timing related properties of a few common, both legacy and available, both suitable and unsuitable, network interfaces. Suitability for the purpose of replacing the digitizer of the absolute method lays in the combination of capabilities of hardware timestamping of PTP frame types, referencing and/or sourcing external timing, and possibly high precisions inter-PHC measurements. 21NRM02 - Digital-IT 34 of 40 Table 11 Properties of commonly available network interface types, either as pluggable PCIe or found as integrated devices found in COTS server hardware. Unless SFP is indicated, interfaces support electrical twisted pair. Green rows indicate suitability. Type Driver IFace PHC TSHW_CLK TSHW_TX TSHW_RX HWFLT_TX HWFLT_RX all EXT_PINS EXTTS PULSE_OUT CROSS_TS SYS_PPS PTM Granualarity I210 igb 1GE 1 y y y y y 4 2 2 n y n - I225/6 igc 2.5GE 1 y y y y y 4 2 2 y y y 4 I350 igb 1GE 1 y y y y y 0 0 0 n n n - 82576 igb 1GE 1 y y y y n 0 0 0 n n n - BCM5719 tg3 1GE 1 y y y y n 0 0 1 n n n - BCM5720 tg3 1GE 1 y y y y n 0 0 1 n n n - X553 ixgbe 1GE 0 y y y y y 0 0 0 n n n - E810 - XXV ice 25GE 4, 2xSFP 1 y y y y y 0 3 4 n n n - ConnectX - 6 Dx mlx5_core 25GE 2xSFP 2 y y y y y 0 0 0 n n n - RPi CM4 bcmgenet 1GE 1 y y y y n 1 1 1 n n n - Figure 29 Example of the estimation of free running PHCs with respect to an external reference UTC(SP). The left graph shows a snapshot of the timing of the synchronization interface using external 1PPS input/output. The clock is estimated using high order least squares that allow the short-term interpolation of the clock. The one second time deviation is in the order of 2 ns, as are the residual statistics of the window sized 300 s. The right graph shows the timing estimate of one of the capture interfaces. The measurements involve PTM via the system clock which is managed by NTP. Possible rapid changes in the system clock phase are mainly in common mode in all the measurements. The estimated noise in the capture clock with respect to the reference is about 8 ns. For practical use a simple PCAP application written in a suitable programming language such as C or C++ can integrate all required settings of the PHC infrastructure and coordinate the necessary time measurements and corrections to yield correct time stamps. Decoding of the captured traffic extracts the same information as with the oscilloscope method and yields the same functionality. For reference and example can be found in [14] Below an example of the output capturing time stamps for symmetric Peer Delay exchange including an estimate of the clocks involved as depicted in Figure 28. Interface ENP1S0 is used as clock input and management interface of the system that includes time of day disambiguation of the connected reference UTC(SP). Interfaces ENO1 and ENO2 are capture interfaces, where the system identifies as NTS2 providing the common clock for all measurements. 21NRM02 - Digital-IT 35 of 40 #clock measurements offset_ns unc clock_difference 1759126403.9995453358 -8288.0 0.0 5.0 SP_NTS2-SP_NTS2ENO1 1 1759126403.9996111393 -11489.0 0.0 5.0 SP_NTS2-SP_NTS2ENO2 2 1759126403.9997596741 62570.0 0.0 5.0 SP_NTS2-SP_NTS2ENP1S0 0 1759126404.000000000 95332.0 0.0 5.0 SP_UTC-SP_NTS2ENP1S0 1759126404.000000000 24474.0 0.0 5.0 SP_UTC-SP_NTS2ENO1 1759126404.000000000 21273.0 0.0 5.0 SP_UTC-SP_NTS2ENO2 #captured traffic type sequence packet_corr packet_time_stamp foreign_PTP_port message local_clock 1759126404.0364035691 02 0000 0 18720 0.00000000000 1759126441.36400985718 ec:46:70:ff:fe:0a:b2:ff:00:01 Pdelay_Req SP_NTS2ENO2 1759126404.0364179027 03 0200 0 18720 0.00000000000 1759126441.36405777931 ec:46:70:ff:fe:0a:b3:00:00:01 Pdelay_Resp SP_NTS2ENO1 1759126404.0364238865 10 0000 0 18720 0.00000000000 1759126441.36420416832 ec:46:70:ff:fe:0a:b3:00:00:01 Pdelay_Resp_Follow_Up SP_NTS2ENO1 1759126404.0539775662 00 0200 0 11477 0.00000000000 1759126441.53977155685 ec:46:70:ff:fe:0a:b2:ff:00:01 Sync SP_NTS2ENO2 1759126404.0539835182 08 0000 0 11477 0.00000000000 1759126441.53979754448 ec:46:70:ff:fe:0a:b2:ff:00:01 Follow_Up SP_NTS2ENO2 1759126404.0930478902 02 0000 0 18735 0.00000000000 1759126441.93045997620 ec:46:70:ff:fe:0a:b3:00:00:01 Pdelay_Req SP_NTS2ENO1 1759126404.0930630566 03 0200 0 18735 0.00000000000 1759126441.93050408363 ec:46:70:ff:fe:0a:b2:ff:00:01 Pdelay_Resp SP_NTS2ENO2 1759126404.0930695404 10 0000 0 18735 0.00000000000 1759126441.93065261841 ec:46:70:ff:fe:0a:b2:ff:00:01 Pdelay_Resp_Follow_Up SP_NTS2ENO2 The use of network interface cards has many practical advantages but suffers from difficulties determining the TAP related delays t1..5 , as shown in Figure 8. TAPs should be passive, such as optical splitters, but for twisted pair interfaces faster than 100BASE, active monitoring TAPs are the only feasible option. As the time stamp points of the Rx and Tx paths are not directly accessible, the estimation of t5, with possibly different values t5Rx and t5Tx, requires a calibrated PTP source, such as a White Rabbit Grand Master, to be used to calibrate the setup. Thus, uncertainties of this convenient approach are significantly higher than the oscilloscope method and can be generally to be about 50 ns, which is still sufficient for the application of digital substations, and possibly many other uses. Most modern Ethernet physical layer variants can easily be accessed with the same methodology, up to 10GBASE-T and 25GBASE-xR can be demonstrated. 21NRM02 - Digital-IT 36 of 40 9 Comparison of absolute and relative calibration methods For validating the methods presented in this good practice guide, a comparison measurement between VTT’s absolute calibration method and VSL’s relative calibration method was performed. Both setups were transferred to a common location. VTT’s probe was set up to monitor the traffic between VSL’s master and slave clocks. The time tag associated with PTP event messages and the internal PTP time inside the devices calculated from the data were compared against a master clock’s 1PPS output. The Slave clock 1PPS output timing was calculated based on time interval comparison in the time interval counter (TIC). This way, both master and slave clock’s internal time could be compared against their respective 1PPS outputs. The setups are shown in Figure 30 and Figure 31. The clocks are connected to one another using a 100BaseTX link, with the TAP situated between them. Clock 1 was first configured as a master (Figure 31), itself synchronized from a local UTC realization via 1PPS and NTP. Both master and slave 1PPS outputs were in turn compared to the residual PTP errors calculated from the traffic between the clocks. The master-slave hierarchy was then reversed (Figure 31), with 1PPS output of the new master compared to the traffic similarly to the first case. Figure 30. Measurement setup for comparing absolute and relative clock calibration methods. Clock 1 is master. Figure 31. Reversed measurement setup for comparing absolute and relative clock calibration methods. Clock 2 is master. In both cases, master port 1PPS output is defined as the timing reference plane. All results using the probing method are thus referenced to it. While not drawn in FIGURE, internal time tm/s and asymmetry ta,m/s of both clocks are calculated from the probing data, with positive asymmetry in all cases towards network. Using the probing data, the slave clock timing can be calculated in two ways, directly assuming ts = td,s and indirectly from apparent master clock time t’m = tm + ta,m – ta,s. Slave clock internal time with respect to its 1PPS output may conveniently be calculated from the interval measurement using the TIC. If we define the slave to master TIC measurement as tSM, with a positive sign when the master is ahead, slave 1PPS output error may be defined as 𝑡1𝑃𝑃𝑆,𝑠=𝑡𝑚+𝑡𝑆𝑀−𝑡𝑠 In a no-error scenario when tm = 0, t1PPS,s = 0 and tSM = ts the slave clock 1PPS output should be aligned with master clock time (t1PPS,s = tm), indicating a perfect operation of the PTP pair. It should be noted that here it is assumed that master clock 1PPS output error t1PPS,m is by definition the same as master clock internal time, but with an opposing sign. In any case, it is included in the calculation through probing the master clock internal time and defining the 1PPS output as the reference plane. If an error such that tSM ≠ts is introduced by any of the components in the link, i.e. either asymmetry in master or slave clocks or communications path, it should 21NRM02 - Digital-IT 37 of 40 be visible in both slave internal time and slave 1PPS output time and its magnitude measurable using the probing method or the TIC. Measurement data including standard deviations from the comparison are summarized in Table 12. The measured 1PPS difference tSM in both measurement setups is shown in Figure 32 and Figure 33. The figures show most importantly the long-term stability of the link, while in the short term, the difference occasionally jumps by multiples of roughly 8 ns. This is due to quantization of the 100 Mbit/s link bit rate and a highbandwidth clock servo loop in the slave clock, which clearly tracks the instantaneous time delivered by the PTP messages. Figure 32. Time interval measurement between master and slave clock 1PPS outputs in setup 1. Figure 33. Time interval measurement between master and slave clock 1PPS outputs in setup 2. The tabulated data first and foremost tells that the clocks in use are as good as they can be in a 100 Mbit/s network. All errors are within the standard deviation of the data. The standard deviation is mostly due to quantization of Layer 1 bit timing, which is visible in both the direct 1PPS measurement in Figure 32 and Figure 33 as well as in individual readings in the data received using the probing method. Both clocks appear to have very low asymmetry, with good repeatability between the measurements performed in the two setups. Master clock internal time, if compared to its 1PPS output is likewise nearly perfect, with a value hardly discernible from zero due to deviation of readings. Slave clock internal time ts may be calculated in two ways from the probing data: directly from PTP message time tag errors and indirectly from apparent master clock time and slave port asymmetry. Both readings agree within the standard deviation of the data. For calculating slave clock error t1PPS,s, the direct method for slave internal time is used. The error is given on the last row in the table. It should be compared to master clock internal time, as in the formula above. The difference between the two clocks is within the standard deviations in both setups, indicating that the clocks work well. Consequently, the comparison shows a clear agreement between the methods, since both methods were utilized to reach the conclusion. 21NRM02 - Digital-IT 38 of 40 A more interesting comparison case would have been if the clocks performed worse, e.g. if their asymmetries had been large enough to generate an appreciable timing error at the slave device. Adding asymmetry in the link would have accomplished the same. Link asymmetry effect was clearly visible in section 6.5, where a SAMU was calibrated using the probing method to determine master clock error in VTT’s calibration setup. Table 12. Results from the comparison measurments Property SETUP 1 SETUP 2 Master clock clock 1 clock 2 Slave clock clock 2 clock 1 Asymmetry clock 1 ta,m = (5.3 ± 4.1) ns ta,s = (7.0 ± 4.3) ns Asymmetry clock 2 ta,s = (3.6 ± 6.3) ns ta,m = (2.2 ± 4.1) ns Master clock time tm = td,m = -t1PPS,m (-6.3 ± 4.1) ns -3.4 ± 4.1 Apparent master clock time t’m = tm + ta,m = td,m + ta,m (-1.0 ± 5.8) ns -1.2 ± 5.9 1PPS difference slave to master tSM (6.2 ± 5.1) ns 4.5 ± 4.2 Slave clock time direct measurement using PTP data ts = td,s (1.1 ± 6.3) ns (-0.3 ± 4.3) ns indirect from apparent master clock time ts = t’m - ta,s (-4.6 ± 8.5) ns (-8.5 ± 7.3) ns Error based direct measurement + TIC data t1PPS,s = tm + tSM - ts(-1.1 ± 9.0) ns (1.4 ± 7.3) ns 21NRM02 - Digital-IT 39 of 40 10 Conclusion and input to standardisation IEEE 1588-2009 PTPv2 has become a widely accepted standard for synchronizing measurements on a digital substation. Whenever it is used, the timing network inevitably becomes a part of the accuracy and traceability chain for phase displacement measurement data. An analogous situation exists with 1PPS synchronicity, where the timing infrastructure is more accessible and various intermediate devices may be characterized on the field or in a laboratory test setup. While PTPv2 has so far posed an obstacle in traceable calibrations, the Digital-IT project has demonstrated that traceability is possible and, in the end, on a fundamental level the approach is similar to calibrations under 1PPS synchronization. The timing network influence on a calibrated device synchronicity must be determined and taken into account in the calibration result. In a 1PPS case one would define a timing reference plane, which supplies a 1PPS signal to both the reference device and to the DUC. With a proper calibration of the DUC, the phase displacement of the DUC may be determined. In the case of PTP synchronicity, the reference plane should still be defined. But since the time transfer medium is different for both devices, the difference between 1PPS timing for the reference device and PTP synchronicity for the DUC is essential. Definition of a PTPv2 reference plane with respect to an external 1PPS signal corresponds to calibrating the master clock or master port, which supplies the DUC (slave clock) with its timing data. After a calibrated clock is available, the only remaining influence outside the DUC is the path taken by the PTP data between the master and slave clocks. A high level of symmetry is trivial to provide by using short, equal-length optical fibers or a twister pair cable, with guaranteed (tested) Rx/Tx path delays. In practice, it’s nearly impossible to create an appreciable asymmetry in a laboratory setup, where devices are often some meters away from each other. With the two conditions met, the only influence to DUC phase displacement as regard to timing, comes from the DUC itself and is indistinguishable from other implementation errors, which might affect the performance. Three different methods for calibrating a master clock have been presented in this good practice guide. The methods rely either on determining the master clock output time based on the traffic it sends or on calibrating two similar clocks as a pair. A low-level probing method developed by VTT provides an absolute calibration with low uncertainties but is limited to 100Base-X networks, which are currently used in a process bus implementation between switches and SV transmitting devices. Another traffic-based method developed by RISE uses Linur PTP hardware clocks and suitable network interfaces to obtain the same data. VSL’s method on the other hand lends itself to various network speeds and physical media if suitable clocks supporting the requirements are available. While national metrology institutes may use highly customized and sometimes cumbersome methods for traceable calibrations with lowest uncertainties, a calibration laboratory or even an in-house testing department at a protection equipment manufacturers location may prefer a simpler method. Any wider acceptance of newly developed calibration method should be backed by a clear requirement in standardization. This would cause a demand from calibration labs by their customers and a clear business case, which helps justify investment in developing calibration services. A governing standard should therefore not only set the requirements, but also suggest best practices, such as in IEC 61869-1 for analog output ITs and LPITs, and in IEC 61869-9 for digital output LPITs. The digital output for LPITs is defined in IEC 61869-9 along with test circuits using two different reference measurement chains in Annex 9D. The 2016 version of the standard suggests using IEC 61850-9-3 for time synchronicity, but omits its implications to phase displacement. The test setups in the annex are ambiguous as regards to the type of time synchronization signal. Based on the work presented in this good practice guide, IEC TC 38 might be able to introduce a more detailed explanation in the annex as to how a calibration should be performed under PTPv2 synchronicity. For a test laboratory or any other actor, the only requirement would be to build a test setup around a calibrated PTP master clock. As is usual, the traceability for the clock could come from a national metrology laboratory, who could leverage the methods presented here. While no data exists yet, it is foreseen that a master clock, itself locked to a traceable, stable time source, would have no long-term drift. The clock calibration would therefore be a one-off event. 21NRM02 - Digital-IT 40 of 40 11 REFERENCES [1] Communication networks and systems for power utility automation - Part 9-2: Specific communication service mapping (SCSM) - Sampled values over ISO/IEC 8802-3, IEC Standard 61850-9-2:2004, April 2004. [2] Implementation Guideline for Digital Interface to Instrument Transformers Using IEC 61850-9-2, UCAIug Guide IEC 61850-9-2 LE, July 2004. [3] IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems, IEEE Std 1588-2008, July 2008. [4] Instrument transformers - Part 9: Digital interface for instrument transformers, IEC Standard 61869-9:2016, April 2016. [5] IEEE Standard Profile for Use of IEEE 1588 Precision Time Protocol in Power System Applications, IEEE Std C37.238-2011, July 2011. [6] M. Adamiak, Revision of IEEE Std C37.238, Power Profile for IEEE-1588: Why The Big Changes? [7] Communication networks and systems for power utility automation - Part 9-3: Precision time protocol profile for power utility automation, IEC/IEEE 61850-9-3:2016 [8] IEEE Standard Profile for Use of IEEE 1588 Precision Time Protocol in Power System Applications, IEEE Std C37.238-2017, June 2017. [9] scapy (2025). Available at: https://scapy.net/ (Accessed: 29.9.2025) [10] The Tcpdump Group (2025) TCPDUMP & LIBPCAP. Available at: https://www.tcpdump.org/ (Accessed: 29.9.2025) [11] The kernel development community (2025) PTP hardware clock infrastructure for Linux — The Linux Kernel documentation. Available at: https://docs.kernel.org/driver-api/ptp.html (Accessed: 29.9.2025) [12] The Linux PTP Project. Available at: https://linuxptp.sourceforge.net/ (Accessed: 29.9.2025) [13] Open Compute Project (2025): PTM Readiness. Available at: https://www.opencompute.org/wiki/PTM_Readiness (accessed: 29.9.2025) [14] digital_it_pcap · GitLab