Skip to main content

Time Synchronization

Facts​

Introduction​

This guide describes the steps to setup time-synchronization for embedded single-board computers (SBCs) such as a Raspberry Pi. The time-synchronization relies on a commodity GPS module providing a pulse-per-second (PPS) output to a GPIO pin of the SBC.

Employed technologies​

Applications​

  • Time-delay compensation
  • Logging/time-stamping/tracing of interface signals
  • Network monitoring

Functional Requirements​

  • Microsecond accuracy
  • Auto-configuration of synchronization source
  • Support for multiple synchronization sources
  • Reporting of synchronization status

Architecture​

DaemonSet​

Time-synchronization is implemented by a cluster-wide DaemonSet which spawns a single Pod on each of the cluster nodes. These Pods execute three containers:

  • chronyd
  • chronyd-monitor
  • gpsd

All containers in the Pod communicate via sockets which are placed into a shared emptyDir volume. There are three sockets created:

  • gpsd -> chronyd coarse date-time: /run/chrony.ttyAMA0.sock
  • gpsd -> chronyd precise PPS timing: /run/chrony.pps0.sock
  • chronyd-monitor -> chronyd status reporting: /run/chrony

All sockets are created by chronyd and subsequentially opened by gpsd and the chronyd-monitor script. The first two sockets are used to provide rough date-time as well as PPS timing information from gpsd to chronyd. The last socket is usually used by the chronyc command line tool to interact with the chronyd daemon. Here the socket is also used by the chronyd-monitor script to periodically check the sync status and publish it to the Kubernetes api-server.

In addition to the shared sockets, some physical devices are mounted from the host into the containers using a hostPath volume. The gpsd container gets both the serial device /dev/ttyAMA0 as well as the /dev/pps0 device mounted. The chronyd container only gets the PPS device /dev/pps0 mounted.

Linux kernel-level PPS device​

Both gpsd and chronyd use the kernel-based PPS device order for the PPS /dev/pps0 device to be created a special devicetree overlay needs to be loaded during boot-up.

Status reporting​

As mentioned above a dedicated chronyd-monitor container in the time-sync Pods is used to periodically publish the current synchronization state from chronyd in the form of a Node status-condition to the Kubernetes api-server.

This container runs a Python script which periodically calls the chronyc tracking and chronyc sources commands to query the current synchronization status. Afterwards, the script will publish this status as Node status-condition to the Kubernetes api-server as well include more details a annotations to the Node resource.

Synchronization sources​

In the order of their priority:

  1. Pulse-per-second signal (PPS)
    1. Kernel-based PPS (kPPS)
    2. User-space PPS
  2. Precision Time Protocol (PTP)
  3. Network Time Protocol (NTP)
  4. Real-time Clock (RTC)

Testing​

  1. Local testing

    • Two Raspberry Pis
    • Independent synchronization sources
    • Equal or different synchronization types
    • Produce periodic rising edge via GPIO
    • Use oscilloscope to measure time offset between edges

  2. Reporting synchronization status between RIs

    • Retrieve sync status from Chrony

Further Reading​