4. Configure Chrony - josh-blake/pixie GitHub Wiki

Chrony sets the system time based on the GPS input and serves this via an NTP server. It has a relatively high compute demand relatively speaking. The premise here is to not only set up Chrony optimally, but to minimise disturbances it may create (and vice versa) from the responsiveness we are asking of the hardware setup.

Edit `chrony.conf`

Note this configuration is for Chrony 4.8.1; some parameters (including PTP reference) may not be available in older versions.

Adjust your configuration file, save and exit:

sudo nano /etc/chrony/chrony.conf

# Welcome to the chrony configuration file. See chrony.conf(5) for more
# information about usable directives.

# Include configuration files found in /etc/chrony/conf.d.
confdir /etc/chrony/conf.d

# Use NTP sources found in /etc/chrony/sources.d.
sourcedir /etc/chrony/sources.d

# This directive specify the location of the file containing ID/key pairs for
# NTP authentication.
keyfile /etc/chrony/chrony.keys

# This directive specify the file into which chronyd will store the rate
# information.
driftfile /var/lib/chrony/chrony.drift

# Save NTS keys and cookies.
ntsdumpdir /var/lib/chrony

## Logging
# Log file location.
logdir /var/log/chrony
# Don't put a header row
logbanner 0
# Log the following
#log refclocks

# Don't page out chrony
lock_all

# Stop bad estimates upsetting machine clock.
#maxupdateskew 100.0

#initstepslew 0.1 ntp.sydney.nmi.gov.au

# This directive enables kernel synchronisation (every 11 minutes) of the
# real-time clock. Note that it can't be used along with the 'rtcfile' directive.
rtcsync
rtconutc

#Fallback to local RTC
local stratum 10

# Step the system clock instead of slewing it if the adjustment is larger than
# one second, but only in the first five clock updates.
makestep 0.1 5

# Get TAI-UTC offset and leap seconds from the system tz database.
# This directive must be commented out when using time sources serving
# leap-smeared time.
leapseclist /usr/share/zoneinfo/leap-seconds.list

# Defining the networks allowed to access the service
allow
broadcast 1 192.168.1.255
ptpport 319
# High Priority Expedited Forwarding DSCP directive traffic
dscp 46

# Enables hardware timestamping
hwtimestamp *

#SOCK GPS with PPS
refclock SOCK /run/chrony.ttyAMA0.sock refid SOCK precision 1e-9 offset 0 poll 0 noselect
#refclock PPS /dev/pps0 refid PPS precision 1e-9 poll 0 lock sPPS
#refclock SOCK /run/chrony.pps0.sock refid sPPS precision 1e-9 offset 0 poll 0 noselect
#refclock SOCK /run/chrony.satpulse.sock refid sPPS precision 1e-9 offset 0 poll 0 filter 10 lock SOCK
refclock PPS /dev/pps0 refid PPS precision 1e-9 poll 0 lock SOCK filter 10
#refclock SOCK /run/chrony.ptp.sock refid SOCK precision 1e-9 offset 0 poll 0

#SHM GPS with PPS
#refclock SHM 0 poll 0 refid NMEA precision 1e-9 offset 0 delay 0 noselect
#refclock PPS /dev/pps0 refid PPS precision 1e-9 poll 0 lock NMEA
#refclock SHM 1 poll 0 refid SHM precision 1e-9 lock NMEA

## Realtime clock
#refclock RTC /dev/rtc0:utc refid RTC precision 1e-9 poll 3 local noselect

#PHC eth0 Refclock; For monitoring purposes
#refclock PHC /dev/ptp0:extpps:pin=0 tai refid PHC precision 1e-9 poll 0 rate 1 width 0.1 lock SOCK
#refclock PHC /dev/ptp0 tai refid PHC precision 1e-9 dpoll 0 poll 0 noselect
#refclock PHC /dev/ptp0:extpps:pin=0 tai refid PHC precision 1e-9 dpoll 0 poll 0 rate 1 width 0.1 noselect

Restart chrony (and GPSD) with:

sudo systemctl restart chrony gpsd

Note that GPSD must also be restarted with the above configuration as it relies on sockets that Chrony creates to send time information to Chrony.

Explanation

Some points in the configuration file are straight forward. Feel free to segment it as you please, or to split it up into drop-ins.

The lock_all directive I think is important here as it keeps Chrony in RAM rather than paging it out of memory. This significantly reduces latency. The same directive can be specified -m flag in /etc/default/chrony (this file contains the command line arguments that systemd uses when launching Chrony as a service).

The rtcsync and rtconutc directives ensure that the RTC is periodically updated once the clock synchronises and every 11 mins thereafter. This ensures that a reboot of the system will allow the clock to come back up with a sane timebase and minimise time skewing.

Chrony is allowed to receive both IPv4 and IPv6 packets from all addresses with the allow directive. It also broadcasts the time every second on the broadcast address for my local subnet with the broadcast 1 192.168.1.255 directive. The PTP port number for NTP over PTP is specified by ptpport 319. The priority of UDP packets sent is dscp 46, and ensures that these packets are triaged as high priority, low latency packets. This is particularly important in minimising hop delay over the internet when acting as a time server.

The hwtimestamp * enables hardware based timestamps for all packets both incoming and outgoing.

There are a number of permutations that I tried in how Chrony received timing data from GPSD. I have included these, commented out, in the above configuration example. Traditionally, the kernel shared-memory segment was used (SHM) although this has fallen out of vogue with Chrony in favour of sockets based communication. Sharing time information by SHM requires Chrony to poll the memory segment for new information; this places a theoretical limit on Chrony's accuracy and makes it entirely dependent on the kernel scheduler and kernel tick frequency. Instead, Chrony implements socket-based communication that does NOT require polling and instead creates a callback function that gets executed whenever a new datagram is received. I have found that the most accurate timestamping comes from a Chrony socket to the GPSD serial data (/dev/ttyAMA0), and disciplined by the PPS signal created on /dev/pps0. The PPS time discipline is also made available via GPSD sockets as well, however I would have thought an extra layer of indirection would add delay. Feel free to play around here. Some day, I will collect some data and put it up for review. By eyeball, it adds about 20ns of jitter, and delays by 40us.

Note that a PPS signal is required to discipline the time message received by GPSD via the serial port.

Finally, I have included some references to monitor the NIC time as PHC sources in Chrony. In particular, the refclock PHC /dev/ptp0:extpps:pin=0 tai refid PHC precision 1e-9 dpoll 0 poll 0 rate 1 width 0.1 noselect reference monitors the NIC and uses the SYNC_OUT pin to receive the same PPS time pulse from the GPS HAT. It should be noted that ts2phc should NOT be running concurrently if you are monitoring the PHC this way as it creates a race condition between the two processes. The timestamping on the SYNC_OUT pin is cleared by the process that attributes the timestamp meaning only one process can read it. Both Chrony and ts2phc will report intermittent timestamps otherwise (depending on which process accessed the pin first). I use ts2phc for NIC timestamping and instead poll the NIC with refclock PHC /dev/ptp0 tai refid PHC precision 1e-9 dpoll 0 poll 0 noselect. I only enable this as a sanity check, and otherwise leave it disabled in Chrony. This way, Chrony is responsible for setting the system time, ptp4l copies the system time to the NIC, and the PPS timestamp is added by ts2phc. The other option is to configure ptp4l to read the GPS unit directly, completely eliminating the need for GPSD, and to use ts2phc to discipline the NIC. I have found that ptp4l is buggy in this regard (and requires the NMEA TPV sentence to be enabled). Chrony can then be used to read the time from the NIC and serve based on that. I prefer the former rather than the latter.

Edit Chrony systemd unit file

The last thing to do is edit the chrony.service systemd unit file to prioritise the process, ensure that its startup sequence is correct, and isolate the service to CPU core (this minimises disruption from other system processes like logging in over SSH, system updates etc).

Adjust the chronyd.service file to resemble the following, save and exit:

Do not forget to comment out the following lines if you are not yet running a realtime kernel:

CPUSchedulingPolicy=rr
CPUSchedulingPriority=20
CPUAffinity=2

sudo nano /etc/systemd/system/chronyd.service

[Unit]
Description=chrony, an NTP client/server
Documentation=man:chronyd(8) man:chronyc(1) man:chrony.conf(5)
Conflicts=openntpd.service ntp.service ntpsec.service
ConditionVirtualization=!container
ConditionCapability=CAP_SYS_TIME
After=network.target

[Service]
Type=notify
PIDFile=/run/chrony/chronyd.pid
EnvironmentFile=-/etc/default/chrony
User=_chrony
CPUSchedulingPolicy=rr
CPUSchedulingPriority=20
CPUAffinity=2
# Daemon is started as root, but still sandboxed
ExecStart=!/usr/sbin/chronyd -n $DAEMON_OPTS

CapabilityBoundingSet=~CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE
CapabilityBoundingSet=~CAP_BLOCK_SUSPEND CAP_KILL CAP_LEASE CAP_LINUX_IMMUTABLE
CapabilityBoundingSet=~CAP_MAC_ADMIN CAP_MAC_OVERRIDE CAP_MKNOD CAP_SYS_ADMIN
CapabilityBoundingSet=~CAP_SYS_BOOT CAP_SYS_CHROOT CAP_SYS_MODULE CAP_SYS_PACCT
CapabilityBoundingSet=~CAP_SYS_PTRACE CAP_SYS_RAWIO CAP_SYS_TTY_CONFIG CAP_WAKE_ALARM
DeviceAllow=char-pps rw
DeviceAllow=char-ptp rw
DeviceAllow=char-rtc rw
DevicePolicy=closed
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoNewPrivileges=yes
PrivateTmp=yes
ProcSubset=pid
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
ProtectSystem=strict
# Used for gps refclocks
ReadWritePaths=/run
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
RestrictNamespaces=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=~@cpu-emulation @debug @module @mount @obsolete @raw-io @reboot @swap

ConfigurationDirectory=chrony
RuntimeDirectory=chrony
RuntimeDirectoryMode=0700
# See dumpdir in chrony.conf(5)
RuntimeDirectoryPreserve=restart
StateDirectory=chrony
StateDirectoryMode=0750
LogsDirectory=chrony
LogsDirectoryMode=0750

# Adjust restrictions for /usr/sbin/sendmail (mailonchange directive)
NoNewPrivileges=no
ReadWritePaths=-/var/spool
RestrictAddressFamilies=AF_NETLINK

[Install]
Alias=chronyd.service
WantedBy=multi-user.target

Don't forget to reload the service, and restart Chrony (and GPSD!)

sudo systemctl daemon-reload