Raspbian Stretch: issue with PPS accuracy of NTP.

[beta-tester]

i tested my setup of GPS + PPS + NTP Stratum 1 server on Raspbian Stretch
(2017-08-16-raspbian-stretch.img / 2017-08-16-raspbian-stretch-lite.img)...

but i can't get the same accuracy as i got under Raspbian Jessie
(2017-07-05-raspbian-jessie.img / 2017-07-05-raspbian-jessie-lite.img).

under Jessie i was using NTP v4.2.8p10 compiled by myself, because the only ntp version of jessie was an old version without PPS support.
on Stretch, NTP has already version 4.2.8p10.

on Jessie the jitter of PPS against system clock was constant 0.002 after 2 hours.
on Stretch the jitter of PPS against system clock is jumping somewhere between 0.047... 0.006 after hours.

is there something installed in Raspbian Stretch, that will concurrently tune the system clock so the NTP is fighting with an other application to discipline the system clock?

Code: Select all

# /etc/ntp.conf
## Stratum1

driftfile /var/lib/ntp/ntp.drift

# Enable this if you want statistics to be logged.
statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen  loopstats  file loopstats  type week  enable
filegen  peerstats  file peerstats  type week  enable
filegen  clockstats  file clockstats  type week  enable


#enable stats


# PPS(0), gpsd: /dev/pps0: Kernel-mode PPS ref-clock for the precise seconds
# http://doc.ntp.org/current-stable/drivers/driver22.html
server  127.127.22.0  minpoll 3  maxpoll 3  prefer  true
fudge   127.127.22.0  refid PPS  time1 -0.0045

# SHM(0), gpsd: Server from shared memory provided by gpsd
# # http://doc.ntp.org/current-stable/drivers/driver28.html
server  127.127.28.0  minpoll 4  maxpoll 5  prefer  true
fudge   127.127.28.0  refid NMEA  time1 0.450  stratum 10

# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: <http://www.pool.ntp.org/join.html>
#server  0.debian.pool.ntp.org  iburst
#server  1.debian.pool.ntp.org  iburst
#server  2.debian.pool.ntp.org  iburst
#server  3.debian.pool.ntp.org  iburst


# By default, exchange time with everybody, but do not allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

[Anonymous]

IIRC, stock Debian NTP doesn't have PPS support enabled so you'll have to recompile NTP with --enable-ATOM in the ./configure line, making sure the file timepps.h is in /usr/include. That should improve matters.

[beta-tester]

oooh , i thought only the old stock debian NTP version did not had the PPS option included.
that was the reason, why i was compiling NTP by my self on Raspbian Jessie.
so on Raspbian Stretch, i still have to compile the NTP even the version numbers are now the same as the one i was vcompiling on Jessie (at the moment)...
this is extremely annoying...

i will compile it tomorrow - and see if it will fix my issue.
thank you.

[beta-tester]

ok... conclusion:

1. i don't know why, but the observed huge jitter, i can not reproduce anymore.

2. on raspbian stretch it makes no difference if you use the stock ntp (4.2.8p10) via apt-get or if you download the source and compile ntp (4.2.8p10) with option --enable-ATOM. (sys/timepps.h was marked "Yes")
i reflashed and installed raspbian stretch from scratch multiple times with using ntp via apt-get, using ntp compiled with --enable-ATOM.
each time, i kept running for a day. all the time, the final jitter was in the range of 0.004..0.002.
(on raspbian jessie i got it down to 0.002)
but the point is, in my setup it makes no difference if you compile the ntp, or if you take the ntp from stock via apt-get.
maybe the stock ntp is compiled with --enable-ATOM option aready, or the way, how i use my pps is not affected from this option.
i use the ntp driver 22 (/dev/pps0) and 28 only.
(on raspbian jessie, the driver 22 of the old stock ntp wasn't working at all, so there was the need to compile ntp for sure)

maybe the reason, why Raspbian Stretch has a slightly more worse jitter, is to find somewhere in the kernel, maybe different optimization, signal handling, priority or other different stuff is going on under the hood.

i guess, i have to accept that.

[karlshea]

I'm also having issues with PPS on Stretch. My PPS source keeps getting marked as a false ticker, which just leaves the output from the GPS chip over serial and that has ~300ms swings.

Testing using `ppstest /dev/pps0` seems to work fine.

Any idea what I should try and do?

[beta-tester]

I'm also having issues with PPS on Stretch.

do you just have tested stretch,
or is it on jessie fine but stretch has issues?

My PPS source keeps getting marked as a false ticker, which just leaves the output from the GPS chip over serial and that has ~300ms swings.

Testing using `ppstest /dev/pps0` seems to work fine.

Any idea what I should try and do?

what are your settings in the ntp.conf file

your PPS and NMEA has to be marked as prefer an trust and in addition NMEA has to be adjusted as close as possible to do not reach the falsetick limit (or take a look to the dokumentation, there is an option to change the amount of ms that are accepted before the time will be marked as false tick)

because PPS does not provide the date/time, NMEA has to be accepted first by ntps, before PPS can continue to make the clock more precise.
(assuming there is no ntp sync via internet)

PS.: maybe take a look also to chrony an alternative to ntpd. easyer to setup.
i changed my script RPi-GPS-PPS-StratumOne from using ntpd to chrony

[karlshea]

I played with this a bunch the other day and I think I might have fixed it, but...

do you just have tested stretch,
or is it on jessie fine but stretch has issues?

On jessie I was using the unstable ntp package with PPS support so I didn't have to compile it, and that was working just fine. Directly after the stretch upgrade it started breaking.

I removed the unstable package and am now using stretch's version (currently [email protected]) which did not fix it all by itself.

your PPS and NMEA has to be marked as prefer an trust and in addition NMEA has to be adjusted as close as possible to do not reach the falsetick limit (or take a look to the dokumentation, there is an option to change the amount of ms that are accepted before the time will be marked as false tick)

because PPS does not provide the date/time, NMEA has to be accepted first by ntps, before PPS can continue to make the clock more precise.
(assuming there is no ntp sync via internet)

I did some reading and it seems like you don't want "prefer" on your PPS source, but in order to use PPS you must have "prefer" on another source.

Do you mean "true" instead of "trust"? I don't see docs for "trust" anywhere, but adding "true" to my PPS source might have helped. I should remove it and test again now that it's currently working.

I tried using NMEA with gpsd (/dev/gps0, /dev/gpspps0) and that seemed to have more jitter than using SHM (which I've been using all along), so I switched back.

This is what I've got now that's been working for a couple of days:

Code: Select all

# kPPS
server 127.127.22.0 minpoll 4 maxpoll 4 true
fudge 127.127.22.0 refid PPS

# GPS serial
server 127.127.28.0 prefer
fudge 127.127.28.0 time1 +0.500 refid GPS stratum 2

# Fix false tickers
tos mindist 0.5

I think the mindist change is what seems to have done it, I think something must be different with serial in stretch where there were enough pauses or something where it varied too much and was marked false.

In any case, with that config I'm averaging -3ms offset and 1ns jitter over the last two days, compared to 500ms offset and 100ms jitter after the stretch upgrade.

[beta-tester]

i didn't know, that you are already optimized your ntp.conf settings.

sorry, with NMEA i meant the SHM[0] (127.127.28.0) time source of gpsd, and not the raw serial output from the GPS device.

do you use Raspbian Stratch Lite ore the Full version with the Desktop?
i always used Raspbian Stretch Lite, so i have no esperience with the Full desktop version.

maybe the full desktop version produces much more jitter

and i use for time server the very first RPi 1.
enabled turbo (force_turbo=1), so the processor does not step up and down the processing frequency at idle time.


BTW: is your gpsd using the PPS signal of the kernel (/dev/pps0)?
(does "gpsmon" show any activity at PPS: at the bottom of the middle block "GSA + PPS")
i realized that very late, that gpsd didn't used PPS at all, to provide an additional time source on shared memory.

Code: Select all

gpsmon
...
│ 7  ?? ??? ??  ?? │┌────────────────────────────┐┌────────────────────────────┐
│ 8  ?? ??? ??  ?? ││Mode: ?? Sats: ? ?? ?? ??   ││UTC:           RMS:         │
│ 9  ?? ??? ??  ?? ││DOP: H=?.??  V=?.??  P=?.?? ││MAJ:           MIN:         │
│10  ?? ??? ??  ?? ││TOFF:  0.?????????          ││ORI:           LAT:         │
│11  ?? ??? ??  ?? ││PPS: -0.000000083           ││LON:           ALT:         │
└────── GSV ───────┘└──────── GSA + PPS ─────────┘└─────────── GST ────────────┘
------------------- PPS offset:  0.000000020 ------
 ------------------- PPS offset:  0.000000020 ------
 ...

in the past i was simple using

Code: Select all

sudo gpsd -D 5 -N -n /dev/ttyAMA0 -F /var/run/gpsd.sock

where gpsd not used the PPS signal.

Code: Select all

sudo gpsd -D 5 -N -n /dev/ttyAMA0 /dev/pps0 -F /var/run/gpsd.sock

with pps, gpsd provides an additinal "PPS accurate" time source for ntpd. should be at index 1 or 2 of the shared memory.
it is, as far as i know, merged data of PPS + NMEA, so that it includes also the date and time and not only the PPS like from the kernel pps.

Code: Select all

# 28; SHM(2), gpsd: PPS data from shared memory provided by gpsd
# # http://doc.ntp.org/current-stable/drivers/driver28.html
server  127.127.28.2  minpoll 3  maxpoll 3  true
fudge   127.127.28.2  refid SHM2  stratum 1

(on my system, it was at index 2 /SHM[2])
feeding_ntpd_from_gpsd

the advantage could be, you do not need the SHM(0)/GPS/NMEA in combination with kPPS to get a time source that ntpd accepts.
use all three available time sources in ntpd and see, if it will help you.
the priority and accuracy of the available timesources are:
1. kernel PPS (/dev/pps0) with only sync pulses.
2. gpsd shared memory SHM(1 or 2) with sync pulses of kernel PPS combined with date/time from NMEA data.
3. gpsd shared memory SHM(0) with date/time from only NMEA data.

as i said before, i switched over to chrony, because it is easier to setup for that porpose. it is not that restrictive in time sources as ntpd.
feeding_chrony_from_gpsd

EDIT: did you disabled the hardware clock time sync in stretch?

Code: Select all

sudo systemctl stop systemd-timesyncd
sudo systemctl daemon-reload
sudo systemctl disable systemd-timesyncd

[karlshea]

do you use Raspbian Stratch Lite ore the Full version with the Desktop?
i always used Raspbian Stretch Lite, so i have no esperience with the Full desktop version.

Lite, no desktop.

and i use for time server the very first RPi 1.
enabled turbo (force_turbo=1), so the processor does not step up and down the processing frequency at idle time.

This is a Pi 3 model b, turbo off.

BTW: is your gpsd using the PPS signal of the kernel (/dev/pps0)?
(does "gpsmon" show any activity at PPS: at the bottom of the middle block "GSA + PPS")
...
gpsd provides an additinal "PPS accurate" time source for ntpd. should be at index 1 or 2 of the shared memory.
it is, as far as i know, merged data of PPS + NMEA, so that it includes also the date and time and not only the PPS like from the kernel pps.

Hmm, no, the PPS line in gpsmon was empty. That's interesting, because I just checked the gpsd docs and they say that using /dev/ttyAMA0 has the side effect of opening /dev/pps0 for PPS data as well. But it doesn't, and when I added /dev/pps0 to the devices list now it's filling in.

the advantage could be, you do not need the SHM(0)/GPS/NMEA in combination with kPPS to get a time source that ntpd accepts.
use all three available time sources in ntpd and see, if it will help you.
the priority and accuracy of the available timesources are:
1. kernel PPS (/dev/pps0) with only sync pulses.
2. gpsd shared memory SHM(1 or 2) with sync pulses of kernel PPS combined with date/time from NMEA data.
3. gpsd shared memory SHM(0) with date/time from only NMEA data.

Cool, I'll give that a shot. This is what I've got so far after letting it run a couple of minutes:

Code: Select all

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
oPPS(0)          .PPS.            0 l    6   16  377    0.000   -0.004   0.007
+SHM(2)          .SHM2.           1 l    5   16  377    0.000   -0.004   0.006
*SHM(0)          .GPS.            2 l    4   64   77    0.000  -12.140  11.662

as i said before, i switched over to chrony, because it is easier to setup for that porpose. it is not that restrictive in time sources as ntpd.
feeding_chrony_from_gpsd

I'd give that a shot, but I'm monitoring it with LibreNMS and I don't think there's a chrony plugin. It's a drop-in replacement for NTP?

EDIT: did you disabled the hardware clock time sync in stretch?

Code: Select all

sudo systemctl stop systemd-timesyncd
sudo systemctl daemon-reload
sudo systemctl disable systemd-timesyncd

Strange, I don't think that I did, but when I do status it says "inactive (dead)" and "start condition failed". But I disabled it anyways.

Thanks so much for the help and tips!! NTP seems like witchcraft sometimes.

[beta-tester]

BTW: is your gpsd using the PPS signal of the kernel (/dev/pps0)?
(does "gpsmon" show any activity at PPS: at the bottom of the middle block "GSA + PPS")
...
gpsd provides an additinal "PPS accurate" time source for ntpd. should be at index 1 or 2 of the shared memory.
it is, as far as i know, merged data of PPS + NMEA, so that it includes also the date and time and not only the PPS like from the kernel pps.

Hmm, no, the PPS line in gpsmon was empty. That's interesting, because I just checked the gpsd docs and they say that using /dev/ttyAMA0 has the side effect of opening /dev/pps0 for PPS data as well. But it doesn't, and when I added /dev/pps0 to the devices list now it's filling in.

i think the Raspbian Stretch dirsto is using an older version of gpsd and the documentation is for the most actual version.
or the Raspberry Pi is a special case, because the PPS signal is on GPIO and not coming from the DTC line of the serial port as on a PC.
who knows...