Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Hayden, The time is in UTC, as reported by the RTCM. The system time is typically a couple minutes off, but it wanders a bit. Ergo, each time command 98 is executed, a different delta is reported. I’m not aware of a full calibration routine, what are you referring to? The GPS is connected at 9600 baud. The RTCM’s have been modified to accept an external 9.6MHz reference, which is locked to GPS. This is the relevant output from command 98: Current Time: Fri Jun 01, 2018 00:17:43.740 Last Rx Pkt System time: 06/01/2018 00:16:08.440, diff: 95300 msec Last Rx Pkt Timestamp time: 06/01/2018 00:16:08.440, diff: 0 msec Last Rx Pkt index: 160, inbounds: 1 Does “System Time” reference the time on the computer which apt_rpt executes? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618, ,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Chuck, A couple facts: The network has very low latency (sub ms) across all nodes on the VLAN. The site labeled LAB has a lab quality GPSDO, by NOVUS Power. The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage. Question: Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output. Here is an example of the output from my implementation: GPSTime (Lab): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.220000 DrainTime: 06/11/18 20:16:57.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71 GPSTime (Valparaiso): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5 GPSTime (Crown_Point): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6 GPSTime (LaPorte): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6 GPSTime (Lab): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.220000 DrainTime: 06/11/18 20:17:19.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38 GPSTime (Valparaiso): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.260000 DrainTime: 06/11/18 20:17:19.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7 GPSTime (Crown_Point): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5 GPSTime (LaPorte): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1 Results AFTER resetting the MASTER: GPSTime (Lab): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.220000 DrainTime: 06/11/18 20:31:23.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86 GPSTime (Valparaiso): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8 GPSTime (LaPorte): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7 GPSTime (Crown_Point): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5 Thoughts? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Chuck Henderson Sent: Monday, June 4, 2018 4:51 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge. I am also using a Raspberry Pi 3B (not plus) The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Chuck, “Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 3:35 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge. I am also using a Raspberry Pi 3B (not plus) The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, A couple facts: The network has very low latency (sub ms) across all nodes on the VLAN. The site labeled LAB has a lab quality GPSDO, by NOVUS Power. The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage. Question: Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output. Here is an example of the output from my implementation: GPSTime (Lab): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.220000 DrainTime: 06/11/18 20:16:57.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71 GPSTime (Valparaiso): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5 GPSTime (Crown_Point): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6 GPSTime (LaPorte): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6 GPSTime (Lab): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.220000 DrainTime: 06/11/18 20:17:19.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38 GPSTime (Valparaiso): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.260000 DrainTime: 06/11/18 20:17:19.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7 GPSTime (Crown_Point): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5 GPSTime (LaPorte): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1 Results AFTER resetting the MASTER: GPSTime (Lab): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.220000 DrainTime: 06/11/18 20:31:23.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86 GPSTime (Valparaiso): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8 GPSTime (LaPorte): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7 GPSTime (Crown_Point): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5 Thoughts? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Monday, June 4, 2018 4:51 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server. On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 3:35 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Chuck, Is Systime coming from the RTCM or app_rpt? Any thoughts on what could be causing this anomaly (Systime not syncing with GPS time)? You reference servers, by that do you mean instances of app_rpt? I’ll change to the LAB source for master timing and see if anything changes. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 9:23 PM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server. On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, “Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 3:35 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge. I am also using a Raspberry Pi 3B (not plus) The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, A couple facts: The network has very low latency (sub ms) across all nodes on the VLAN. The site labeled LAB has a lab quality GPSDO, by NOVUS Power. The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage. Question: Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output. Here is an example of the output from my implementation: GPSTime (Lab): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.220000 DrainTime: 06/11/18 20:16:57.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71 GPSTime (Valparaiso): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5 GPSTime (Crown_Point): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6 GPSTime (LaPorte): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6 GPSTime (Lab): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.220000 DrainTime: 06/11/18 20:17:19.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38 GPSTime (Valparaiso): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.260000 DrainTime: 06/11/18 20:17:19.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7 GPSTime (Crown_Point): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5 GPSTime (LaPorte): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1 Results AFTER resetting the MASTER: GPSTime (Lab): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.220000 DrainTime: 06/11/18 20:31:23.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86 GPSTime (Valparaiso): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8 GPSTime (LaPorte): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7 GPSTime (Crown_Point): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5 Thoughts? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Monday, June 4, 2018 4:51 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
yes, by servers I mean other Raspberry Pi-3B computers running app_rpt On Tue, Jun 12, 2018 at 9:45 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
Is Systime coming from the RTCM or app_rpt? Any thoughts on what could be causing this anomaly (Systime not syncing with GPS time)? You reference servers, by that do you mean instances of app_rpt? I’ll change to the LAB source for master timing and see if anything changes.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 9:23 PM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work.
I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime.
I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server.
On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 3:35 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
I recommend restarting asterisk after the changes. On Tue, Jun 12, 2018 at 9:45 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
Is Systime coming from the RTCM or app_rpt? Any thoughts on what could be causing this anomaly (Systime not syncing with GPS time)? You reference servers, by that do you mean instances of app_rpt? I’ll change to the LAB source for master timing and see if anything changes.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 9:23 PM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work.
I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime.
I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server.
On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 3:35 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Done… no change. I’m really curious why the systime doesn’t track the gpstime. Is systime influenced by the directly connected GPS receiver, or is this a reflection of the asterisk server? Does the master influence asterisk/app_rpt which in turn influences the slave RTCM’s? I’m trying to understand the flow of signaling, this will signal where to look for the problem. (i.e. is it an RTCM or Asterisk/app_rpt code issue?) Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 10:03 PM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** I recommend restarting asterisk after the changes. On Tue, Jun 12, 2018 at 9:45 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, Is Systime coming from the RTCM or app_rpt? Any thoughts on what could be causing this anomaly (Systime not syncing with GPS time)? You reference servers, by that do you mean instances of app_rpt? I’ll change to the LAB source for master timing and see if anything changes. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 9:23 PM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server. On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, “Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 3:35 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge. I am also using a Raspberry Pi 3B (not plus) The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, A couple facts: The network has very low latency (sub ms) across all nodes on the VLAN. The site labeled LAB has a lab quality GPSDO, by NOVUS Power. The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage. Question: Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output. Here is an example of the output from my implementation: GPSTime (Lab): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.220000 DrainTime: 06/11/18 20:16:57.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71 GPSTime (Valparaiso): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5 GPSTime (Crown_Point): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6 GPSTime (LaPorte): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6 GPSTime (Lab): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.220000 DrainTime: 06/11/18 20:17:19.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38 GPSTime (Valparaiso): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.260000 DrainTime: 06/11/18 20:17:19.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7 GPSTime (Crown_Point): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5 GPSTime (LaPorte): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1 Results AFTER resetting the MASTER: GPSTime (Lab): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.220000 DrainTime: 06/11/18 20:31:23.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86 GPSTime (Valparaiso): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8 GPSTime (LaPorte): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7 GPSTime (Crown_Point): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5 Thoughts? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Monday, June 4, 2018 4:51 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
This may be a bit off topic but I've had problems with the BG7TBL units. One won't get a RTCM GPS lock with initial power on after it's been off for a while... like a day or more. The GPS Lock LED on the GPS comes on but doesn't on the RTCM. Subsequent immediate power cycle cures that. Another I've seen shifts it's time by one second after it's been running for some period. On the other hand I have one that works flawlessly. So I'm a little concerned about these. I don't know if there some incompatibility with the RTCM or if the BG7TLB units have inconsistent performance. Any thoughts on that possibly being repeated to the issue Kevin is having? On Tue, Jun 12, 2018 at 7:24 PM Chuck Henderson < rpt2@chuck.midlandsnetworking.com> wrote:
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server.
On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 3:35 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Yes, many GPS will sometimes be off by one second in the RTCM. If I have to power cycle any of my voters/RTCMs I usually then do a reboot via the RTCM menu so that the GPS is already up and accurate when the RTCM boots and this seems to eliminate the one second off problem, most of the time. Both Jim and I tried to fix this and neither of us could. When all set up to troubleshoot it the problem never happens. The software has a couple of code segments to try to make it work even when exactly one second off but I don't think that works all the time either. Also in looking at the code I see that SysTime does not matter at all. It is just a reference to look at in the debug output. Mine exactly matches because I also sync my Linux time to a GPS reference separately from the app_rpt voter code. But I tried disabling that and setting the date and time both wrong and the voter still works because it keeps it's own time that is referenced to the master and not dependent on the system time. So the master time does need to be the first time that the voter channel driver receives each second. Other RTCMs time must arrive later than the master arrives but before the buffer length later. The RTCM-GPS time can be any amount wrong as long as all RTCM-GPSs are exactly the same as each other. Start with much bigger buffers than you need and gradually reduce them. On Tue, Jun 12, 2018 at 10:03 PM Tim Sawyer <tisawyer@gmail.com> wrote:
This may be a bit off topic but I've had problems with the BG7TBL units. One won't get a RTCM GPS lock with initial power on after it's been off for a while... like a day or more. The GPS Lock LED on the GPS comes on but doesn't on the RTCM. Subsequent immediate power cycle cures that. Another I've seen shifts it's time by one second after it's been running for some period.
On the other hand I have one that works flawlessly. So I'm a little concerned about these. I don't know if there some incompatibility with the RTCM or if the BG7TLB units have inconsistent performance.
Any thoughts on that possibly being repeated to the issue Kevin is having?
On Tue, Jun 12, 2018 at 7:24 PM Chuck Henderson < rpt2@chuck.midlandsnetworking.com> wrote:
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server.
On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 3:35 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Chuck, How does one go about syncing the server in app_rpt? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 11:50 PM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Yes, many GPS will sometimes be off by one second in the RTCM. If I have to power cycle any of my voters/RTCMs I usually then do a reboot via the RTCM menu so that the GPS is already up and accurate when the RTCM boots and this seems to eliminate the one second off problem, most of the time. Both Jim and I tried to fix this and neither of us could. When all set up to troubleshoot it the problem never happens. The software has a couple of code segments to try to make it work even when exactly one second off but I don't think that works all the time either. Also in looking at the code I see that SysTime does not matter at all. It is just a reference to look at in the debug output. Mine exactly matches because I also sync my Linux time to a GPS reference separately from the app_rpt voter code. But I tried disabling that and setting the date and time both wrong and the voter still works because it keeps it's own time that is referenced to the master and not dependent on the system time. So the master time does need to be the first time that the voter channel driver receives each second. Other RTCMs time must arrive later than the master arrives but before the buffer length later. The RTCM-GPS time can be any amount wrong as long as all RTCM-GPSs are exactly the same as each other. Start with much bigger buffers than you need and gradually reduce them. On Tue, Jun 12, 2018 at 10:03 PM Tim Sawyer <tisawyer@gmail.com <mailto:tisawyer@gmail.com> > wrote: This may be a bit off topic but I've had problems with the BG7TBL units. One won't get a RTCM GPS lock with initial power on after it's been off for a while... like a day or more. The GPS Lock LED on the GPS comes on but doesn't on the RTCM. Subsequent immediate power cycle cures that. Another I've seen shifts it's time by one second after it's been running for some period. On the other hand I have one that works flawlessly. So I'm a little concerned about these. I don't know if there some incompatibility with the RTCM or if the BG7TLB units have inconsistent performance. Any thoughts on that possibly being repeated to the issue Kevin is having? On Tue, Jun 12, 2018 at 7:24 PM Chuck Henderson <rpt2@chuck.midlandsnetworking.com <mailto:rpt2@chuck.midlandsnetworking.com> > wrote: The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server. On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, “Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Tuesday, June 12, 2018 3:35 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge. I am also using a Raspberry Pi 3B (not plus) The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Chuck, A couple facts: The network has very low latency (sub ms) across all nodes on the VLAN. The site labeled LAB has a lab quality GPSDO, by NOVUS Power. The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage. Question: Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output. Here is an example of the output from my implementation: GPSTime (Lab): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.220000 DrainTime: 06/11/18 20:16:57.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71 GPSTime (Valparaiso): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5 GPSTime (Crown_Point): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6 GPSTime (LaPorte): 06/11/18 20:16:57.000000000 SysTime: 06/11/18 20:16:57.280000 DrainTime: 06/11/18 20:16:57.220 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6 GPSTime (Lab): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.220000 DrainTime: 06/11/18 20:17:19.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38 GPSTime (Valparaiso): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.260000 DrainTime: 06/11/18 20:17:19.220 Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7 GPSTime (Crown_Point): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5 GPSTime (LaPorte): 06/11/18 20:17:19.000000000 SysTime: 06/11/18 20:17:19.280000 DrainTime: 06/11/18 20:17:19.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1 Results AFTER resetting the MASTER: GPSTime (Lab): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.220000 DrainTime: 06/11/18 20:31:23.180 Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86 GPSTime (Valparaiso): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8 GPSTime (LaPorte): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7 GPSTime (Crown_Point): 06/11/18 20:31:23.000000000 SysTime: 06/11/18 20:31:23.280000 DrainTime: 06/11/18 20:31:23.240 Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5 Thoughts? Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Chuck Henderson Sent: Monday, June 4, 2018 4:51 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> > Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
Chuck, I have one now that's off 2 seconds. This one has been not voting for a few days. Seems the time may be drifting. I've never seen this happen except with the BG7 units. Do you want to Teamview to trouble shoot? On Tue, Jun 12, 2018 at 9:51 PM Chuck Henderson < rpt2@chuck.midlandsnetworking.com> wrote:
Yes, many GPS will sometimes be off by one second in the RTCM. If I have to power cycle any of my voters/RTCMs I usually then do a reboot via the RTCM menu so that the GPS is already up and accurate when the RTCM boots and this seems to eliminate the one second off problem, most of the time. Both Jim and I tried to fix this and neither of us could. When all set up to troubleshoot it the problem never happens. The software has a couple of code segments to try to make it work even when exactly one second off but I don't think that works all the time either.
Also in looking at the code I see that SysTime does not matter at all. It is just a reference to look at in the debug output. Mine exactly matches because I also sync my Linux time to a GPS reference separately from the app_rpt voter code. But I tried disabling that and setting the date and time both wrong and the voter still works because it keeps it's own time that is referenced to the master and not dependent on the system time.
So the master time does need to be the first time that the voter channel driver receives each second. Other RTCMs time must arrive later than the master arrives but before the buffer length later. The RTCM-GPS time can be any amount wrong as long as all RTCM-GPSs are exactly the same as each other. Start with much bigger buffers than you need and gradually reduce them.
On Tue, Jun 12, 2018 at 10:03 PM Tim Sawyer <tisawyer@gmail.com> wrote:
This may be a bit off topic but I've had problems with the BG7TBL units. One won't get a RTCM GPS lock with initial power on after it's been off for a while... like a day or more. The GPS Lock LED on the GPS comes on but doesn't on the RTCM. Subsequent immediate power cycle cures that. Another I've seen shifts it's time by one second after it's been running for some period.
On the other hand I have one that works flawlessly. So I'm a little concerned about these. I don't know if there some incompatibility with the RTCM or if the BG7TLB units have inconsistent performance.
Any thoughts on that possibly being repeated to the issue Kevin is having?
On Tue, Jun 12, 2018 at 7:24 PM Chuck Henderson < rpt2@chuck.midlandsnetworking.com> wrote:
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work. I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime. I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server.
On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Tuesday, June 12, 2018 3:35 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Just curious Chuck, what are you using for your GPS? On Tue, Jun 12, 2018, 4:36 AM Chuck Henderson < rpt2@chuck.midlandsnetworking.com> wrote:
Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
The original Garmin hockey puck GPS is what I use. I have tried several others that worked but I prefer the Garmin, I just put it outside magnetically attached to the tower and extend the cable with some surplus cat 5 if needed. I have never had a problem with them, some bought back in 2011 and others bought in 2016 and 2017, I have never had one fail YET. (I hope Murphy didn't hear that). On Tue, Jun 12, 2018 at 9:54 PM Joe Moskalski <kc2irv@gmail.com> wrote:
Just curious Chuck, what are you using for your GPS?
On Tue, Jun 12, 2018, 4:36 AM Chuck Henderson < rpt2@chuck.midlandsnetworking.com> wrote:
Kevin, Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly. Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich@gmail.com> wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Chuck Henderson *Sent:* Monday, June 4, 2018 4:51 AM *To:* Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1
RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com> wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
*From:* App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> *On Behalf Of *Hayden Honeywood *Sent:* Thursday, May 31, 2018 5:56 PM
*To:* app_rpt-users@lists.allstarlink.org *Subject:* Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
Chuck, One other system fact: App_rpt is running on a Raspberry Pi3, the network is operated with both SIMULCAST and VOTER active. Can you please define the source and meaning of the following: GPSTime **I understand this to be the time reported by the GPSDO. SysTime **Is this the time which the VOTER is reporting it is receiving from the MASTER or app_rpt? DrainTime **The time which packets left the VOTER for app_rpt? Ultimately, I’m attempting to understand this in order to troubleshoot the implementation, as I have it. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org> On Behalf Of Chuck Henderson Sent: Monday, June 4, 2018 4:51 AM To: Users of Asterisk app_rpt <app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Kevin, I just checked 3 of my RTCM's and got the following results. RTCM1: Current Time: Mon Jun 04, 2018 08:55:43.640 Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec Last Rx Pkt index: 1080, inbounds: 1 RTCM2: Current Time: Mon Jun 04, 2018 08:58:20.300 Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 RTCM3: Current Time: Mon Jun 04, 2018 08:58:48.060 Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec Last Rx Pkt index: 0, inbounds: 0 I would not be concerned about the times. They are not important. A better thing to look at is in the Asterisk CLI issue the command voter debug level 3 Then expect output like the following: GPSTime (South): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:16.000000000 SysTime: 06/04/18 04:07:16.000000 DrainTime: 06/04/18 04:07:15.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (East): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:17.000000000 SysTime: 06/04/18 04:07:17.000000 DrainTime: 06/04/18 04:07:16.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 GPSTime (South): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:18.000000000 SysTime: 06/04/18 04:07:18.000000 DrainTime: 06/04/18 04:07:17.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 GPSTime (North): 06/04/18 04:07:19.000000000 SysTime: 06/04/18 04:07:19.000000 DrainTime: 06/04/18 04:07:18.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 sending KEEPALIVE (GPS) packet to client East digest 99479120 GPSTime (East): 06/04/18 04:07:19.000000000 sending KEEPALIVE (GPS) packet to client North digest dd455bb0 SysTime: 06/04/18 04:07:19.000000 sending KEEPALIVE (GPS) packet to client South digest 4a396b22 DrainTime: 06/04/18 04:07:18.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (North): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.960 Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4 GPSTime (East): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2 GPSTime (South): 06/04/18 04:07:20.000000000 SysTime: 06/04/18 04:07:20.000000 DrainTime: 06/04/18 04:07:19.980 Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7 Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them. Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server). If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one. I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock. If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM. If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs). Chuck On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich@gmail.com <mailto:kevin.babich@gmail.com> > wrote: Hayden, When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so: 06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018 I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity. Kindest Regards, Kevin Babich | N9IAA Valparaiso, IN 46383 From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Thursday, May 31, 2018 5:56 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** Is the time in UTC? How far off is the system time? Have you done a full calibration of each board? What baud rate do you run on your GPS's? I tend to use 9600 baud and have had no issues with the GPS's I have used. There is various GPS debug tools available in the RTCM. They are documented here - https://wiki.allstarlink.org/wiki/RTCM_Client#GPS Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency. I can't comment on that as I built all my boards and they have external frequency references. Regards Hayden VK7HH From: App_rpt-users <app_rpt-users-bounces@lists.allstarlink.org <mailto:app_rpt-users-bounces@lists.allstarlink.org> > On Behalf Of Hayden Honeywood Sent: Wednesday, May 30, 2018 5:34 PM To: app_rpt-users@lists.allstarlink.org <mailto:app_rpt-users@lists.allstarlink.org> Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated*** What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network. Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments. Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously? Regards Hayden VK7HH
participants (5)
-
Chuck Henderson -
Hayden Honeywood -
Joe Moskalski -
Kevin Babich -
Tim Sawyer