[App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin Babich
kevin.babich at gmail.com
Wed Jun 13 06:22:09 UTC 2018
Chuck,
How does one go about syncing the server in app_rpt?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
From: App_rpt-users <app_rpt-users-bounces at lists.allstarlink.org> On Behalf Of Chuck Henderson
Sent: Tuesday, June 12, 2018 11:50 PM
To: Users of Asterisk app_rpt <app_rpt-users at lists.allstarlink.org>
Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Yes, many GPS will sometimes be off by one second in the RTCM. If I have to power cycle any of my voters/RTCMs I usually then do a reboot via the RTCM menu so that the GPS is already up and accurate when the RTCM boots and this seems to eliminate the one second off problem, most of the time. Both Jim and I tried to fix this and neither of us could. When all set up to troubleshoot it the problem never happens. The software has a couple of code segments to try to make it work even when exactly one second off but I don't think that works all the time either.
Also in looking at the code I see that SysTime does not matter at all. It is just a reference to look at in the debug output. Mine exactly matches because I also sync my Linux time to a GPS reference separately from the app_rpt voter code. But I tried disabling that and setting the date and time both wrong and the voter still works because it keeps it's own time that is referenced to the master and not dependent on the system time.
So the master time does need to be the first time that the voter channel driver receives each second. Other RTCMs time must arrive later than the master arrives but before the buffer length later.
The RTCM-GPS time can be any amount wrong as long as all RTCM-GPSs are exactly the same as each other.
Start with much bigger buffers than you need and gradually reduce them.
On Tue, Jun 12, 2018 at 10:03 PM Tim Sawyer <tisawyer at gmail.com <mailto:tisawyer at gmail.com> > wrote:
This may be a bit off topic but I've had problems with the BG7TBL units. One won't get a RTCM GPS lock with initial power on after it's been off for a while... like a day or more. The GPS Lock LED on the GPS comes on but doesn't on the RTCM. Subsequent immediate power cycle cures that. Another I've seen shifts it's time by one second after it's been running for some period.
On the other hand I have one that works flawlessly. So I'm a little concerned about these. I don't know if there some incompatibility with the RTCM or if the BG7TLB units have inconsistent performance.
Any thoughts on that possibly being repeated to the issue Kevin is having?
On Tue, Jun 12, 2018 at 7:24 PM Chuck Henderson <rpt2 at chuck.midlandsnetworking.com <mailto:rpt2 at chuck.midlandsnetworking.com> > wrote:
The server needs to have the earliest time of all GPS time receivers. Master RTCM should be on the same LAN switch as the server and all other GPS time receivers must be at least some fraction of nano seconds delayed from the master time. Your LAB time arrives at the server before the master time arrives and that is not allowed for in the software. I am surprised that it even works at all. It should not work.
I also am surprised that the SysTime does not exactly match with the Master Time “Valparaiso” as I checked 3 servers of mine here and the SysTime always exactly matches the master GPSTime. Master GPSTime is the reference for SysTime.
I am reviewing the source to see what a non master time arriving before the master time might mess up. I know it has never worked for me when I have tried using an even slightly delayed master time like putting a couple of store and forward ethernet devices between the master time and the server.
On Tue, Jun 12, 2018 at 4:31 PM Kevin Babich <kevin.babich at gmail.com <mailto:kevin.babich at gmail.com> > wrote:
Chuck,
“Valparaiso” is the master, while “Lab” is closer to the server, it isn’t regularly online. The PPS Polarity is set to non-inverted, the only selection which works. We have more of the NOVUS GPSDO’s, I will try it to see if it remains in-sync with the other. Perhaps the BG7TBL oscillator has an issue. I know that one of his models had an issue with floating point math and didn’t report time accurately, I believe it was corrected in mine, but, maybe not.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
From: App_rpt-users <app_rpt-users-bounces at lists.allstarlink.org <mailto:app_rpt-users-bounces at lists.allstarlink.org> > On Behalf Of Chuck Henderson
Sent: Tuesday, June 12, 2018 3:35 AM
To: Users of Asterisk app_rpt <app_rpt-users at lists.allstarlink.org <mailto:app_rpt-users at lists.allstarlink.org> >
Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
Which one is the master? From the looks of it I would say that "lab" should be the master as it appears to be the first one to arrive at the server so must have the lowest network delay or they are inaccurate. Inaccuracy could be caused by having the PPS polarity wrong in the RTCM and are setting time based on the trailing edge of the pulse rather than the leading edge.
I am also using a Raspberry Pi 3B (not plus)
The master should be the first one to get a time packet to the server, or they should all be tied exactly.
Chuck
On Mon, Jun 11, 2018 at 8:46 PM Kevin Babich <kevin.babich at gmail.com <mailto:kevin.babich at gmail.com> > wrote:
Chuck,
A couple facts:
The network has very low latency (sub ms) across all nodes on the VLAN.
The site labeled LAB has a lab quality GPSDO, by NOVUS Power.
The other sites utilize the GPSDO, by BG7TBL, as available on eBay, all are the 2016 vintage.
Question:
Is the reported system time exactly as reported by the timebase? If so, it would appear I have an issue with the accuracy of the GPSDO NMEA output.
Here is an example of the output from my implementation:
GPSTime (Lab): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.220000
DrainTime: 06/11/18 20:16:57.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.71
GPSTime (Valparaiso): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.06W, Elev: 299.5
GPSTime (Crown_Point): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.6
GPSTime (LaPorte): 06/11/18 20:16:57.000000000
SysTime: 06/11/18 20:16:57.280000
DrainTime: 06/11/18 20:16:57.220
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.6
GPSTime (Lab): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.220000
DrainTime: 06/11/18 20:17:19.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 233.38
GPSTime (Valparaiso): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.260000
DrainTime: 06/11/18 20:17:19.220
Got GPS (Valparaiso): Lat: 4131.82N, Lon: 08702.05W, Elev: 293.7
GPSTime (Crown_Point): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 217.5
GPSTime (LaPorte): 06/11/18 20:17:19.000000000
SysTime: 06/11/18 20:17:19.280000
DrainTime: 06/11/18 20:17:19.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 324.1
Results AFTER resetting the MASTER:
GPSTime (Lab): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.220000
DrainTime: 06/11/18 20:31:23.180
Got GPS (Lab): Lat: 4126.20N, Lon: 08654.13W, Elev: 231.86
GPSTime (Valparaiso): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Valparaiso): Lat: 4131.81N, Lon: 08702.05W, Elev: 301.8
GPSTime (LaPorte): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (LaPorte): Lat: 4139.38N, Lon: 08645.96W, Elev: 327.7
GPSTime (Crown_Point): 06/11/18 20:31:23.000000000
SysTime: 06/11/18 20:31:23.280000
DrainTime: 06/11/18 20:31:23.240
Got GPS (Crown_Point): Lat: 4121.15N, Lon: 08724.18W, Elev: 218.5
Thoughts?
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
From: App_rpt-users <app_rpt-users-bounces at lists.allstarlink.org <mailto:app_rpt-users-bounces at lists.allstarlink.org> > On Behalf Of Chuck Henderson
Sent: Monday, June 4, 2018 4:51 AM
To: Users of Asterisk app_rpt <app_rpt-users at lists.allstarlink.org <mailto:app_rpt-users at lists.allstarlink.org> >
Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Kevin,
I just checked 3 of my RTCM's and got the following results.
RTCM1:
Current Time: Mon Jun 04, 2018 08:55:43.640
Last Rx Pkt System time: 06/04/2018 02:55:56.040, diff: 21587600 msec
Last Rx Pkt Timestamp time: 06/04/2018 02:55:56.040, diff: 0 msec
Last Rx Pkt index: 1080, inbounds: 1
RTCM2:
Current Time: Mon Jun 04, 2018 08:58:20.300
Last Rx Pkt System time: <System Time Not Set>, diff: -905657056 msec
Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec
Last Rx Pkt index: 0, inbounds: 0
RTCM3:
Current Time: Mon Jun 04, 2018 08:58:48.060
Last Rx Pkt System time: <System Time Not Set>, diff: -905629296 msec
Last Rx Pkt Timestamp time: <System Time Not Set>, diff: 0 msec
Last Rx Pkt index: 0, inbounds: 0
I would not be concerned about the times. They are not important.
A better thing to look at is in the Asterisk CLI issue the command
voter debug level 3
Then expect output like the following:
GPSTime (South): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:16.000000000
SysTime: 06/04/18 04:07:16.000000
DrainTime: 06/04/18 04:07:15.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (East): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:17.000000000
SysTime: 06/04/18 04:07:17.000000
DrainTime: 06/04/18 04:07:16.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
GPSTime (South): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:18.000000000
SysTime: 06/04/18 04:07:18.000000
DrainTime: 06/04/18 04:07:17.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
GPSTime (North): 06/04/18 04:07:19.000000000
SysTime: 06/04/18 04:07:19.000000
DrainTime: 06/04/18 04:07:18.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
sending KEEPALIVE (GPS) packet to client East digest 99479120
GPSTime (East): 06/04/18 04:07:19.000000000
sending KEEPALIVE (GPS) packet to client North digest dd455bb0
SysTime: 06/04/18 04:07:19.000000
sending KEEPALIVE (GPS) packet to client South digest 4a396b22
DrainTime: 06/04/18 04:07:18.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (North): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.960
Got GPS (North): Lat: 4030.79N, Lon: 08857.99W, Elev: 245.4
GPSTime (East): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (East): Lat: 4030.02N, Lon: 08855.92W, Elev: 250.2
GPSTime (South): 06/04/18 04:07:20.000000000
SysTime: 06/04/18 04:07:20.000000
DrainTime: 06/04/18 04:07:19.980
Got GPS (South): Lat: 4020.80N, Lon: 08856.35W, Elev: 225.7
Ignore the lines I marked in green and only be concerned with the lines that have "Time" in them.
Notice that each RTCM has 3 timestamp lines that are mostly together and they should be within about 40 ms of each other and should also be within about 40ms of the other RTCMs set of lines for that second. (this is with no signals present and with a very low network latency between RTCMs and the server).
If you find that one RTCM is many ms out of line with the others, I recommend rebooting that one and/or the master one.
I also recommend not being logged into the RTCMs via telnet except to make setting changes. Having an active telnet session to the RTCM can have a negative impact on it's ability to maintain time sync lock. Having active output on that telnet session will have a negative impact on it's ability to maintain time sync lock.
If the above does not solve your problem then try increasing your tx buffer setting to 1400 and see if that eliminates your problem. But do not evaluate if the problem is solved while you have an active telnet session to the RTCM.
If you still have a problem then send the output of your "voter debug level 3" (with no signals present and no active telnet sessions to RTCMs).
Chuck
On Thu, May 31, 2018 at 7:30 PM, Kevin Babich <kevin.babich at gmail.com <mailto:kevin.babich at gmail.com> > wrote:
Hayden,
When running DEBUG = 32, I see the following, but, only on occasion, say every minute or so:
06/01/2018 00:22:12.240 Warning: GPS Data time period elapsed
GPS-DEBUG: $GPRMC,002213.00,A,4131.82046,N,08702.05712,W,0.027,,010618,,,D*6D
GPS-DEBUG: mon: 5, gps_time: 1527812533, ctime: Fri Jun 1 00:22:13 2018
I suspect this has to do with the receiver losing lock. However, it doesn’t make sense as the receiver reports a strong signal and doesn’t show signs of loss of signal on the front alarm panel. The GPS antenna is designed with high RF immunity and is at a site with little activity.
Kindest Regards,
Kevin Babich | N9IAA
Valparaiso, IN 46383
From: App_rpt-users <app_rpt-users-bounces at lists.allstarlink.org <mailto:app_rpt-users-bounces at lists.allstarlink.org> > On Behalf Of Hayden Honeywood
Sent: Thursday, May 31, 2018 5:56 PM
To: app_rpt-users at lists.allstarlink.org <mailto:app_rpt-users at lists.allstarlink.org>
Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
Is the time in UTC? How far off is the system time?
Have you done a full calibration of each board?
What baud rate do you run on your GPS's?
I tend to use 9600 baud and have had no issues with the GPS's I have used.
There is various GPS debug tools available in the RTCM. They are documented here -
https://wiki.allstarlink.org/wiki/RTCM_Client#GPS
Can you possibly try injecting a 9.6MHz signal in place of the crystal on board? Perhaps you have a variant of James KI0KN's problem where his crystals were off frequency.
I can't comment on that as I built all my boards and they have external frequency references.
Regards
Hayden VK7HH
From: App_rpt-users <app_rpt-users-bounces at lists.allstarlink.org <mailto:app_rpt-users-bounces at lists.allstarlink.org> > On Behalf Of Hayden Honeywood
Sent: Wednesday, May 30, 2018 5:34 PM
To: app_rpt-users at lists.allstarlink.org <mailto:app_rpt-users at lists.allstarlink.org>
Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds] ***Updated***
What is your values in voter.conf for RX buffer (960) and in the RTCM client for TX buffer (480). Are all RTCM TX buffers set to the same value? Yes, 480. These values were based upon the math Jim provided, which is a function of maximum latencies on the network. We rarely have greater than 1ms to any host on the network.
Are all the GPS's in a location where they cannot be swamped with RF? Yes, and they utilize PCTEL antennas with filtering for use in RF dense environments.
Some users have had success with a dedicated RTCM as the master timing source (with no radio connected). Has it been suspected that an RTCM doesn’t have the horsepower to handle both functions contemporaneously?
Regards
Hayden VK7HH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.allstarlink.org/pipermail/app_rpt-users/attachments/20180530/24a4675d/attachment-0001.html>
------------------------------
Message: 3
Date: Wed, 30 May 2018 19:51:43 -0500
From: "Kevin Babich" <kevin.babich at gmail.com <mailto:kevin.babich at gmail.com> >
To: "'Users of Asterisk app_rpt'"
<app_rpt-users at lists.allstarlink.org <mailto:app_rpt-users at lists.allstarlink.org> >
Subject: Re: [App_rpt-users] RTCM [RX Packet Out of Bounds]
***Updated***
Message-ID: <010001d3f879$8c2b3710$a481a530$@gmail.com <http://gmail.com> >
Content-Type: text/plain; charset="utf-8"
Hayden,
Do you know what exactly “RX Packet Out of Bounds” refers to? I can infer its meaning, but, one can’t be certain. Any thoughts why the system time is always so far off? It has always been, since I began experimenting with the RTCM in 2012. This is consistent across many RTCM’s.
_______________________________________________
App_rpt-users mailing list
App_rpt-users at lists.allstarlink.org <mailto:App_rpt-users at lists.allstarlink.org>
http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button"
You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
_______________________________________________
App_rpt-users mailing list
App_rpt-users at lists.allstarlink.org <mailto:App_rpt-users at lists.allstarlink.org>
http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button"
You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
_______________________________________________
App_rpt-users mailing list
App_rpt-users at lists.allstarlink.org <mailto:App_rpt-users at lists.allstarlink.org>
http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button"
You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
_______________________________________________
App_rpt-users mailing list
App_rpt-users at lists.allstarlink.org <mailto:App_rpt-users at lists.allstarlink.org>
http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button"
You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
--
Tim WD6AWP
_______________________________________________
App_rpt-users mailing list
App_rpt-users at lists.allstarlink.org <mailto:App_rpt-users at lists.allstarlink.org>
http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button"
You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.keekles.org/pipermail/app_rpt-users/attachments/20180613/a2d6f523/attachment.html>
More information about the App_rpt-users
mailing list