RTCM Inbound (Eth Rx) packet out of bounds
More testing last night and today. I set the TX Buffer to 1200 last night, rebooted both RTCM units and it was working fine last night. Switched off one of the sites (at home) overnight. Came back this morning, powered up and getting the packet out of bounds issues again. Changed up to 1600 and rebooted both units, worked again. buflen set to 250. Is it just a matter of increasing these up and up until it's stable even after a power reset? I've been swapping out ethernet cables too just checking to make sure that is not the issue... Could still be a hardware problem. Next thing to change is the port from 667 to make sure there is no limiting down at the ISP level.
Have you quantified your network parameters and set your buffers using the tuning procedure? http://docs.allstarlink.org/drupal/node/108 Where is your asterisk/allstar server, in relation to your MASTER site? Per the votersystem.pdf: There must be a VOTER (RTCM) board on the same LAN (very low latency) as the Asterisk server implementing the “master site”, which acts as the Master Timing source. This allows chan_voter to have a consistent, reliable, accurate timing source with which the timing information from all other inbound packets are compared and appropriately processed, and from which to generate accurate timing information for time-consistent transmission purposes. Cheers! Lee On Sat, Sep 23, 2017 at 5:41 PM, Hayden Honeywood <haydenph91@gmail.com> wrote:
More testing last night and today.
I set the TX Buffer to 1200 last night, rebooted both RTCM units and it was working fine last night.
Switched off one of the sites (at home) overnight. Came back this morning, powered up and getting the packet out of bounds issues again. Changed up to 1600 and rebooted both units, worked again. buflen set to 250.
Is it just a matter of increasing these up and up until it's stable even after a power reset?
I've been swapping out ethernet cables too just checking to make sure that is not the issue... Could still be a hardware problem. Next thing to change is the port from 667 to make sure there is no limiting down at the ISP level.
_______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/ cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
-- Lee Woldanski, AScT VE7FET
Here's how I do it, seems to work... Calculate the maximum latency in network. If its less than 60 ms use 60ms for the calculations. For RTCM Tx Buffer length, take latency * 8. Minimum therefore is 480 (this value is in packets (8 packets per ms)) For buflen in voter.conf, take latency + 100. Minimum therefore is 160 (this value is in ms) The delay of the system is RTCM Tx Buffer Length /8 + tx buffer length in Asterisk. Minimum therefore is 220ms If you have voter.conf set to 250 the RTCM should be set to 1200. This should make the system able to handle latency spikes to 150ms. Note: If your GPS is weak on the RTCM which is connected to your Asterisk server it'll cause lots of problems. I've also had better luck using a dedicated RTCM for the server timing and using a different one for the radio/repeater, even if they are in the same building. I suspect the UDP TX audio stream combined with the 10Mbps 1/2 duplex RTCM ethernet causes packet collisions which messes up the server timing packets (this is a hunch at this point). Cheers, Jesse
On Sep 23, 2017, at 5:41 PM, Hayden Honeywood <haydenph91@gmail.com> wrote:
More testing last night and today.
I set the TX Buffer to 1200 last night, rebooted both RTCM units and it was working fine last night.
Switched off one of the sites (at home) overnight. Came back this morning, powered up and getting the packet out of bounds issues again. Changed up to 1600 and rebooted both units, worked again. buflen set to 250.
Is it just a matter of increasing these up and up until it's stable even after a power reset?
I've been swapping out ethernet cables too just checking to make sure that is not the issue... Could still be a hardware problem. Next thing to change is the port from 667 to make sure there is no limiting down at the ISP level. _______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
Jesse, I'm curious about your Note: in the email below. I get a lot of there messages "Voter lost master timing source!!" Do you think the two RTCM solution you mentioned would fit that? messages:[Nov 9 07:59:28] NOTICE[28554] chan_voter.c: Voter lost master timing source!! messages:[Nov 24 17:58:49] NOTICE[28554] chan_voter.c: Voter lost master timing source!! messages:[Nov 25 03:58:49] NOTICE[28554] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 4 13:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 4 14:37:07] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 6 17:00:05] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 7 17:00:54] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 13 01:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 22 07:00:05] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 22 15:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 25 13:00:04] NOTICE[28554] chan_voter.c: Voter lost master timing source!! messages.1:[Oct 28 14:59:57] NOTICE[28554] chan_voter.c: Voter lost master timing source!! messages.2:[Sep 17 17:00:05] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.2:[Sep 17 19:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!! messages.3:[Aug 15 07:00:06] NOTICE[713] chan_voter.c: Voter lost master timing source!! messages.3:[Aug 17 03:00:05] NOTICE[713] chan_voter.c: Voter lost master timing source!! messages.3:[Aug 21 15:00:06] NOTICE[713] chan_voter.c: Voter lost master timing source!! messages.3:[Aug 21 19:00:05] NOTICE[713] chan_voter.c: Voter lost master timing source!! messages.3:[Aug 30 15:00:05] NOTICE[713] chan_voter.c: Voter lost master timing source!! On Sat, Sep 23, 2017 at 7:46 PM, Jesse Lloyd <ve7lyd@gmail.com> wrote:
Here's how I do it, seems to work...
Calculate the maximum latency in network. If its less than 60 ms use 60ms for the calculations.
For RTCM Tx Buffer length, take latency * 8. Minimum therefore is 480 (this value is in packets (8 packets per ms))
For buflen in voter.conf, take latency + 100. Minimum therefore is 160 (this value is in ms)
The delay of the system is RTCM Tx Buffer Length /8 + tx buffer length in Asterisk. Minimum therefore is 220ms
If you have voter.conf set to 250 the RTCM should be set to 1200. This should make the system able to handle latency spikes to 150ms.
Note: If your GPS is weak on the RTCM which is connected to your Asterisk server it'll cause lots of problems. I've also had better luck using a dedicated RTCM for the server timing and using a different one for the radio/repeater, even if they are in the same building. I suspect the UDP TX audio stream combined with the 10Mbps 1/2 duplex RTCM ethernet causes packet collisions which messes up the server timing packets (this is a hunch at this point).
Cheers, Jesse
On Sep 23, 2017, at 5:41 PM, Hayden Honeywood <haydenph91@gmail.com> wrote:
More testing last night and today.
I set the TX Buffer to 1200 last night, rebooted both RTCM units and it was working fine last night.
Switched off one of the sites (at home) overnight. Came back this morning, powered up and getting the packet out of bounds issues again. Changed up to 1600 and rebooted both units, worked again. buflen set to 250.
Is it just a matter of increasing these up and up until it's stable even after a power reset?
I've been swapping out ethernet cables too just checking to make sure that is not the issue... Could still be a hardware problem. Next thing to change is the port from 667 to make sure there is no limiting down at the ISP level.
_______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/ cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
_______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/ cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
-- -- Tim
Yes. Using one dedicated for timing has worked better for me. I suspect that if you get a packet collision using the 10 Mbps 1/2 duplex then the delayed packet used for timing breaks things... this is a guess at this point. Jesse On Wed, Nov 29, 2017 at 4:04 PM, Tim Sawyer <tisawyer@gmail.com> wrote:
Jesse,
I'm curious about your Note: in the email below. I get a lot of there messages "Voter lost master timing source!!" Do you think the two RTCM solution you mentioned would fit that?
messages:[Nov 9 07:59:28] NOTICE[28554] chan_voter.c: Voter lost master timing source!!
messages:[Nov 24 17:58:49] NOTICE[28554] chan_voter.c: Voter lost master timing source!!
messages:[Nov 25 03:58:49] NOTICE[28554] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 4 13:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 4 14:37:07] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 6 17:00:05] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 7 17:00:54] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 13 01:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 22 07:00:05] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 22 15:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 25 13:00:04] NOTICE[28554] chan_voter.c: Voter lost master timing source!!
messages.1:[Oct 28 14:59:57] NOTICE[28554] chan_voter.c: Voter lost master timing source!!
messages.2:[Sep 17 17:00:05] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.2:[Sep 17 19:00:06] NOTICE[23633] chan_voter.c: Voter lost master timing source!!
messages.3:[Aug 15 07:00:06] NOTICE[713] chan_voter.c: Voter lost master timing source!!
messages.3:[Aug 17 03:00:05] NOTICE[713] chan_voter.c: Voter lost master timing source!!
messages.3:[Aug 21 15:00:06] NOTICE[713] chan_voter.c: Voter lost master timing source!!
messages.3:[Aug 21 19:00:05] NOTICE[713] chan_voter.c: Voter lost master timing source!!
messages.3:[Aug 30 15:00:05] NOTICE[713] chan_voter.c: Voter lost master timing source!!
On Sat, Sep 23, 2017 at 7:46 PM, Jesse Lloyd <ve7lyd@gmail.com> wrote:
Here's how I do it, seems to work...
Calculate the maximum latency in network. If its less than 60 ms use 60ms for the calculations.
For RTCM Tx Buffer length, take latency * 8. Minimum therefore is 480 (this value is in packets (8 packets per ms))
For buflen in voter.conf, take latency + 100. Minimum therefore is 160 (this value is in ms)
The delay of the system is RTCM Tx Buffer Length /8 + tx buffer length in Asterisk. Minimum therefore is 220ms
If you have voter.conf set to 250 the RTCM should be set to 1200. This should make the system able to handle latency spikes to 150ms.
Note: If your GPS is weak on the RTCM which is connected to your Asterisk server it'll cause lots of problems. I've also had better luck using a dedicated RTCM for the server timing and using a different one for the radio/repeater, even if they are in the same building. I suspect the UDP TX audio stream combined with the 10Mbps 1/2 duplex RTCM ethernet causes packet collisions which messes up the server timing packets (this is a hunch at this point).
Cheers, Jesse
On Sep 23, 2017, at 5:41 PM, Hayden Honeywood <haydenph91@gmail.com> wrote:
More testing last night and today.
I set the TX Buffer to 1200 last night, rebooted both RTCM units and it was working fine last night.
Switched off one of the sites (at home) overnight. Came back this morning, powered up and getting the packet out of bounds issues again. Changed up to 1600 and rebooted both units, worked again. buflen set to 250.
Is it just a matter of increasing these up and up until it's stable even after a power reset?
I've been swapping out ethernet cables too just checking to make sure that is not the issue... Could still be a hardware problem. Next thing to change is the port from 667 to make sure there is no limiting down at the ISP level.
_______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/c gi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
_______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/c gi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
-- -- Tim
_______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/ cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
On 11/29/17 7:07 PM, Jesse Lloyd wrote:
I suspect that if you get a packet collision using the 10 Mbps 1/2 duplex then the delayed packet used for timing breaks things... this is a guess at this point.
What kinda of switch do you have this connected to? A proper switch should eliminate any collisions. I have an EX everything plugs into, and you will need to set the RTCM to full duplex and manually configure the switch to 10/full. The RTCM doesn't do auto negotiation, and the switch will default to 10/half if it fails (this is the 802.3 standard) 73's -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
I've use just an unmanaged switch, which would fail to 10 half. A fully managed one that you can force your duplex setting on would probably work fine. Cheers, Jesse
On Nov 29, 2017, at 4:53 PM, Bryan Fields <Bryan@bryanfields.net> wrote:
On 11/29/17 7:07 PM, Jesse Lloyd wrote: I suspect that if you get a packet collision using the 10 Mbps 1/2 duplex then the delayed packet used for timing breaks things... this is a guess at this point.
What kinda of switch do you have this connected to? A proper switch should eliminate any collisions. I have an EX everything plugs into, and you will need to set the RTCM to full duplex and manually configure the switch to 10/full. The RTCM doesn't do auto negotiation, and the switch will default to 10/half if it fails (this is the 802.3 standard)
73's -- Bryan Fields
727-409-1194 - Voice http://bryanfields.net _______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
My RTCM is set to half duplex. The switch is a MikroTik router. I'll try full duplex and see what happens. So Bryan, you don't have any these "Voter lost master timing source!!" messages in your Asterisk log? On Wed, Nov 29, 2017 at 4:53 PM, Bryan Fields <Bryan@bryanfields.net> wrote:
On 11/29/17 7:07 PM, Jesse Lloyd wrote:
I suspect that if you get a packet collision using the 10 Mbps 1/2 duplex then the delayed packet used for timing breaks things... this is a guess at this point.
What kinda of switch do you have this connected to? A proper switch should eliminate any collisions. I have an EX everything plugs into, and you will need to set the RTCM to full duplex and manually configure the switch to 10/full. The RTCM doesn't do auto negotiation, and the switch will default to 10/half if it fails (this is the 802.3 standard)
73's -- Bryan Fields
727-409-1194 - Voice http://bryanfields.net _______________________________________________ App_rpt-users mailing list App_rpt-users@lists.allstarlink.org http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users
To unsubscribe from this list please visit http://lists.allstarlink.org/ cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button" You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.
-- -- Tim
participants (5)
-
Bryan Fields -
Hayden Honeywood -
Jesse Lloyd -
Lee Woldanski -
Tim Sawyer