Sorry for the double email forgot to add some things...

Sounds good Bryan! Also special shout out to the rest of the team as well.. I think i might have an idea what maybe going on in regards to your question Benjamin.. I have recently noticed heavier than normal latency on the iax ports I am not sure if service providers are doing some funny business with 4569 or what.. But what i recommend is if your having connectivity issues download mtr on to your allstarlink node..  You can install it by typing apt-get install mtr then use this command as an example mtr -rwc 100 ip-p-host-Address

example command would be:
mtr -rwc 100 lorentedford.com

Now this will show you percentages of packet loss for each hop this is generally a good place to start in troubleshooting if your having issues..

Providing the output of the above can be very useful and assist others in seeing what type of network congestion is going on where around the country.. In the future some of us might have to look into consistent VPN services between major hubs to maintain stable connectivity and routing.. Examples being multiple hub systems where you have geographically located allstarlink hubs around the world..

Just a thought

===========================================================================
This is my added on section i forgot to post in the original email

I forgot to add how to do commandline speed testing..
[root@webserver ~]# nano speedtest.sh
#!/bin/bash
wget https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py --no-check-certificate
chmod a+rx speedtest.py
mv speedtest.py /usr/local/bin/
chown root:root /usr/local/bin/speedtest.py
/usr/local/bin/speedtest.py

After the above has been ran at least once I comment out everything to make it look like this..

[root@webserver ~]# cat speedtest.sh
#!/bin/bash
#wget https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py --no-check-certificate
#chmod a+rx speedtest.py
#mv speedtest.py /usr/local/bin/
#chown root:root /usr/local/bin/speedtest.py
/usr/local/bin/speedtest.py

Now to run the speedtest just type sh speedtest.sh you can even have this save the output into a text file later if you wish..

Also below is an example of what an MTR report will look like 
loren@loren-Inspiron-1501:~$ mtr -rwc 100 lorentedford.com
Start: Tue Jun 27 11:25:43 2017
HOST: loren-Inspiron-1501             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- VerizonBRV                       0.0%   100    1.6   0.8   0.6   3.1   0.2
  2.|-- 113.sub-66-174-52.myvzw.com      0.0%   100   35.7  38.6  32.1  52.2   4.2
  3.|-- 34.sub-69-83-89.myvzw.com        0.0%   100   39.5  39.6  24.2  53.0   4.8
  4.|-- 49.sub-69-83-89.myvzw.com        0.0%   100   37.4  39.1  26.1  54.2   4.6
  5.|-- 194.sub-69-83-88.myvzw.com       0.0%   100   39.7  40.7  25.6  55.3   5.1
  6.|-- ???                             100.0   100    0.0   0.0   0.0   0.0   0.0
  7.|-- ???                             100.0   100    0.0   0.0   0.0   0.0   0.0
  8.|-- 101.sub-66-174-49.myvzw.com      0.0%   100   42.1  41.3  29.5  64.5   6.0
  9.|-- 0.et-2-0-0.GW10.CHI2.ALTER.NET   0.0%   100   34.8  40.6  24.2  66.9   7.4
 10.|-- 0.et-11-3-0.GW8.CHI13.ALTER.NET  0.0%   100   38.0  41.5  26.5  69.2   5.9
 11.|-- ???                             100.0   100    0.0   0.0   0.0   0.0   0.0
 12.|-- nyk-bb3-link.telia.net           0.0%   100   56.0  61.8  52.9 232.9  17.9
 13.|-- nyk-b2-link.telia.net            0.0%   100   48.9  61.4  47.5 174.1  13.3
 14.|-- ???                             100.0   100    0.0   0.0   0.0   0.0   0.0
loren@loren-Inspiron-1501:~$





On Tue, Jun 27, 2017 at 11:24 AM, Loren Tedford <lorentedford@gmail.com> wrote:
Sounds good Bryan! Also special shout out to the rest of the team as well.. I think i might have an idea what maybe going on in regards to your question Benjamin.. I have recently noticed heavier than normal latency on the iax ports I am not sure if service providers are doing some funny business with 4569 or what.. But what i recommend is if your having connectivity issues download mtr on to your allstarlink node..  You can install it by typing apt-get install mtr then use this command as an example mtr -rwc 100 ip-p-host-Address

example command would be:
mtr -rwc 100 lorentedford.com

Now this will show you percentages of packet loss for each hop this is generally a good place to start in troubleshooting if your having issues..

Providing the output of the above can be very useful and assist others in seeing what type of network congestion is going on where around the country.. In the future some of us might have to look into consistent VPN services between major hubs to maintain stable connectivity and routing.. Examples being multiple hub systems where you have geographically located allstarlink hubs around the world..

Just a thought


On Tue, Jun 27, 2017 at 10:57 AM, Bryan Fields <Bryan@bryanfields.net> wrote:
On 6/26/17 9:10 PM, Benjamin Naber wrote:
> Over the course of the last six months or so, I have noticed there have
> been some issues with allstarlink.
>
> Either the allstarlink website doesn't work, connections are radanomly
> dropped on known high-reliability networks and connected equipment.
> All without apparent cause.

When and where?

Allstarlink.org is and has been online and stable for some time.

Docs.allstarlink.org had a network outage recently due to a dead switch.  It
was rectified about 5-6 hours later by our network vendor.

> Again today, for no apparent reason, all links on several systems in
> this area were dropped, and were not able to connect to anyone. Some of
> our nodes have "direct access" to other nodes specified in the rpt.conf,
> and those connections worked fine.

Again, when and where?  Connections from node to node are direct, the only
thing ASL does is build a database and push that to the nodes every 10 mins or
so.

If your nodes are listed as online, but they cannot talk, there is a network
issue unrelated to ASL.


> When a node cannot connect to node 2000, or some other random one, there
> is an issue.

This sort of "error" report is lacking.  You would need to give the errors,
dates/times, source node IP and AS path if you can provide that.

> All nodes in this area have different ISPs, so it rules out the
> possibility of ISP issue.

No, it makes it less likely, but it in no way rules it out.  Are you doing NAT
or is each node on it's own IP?

> Other folks I have talked to across the country have stated they have
> noticed similar issues.

This is nothing but scuttlebutt without evidence.

> Has anyone noticed this, and not said anything, or what is going on?

In the last 6 months ASL has lost Jim Dixon, formally incorporated as a
non-profit organization and been forced to document a number of things which
Jim had in his head.

"The death of God left the angels in a strange position."

We had to identify the ASL infrastructure, which was spread out over a number
of different locations.  We've done this and have access to everything, and
backups in case anything fails.  We've assembled an infrastructure team and
have an architecture we're building into (docker).  Over the next few months
we're going to move servers one by one into this environment.

The mailing lists have been moved to a high performance server with real spam
filtering.  Nagios is watching everything and we know in 5-10 minutes when
there is an outage of a service.

This is a huge undertaking.

Tim's working on a new website, and Steve's been running everything else
including development.  Oh, and all the source code is on github now too.

If there's network issues we _want_ to know about them, but it must be in a
detailed manner.  I've responded to people on reddit and said to post details
over on this list.  Without detailed logs and reports we cannot do anything to
confirm them.

73's
--
Bryan Fields

727-409-1194 - Voice
http://bryanfields.net
_______________________________________________
App_rpt-users mailing list
App_rpt-users@lists.allstarlink.org
http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users

To unsubscribe from this list please visit http://lists.allstarlink.org/cgi-bin/mailman/listinfo/app_rpt-users and scroll down to the bottom of the page. Enter your email address and press the "Unsubscribe or edit options button"
You do not need a password to unsubscribe, you can do it via email confirmation. If you have trouble unsubscribing, please send a message to the list detailing the problem.