flak rss random

comcast ping times

Despite their wonky customer service, I have generally been happy with Comcast’s technical service. Occasionally though, I have to question what in the world is going on with their network. Recently I noticed that my internet connection would alternate between working and not on roughly a two minute interval. One minute things work fine, the next minute nothing works at all, the next minute everything is fine. During the blackout minutes, making a new connection would timeout, but established ssh connections would remain up, but nothing would happen until the blackout was over. Here are a few pings I sent out (2014-02-13).

64 bytes from icmp_seq=0 ttl=43 time=98089.012 ms
64 bytes from icmp_seq=2 ttl=43 time=96073.905 ms
64 bytes from icmp_seq=1 ttl=43 time=97083.976 ms
64 bytes from icmp_seq=3 ttl=43 time=95064.000 ms
64 bytes from icmp_seq=4 ttl=43 time=94053.992 ms
64 bytes from icmp_seq=5 ttl=43 time=93044.984 ms
64 bytes from icmp_seq=7 ttl=43 time=91025.183 ms
64 bytes from icmp_seq=6 ttl=43 time=92035.340 ms
64 bytes from icmp_seq=9 ttl=43 time=89005.105 ms
64 bytes from icmp_seq=10 ttl=43 time=87995.872 ms
64 bytes from icmp_seq=8 ttl=43 time=90020.101 ms

Woah, that’s some latency. A few observations. We didn’t lose any packets. They didn’t quite arrive in sequence, but they’re not totally out of order (in fact, 8 is the only packet to be more than one out of place). And they all arrived at the same time. It’s like they were taped to the leg of a carrier pigeon and then when enough packets were queued up somebody opened the cage door.

Anything that works asynchronously, like email, is reasonable, but the blackout minutes makes web browsing abominably painful. I’ve experienced this twice so far in a period of a few months. It lasts for about 24-36 hours. The more recent occurrence ended just as I was setting up some better testing and logging, so I can’t yet determine whether it’s malice or incompetence.

Guessing, it looks like a misconfigured redundant router pair, only one of which can send packets. The broken router simply queues packets until it times out and fails over, then pushes everything down the working pipe. Then for some reason, the system fails back to the broken router.

Another instance from 2015-06-25:

64 bytes from icmp_seq=12 ttl=55 time=9109.139 ms
64 bytes from icmp_seq=13 ttl=55 time=8099.379 ms
64 bytes from icmp_seq=14 ttl=55 time=7089.483 ms
64 bytes from icmp_seq=15 ttl=55 time=6079.572 ms
64 bytes from icmp_seq=16 ttl=55 time=5069.646 ms
64 bytes from icmp_seq=17 ttl=55 time=4059.737 ms
64 bytes from icmp_seq=18 ttl=55 time=3048.371 ms
64 bytes from icmp_seq=19 ttl=55 time=2039.933 ms
64 bytes from icmp_seq=20 ttl=55 time=1029.060 ms

64 bytes from icmp_seq=21 ttl=55 time=9378.332 ms
64 bytes from icmp_seq=22 ttl=55 time=8366.536 ms
64 bytes from icmp_seq=23 ttl=55 time=7356.391 ms
64 bytes from icmp_seq=24 ttl=55 time=6346.711 ms
64 bytes from icmp_seq=25 ttl=55 time=5336.931 ms
64 bytes from icmp_seq=26 ttl=55 time=4327.367 ms
64 bytes from icmp_seq=27 ttl=55 time=3316.542 ms
64 bytes from icmp_seq=28 ttl=55 time=2306.854 ms
64 bytes from icmp_seq=29 ttl=55 time=1296.952 ms
64 bytes from icmp_seq=30 ttl=55 time=289.789 ms

Now the internet is on a ten second burst cycle.

Posted 13 Feb 2014 21:49 by tedu Updated: 25 Jun 2015 17:56