5 ms·

Networking problems can manifest in odd ways. One of my favorite tech support calls was when an app I supported would consistently fail to load for a specific s

Networking problems can manifest in odd ways. One of my favorite tech support calls was when an app I supported would consistently fail to load for a specific set of people in Northern Wisconsin who all worked for the same company but worked from home in different towns. They escalated the problem and "proved" that it was our app - every other site on the web worked just fine, but ours would not load. I don't fault their logic, but I knew our site was up and running just fine. Well, days of troubleshooting later and we found a common network segment they all hit somewhere in Wisconsin, and the owner of that physical network found a bad piece of hardware on a telephone pole. I forget the details of what the hardware was or why it threw a fit with our site but not others... but the bigger point is that there are many layers in a network, so while software might be the problem... it also might not. So my approach has always been that if it makes no sense for software to be causing an observed problem - investigate the other layers.

ungamedplayer 2y ago
I remember having similar issues with one customer who was getting denial of service every time they visited a site with a banned word. The word was fetched with an Ajax request (non https) and it would terminate any connections from the Mac address for a few hours. There was some Kind of adult filtering in place. Took far too long to figure out what was happening.
ay 2y ago
In a similar story, in my TAC support days (I supported the PIX firewall) I got an odd case: an FTP transfer of a particular file always stopped for a customer. At the same place. No other files were affected. I knew that there was nothing to interfere with an active data transfer in a PIX but nonetheless. I asked him to take a sniffer trace outside the PIX. Indeed, one TCP segment just kept getting dropped and retransmitted. I extracted the data from that particular segment, zipped it as a file, and asked him to try transferring this, much shorter file. It failed. And this was when I spotted that each time we’d attempt to transfer the file, the CRC errors on the receiving PIX interface incremented by one. Replacing the cable connecting to that interface had alleviated the problem. Whether it was a cable quality or a cable just not plugged correctly, will remain a mystery, but that case was helpful to always remember that somewhere low enough everything digital is still analog. And yes, having layers in the architecture is very good.
- heavenlyblue 2y ago
  That wouldn't work anymore as connections are encrypted
  - ay 2y ago
    Yeah, since the now the data is essentially random, probably the problem won’t even be possible to happen …
RockRobotRock 2y ago
time to post the old favorite: https://www.ibiblio.org/harris/500milemail.html https://www.ibiblio.org/harris/500milemail.html

ungamedplayer 2y ago

I remember having similar issues with one customer who was getting denial of service every time they visited a site with a banned word. The word was fetched with an Ajax request (non https) and it would terminate any connections from the Mac address for a few hours. There was some Kind of adult filtering in place. Took far too long to figure out what was happening.

ay 2y ago

In a similar story, in my TAC support days (I supported the PIX firewall) I got an odd case: an FTP transfer of a particular file always stopped for a customer. At the same place. No other files were affected. I knew that there was nothing to interfere with an active data transfer in a PIX but nonetheless. I asked him to take a sniffer trace outside the PIX. Indeed, one TCP segment just kept getting dropped and retransmitted. I extracted the data from that particular segment, zipped it as a file, and asked him to try transferring this, much shorter file. It failed. And this was when I spotted that each time we’d attempt to transfer the file, the CRC errors on the receiving PIX interface incremented by one. Replacing the cable connecting to that interface had alleviated the problem. Whether it was a cable quality or a cable just not plugged correctly, will remain a mystery, but that case was helpful to always remember that somewhere low enough everything digital is still analog. And yes, having layers in the architecture is very good.

heavenlyblue 2y ago
That wouldn't work anymore as connections are encrypted
- ay 2y ago
  Yeah, since the now the data is essentially random, probably the problem won’t even be possible to happen …

RockRobotRock 2y ago

time to post the old favorite: https://www.ibiblio.org/harris/500milemail.html https://www.ibiblio.org/harris/500milemail.html