5 ms·

Trying to explain this issue to AT&T support is like trying to convince a doctor you're the only person on earth with a particular disease. Even explaining the

Trying to explain this issue to AT&T support is like trying to convince a doctor you're the only person on earth with a particular disease. Even explaining the issue is hard. It's not an outage, my internet isn't out, it's intermittently wrong. The phone support agents aren't prepared for this, and I can't find any way to escalate or speak to a network engineer. I feel like if I spoke to the right engineer, there'd be a ticket on this and they'd roll at truck to their facilities or the IPX within an hour. It's a major network issue to flip bits, it's costing them bandwidth with retransmits and could be breaking SLAs with their business customers. On the phone the most they could do was roll a truck to me.

deleted 6y ago
[deleted]
dbt00 6y ago
I talked to somebody and he told me he had talked to his "IT department" and they're working on it but who the fuck knows.
warent 6y ago
The obnoxious thing to me is the hubris that must be behind this. Either they considered it and decided they would never encounter a system error like this and refused to implement an escalation route, or they never even considered it. Or perhaps even worse than that, maybe they considered it, decided it was possible, but just don't care because of their insane borderline monopoly. I don't understand how internet companies provide such consistently awful service. Slightly off topic story, I've recently changed to another provider called Starry, and they force you to have a second router in front of your own router which they claim "decodes" their stream between the modem. I don't know the real reason but I'm pretty sure that's not it. If you plug their modem directly into a non-Starry router, the router just doesn't detect a connection. One day, I tried to torrent something, and my internet would immediately get throttled to 0mbps. After investigating I found out that their router had a custom OS which hid a firewall and various security settings. Amusingly you could still access those settings if you just manually entered the page names into the address bar. Now all their stupid settings are disabled and I just feel badly for all the folks who use their service and don't have the savvy I do to actually get what they're paying for.
- dudul 6y ago
  Wait so you have your main cable plugged into their modem, their router plugged into their modem, and then your own router plugged into their router?
  - warent 6y ago
    Yes exactly. Their router has a LAN with my router as the only other device, which it's bridged with, and then my router has the true home LAN. A weird side effect of this is that I'm not using the 192.168.x.x range like usual (because that's what theirs is using), but instead the 10.0.x.x range
    - _rs 6y ago
      So are you bridged then or is it really a double nat?
      - warent 6y ago
        this is really where my limited knowledge of networking shows. I'm not entirely sure but I want to say both or double nat. There's two networks, but my router thinks theirs is a modem and is connected via the "Internet" port, not just a normal device port
        _rs 6y ago
        Ah ok - sounds like a double NAT then, not a bridge. Edit: to go into more detail, their router is acting as a NAT for your public IP, giving you your first subnet, and then your router is getting a single IP on that subnet and creating a NAT where your devices all get IPs. In a bridge there would only be 1 IP space behind a single NAT. In your case with a double NAT a lot of consumer things might not work (like UPnP) and port forwarding would require you to add rules to both routers.
        warent 6y ago
deleted 6y ago
[deleted]
nodesocket 6y ago
From a business standpoint, it's hard to justify paying support to be technical enough to diagnose an issue such as this. Let's be honest, even senior network engineers would have a hard time debugging and diagnosing this. AT&T doesn't want to pay support staff six figure salaries and I assume most senior network engineers don't want to be support agents (customer facing). AT&T (though applies to lost of companies) probably need a "unicorn" role of a very technical person that is paid as such, but able to interface with customers on specific highly technical issues.
gamegoblin 6y ago
When I first moved to Seattle there was a great local ISP called CondoInternet that mainly specialized in high-density downtown buildings. I was once having some packet loss issues and called their support line. I assume the company was really small at the time, because the guy who answered the phone was clearly a network engineer who knew the system inside and out. I read him a couple of traceroutes over the phone and we resolved the issue within minutes. I have never experienced such perfect tech support before or after that with any other ISP.
- Sebguer 6y ago
  They got bought by Wave, who still seem to have pretty solid support? At least over Twitter, it felt like I was talking to someone competent. I haven't had any major issues since installation, though, so maybe I just got lucky.
- cpach 6y ago
  That sounds amazing!
- nitrogen 6y ago
  XMission in Salt Lake City had support like that, back when DSL unbundling was a thing (2006).
- neelc 6y ago
  And when Wave bought them, Wave put incompetent people in charge. I moved to a Seattle suburb and had "Wave G" (post-Wave CondoInternet). I had throughput issues, first on my apartment. That got resolved via Reddit. Then the backbone is slow from time to time which persisted. Sometimes I get a Gigabit, other times I only get 10-20 Mbps. What did Wave say? Let's bring a tech out. I told them "it's a backbone issue" and they didn't understand me. I don't know how I would explain how networking works to a field tech who only knows how to enable a port or run basic diagnostics. Another time, I tripped Port Security, and told them they can re-enable my port. Wave said "we need to bring a tech out" as if that will magically solve everything. This is made worse by my apartment's exclusivity deals: only Comcast and Wave, nothing else. Ziply Fiber (bought Frontier) and Atlas Networks were denied. Bigger companies like Verizon FiOS in NYC had better tech support, I could still get hold of a Level 2 tech for a much smaller issue (and not even on FiOS). Wave G makes AT&T's forced router (but not the flipping bits) seem decent in comparison. I don't want Comcast but I'll happily take AT&T Fiber over Wave G even if it means trading my pfSense box for a crappy AT&T gateway (worse case scenario I could bypass or root the gateway, or pay $15 for static IPs). I fortunately moved (for another reason) and have Google's Webpass. Webpass may not give me a full Gigabit, but it's usually 400-800 Mbps all the time and not 10-20 Mbps 95% of the time with an occasional Gigabit. I wish CondoInternet sold to Webpass instead of Wave (I don't want a merger now, but still). Surprisingly, I may have preferred CenturyLink Fiber if available mainly for GPON over a 60GHz PtP microwave link (less oversubscription!), well unless PPPoE+6rd kills my pfSense. But Webpass works pretty darn well, and I think I'll stay unless CenturyLink suddenly gives me 2 Gbps FTTH or something, or I move.
iandanforth 6y ago
Have you tried saying "shibboleet"?
- _whiteCaps_ 6y ago
  I think you're being unfairly downvoted by people who haven't seen this xkcd: https://xkcd.com/806/ https://xkcd.com/806/
  - closeparen 6y ago
    The real life version of “shibboleet” is your Certified Partner ID number and a serial number with a valid support contract. When I worked for a VAR I could upload logs to Cisco and get experimental patches back. Call up HP, tell them I want an RMA, and they’d just do it. Night and day compared to what consumers get.
- robgolding 6y ago
  My ISP actually supports this[0], though I haven’t had cause to use it yet - they are also very reliable! [0]: https://www.aa.net.uk/broadband/why-choose-aaisp/ https://www.aa.net.uk/broadband/why-choose-aaisp/
013a 6y ago
Have you seen Parks & Rec, and remember that scene in a Home Depot where an associate walks up to Ron, asks him if he needs help with a project, and Ron responds "I know more than you"? I've pulled a variation of that on CSRs at least once, and surprisingly, it can work. Just be cordial, preempt the typical IT Support stuff they always ask, DO NOT say its intermittent (initially, to the front line CSR; if given a chance to expand the issue after escalation, then add that bit), and get technical ASAP (it doesn't hurt to throw in some parallel industry jargon). Basically, build a case where even the information you're giving them is beyond a first-line CSR playbook, and they have to escalate. "Hi there; I've been observing some erroneous TCP packet bit flipping on HTTP requests which route through one of AT&T's data center in Oakland. I've tried restarting my computer, I'm seeing the same thing on my phone, and I actually swapped my router out for a spare one I have, but its still an issue." (that last sentence exhausts literally every playbook a front-line CSR has. it sounds so easy, right? there are four variables in any front-line CSR diagnostic equation: their network, your router, wifi/ethernet, and the endpoint. you just crossed off three of the four variables in one sentence). (Wait, a data center in Oakland? How do you know this? You can tracert a bad request and geolocate the first IP outside your network, but, lets be realistic: You don't. You're fronting; demonstrating knowledge that a front-line CSR can't disprove. You may think this is misleading to whoever this gets escalated to, but it isn't; their tools are FAR more advanced than yours, and they're used to 99% of customers being incorrect idiots, so they're going to be validating and reconfirming every word you say anyway.) Ron's Parks & Rec example above is crass. But here's the magic bit: frontline CSRs generally look for an excuse to escalate, you just need to give them enough CYA to check their job as done, and the higher tier CSRs/network engineers will love you for actually knowing what you're talking about. Its a win-win; be cordial, be forceful, strut what you know.
- zanny 6y ago
  I had something like this happen on an even simpler level this last week. I got a Chase credit card but when I initially did the signup called my brother to ask him if he wanted to be on the account and it timed the sign up session out past the account creation but before finalization. I got the card eventually but now I cannot create an online account with it. I called Chase, got transferred 5 times, and then told I would need to go to a physical bank to verify my identity? to create an account. Absolutely not one of them had any clue what "a broken account exists associated with this card in your database, I can guarantee it, forward me to your technical support team" but thats all above a bank reps pay grade. The nearest Chase bank is 1.5 hours away, by the way. Probably just going to cancel the card after cashing out the sign up bonus.
reader_1000 6y ago
Ten years ago, while using ADSL, for some reason captcha images were not loading. I opened a case to ISP support and they called me back. The support guy did not believe it. He said "this is not an analog network, this is digital, you either have it or not". I said I know what the analog and digital mean and also I know this is a digital network since I am a computer engineer and I checked everything so this was a issue with connection. After a couple hours later, he called me back and said that problem was caused by modem and a driver update would fix it and it did. These were good old days, however, when we can reach someone on the phone and talk about problem. Nowadays everything is either automated response or some random person whose whole job to tell you that he/she cannot do anything about the situation.

deleted 6y ago

[deleted]

dbt00 6y ago

I talked to somebody and he told me he had talked to his "IT department" and they're working on it but who the fuck knows.

warent 6y ago

The obnoxious thing to me is the hubris that must be behind this. Either they considered it and decided they would never encounter a system error like this and refused to implement an escalation route, or they never even considered it. Or perhaps even worse than that, maybe they considered it, decided it was possible, but just don't care because of their insane borderline monopoly. I don't understand how internet companies provide such consistently awful service. Slightly off topic story, I've recently changed to another provider called Starry, and they force you to have a second router in front of your own router which they claim "decodes" their stream between the modem. I don't know the real reason but I'm pretty sure that's not it. If you plug their modem directly into a non-Starry router, the router just doesn't detect a connection. One day, I tried to torrent something, and my internet would immediately get throttled to 0mbps. After investigating I found out that their router had a custom OS which hid a firewall and various security settings. Amusingly you could still access those settings if you just manually entered the page names into the address bar. Now all their stupid settings are disabled and I just feel badly for all the folks who use their service and don't have the savvy I do to actually get what they're paying for.

dudul 6y ago
Wait so you have your main cable plugged into their modem, their router plugged into their modem, and then your own router plugged into their router?
- warent 6y ago
  Yes exactly. Their router has a LAN with my router as the only other device, which it's bridged with, and then my router has the true home LAN. A weird side effect of this is that I'm not using the 192.168.x.x range like usual (because that's what theirs is using), but instead the 10.0.x.x range
  - _rs 6y ago
    So are you bridged then or is it really a double nat?
    - warent 6y ago
      this is really where my limited knowledge of networking shows. I'm not entirely sure but I want to say both or double nat. There's two networks, but my router thinks theirs is a modem and is connected via the "Internet" port, not just a normal device port
      - _rs 6y ago
        Ah ok - sounds like a double NAT then, not a bridge. Edit: to go into more detail, their router is acting as a NAT for your public IP, giving you your first subnet, and then your router is getting a single IP on that subnet and creating a NAT where your devices all get IPs. In a bridge there would only be 1 IP space behind a single NAT. In your case with a double NAT a lot of consumer things might not work (like UPnP) and port forwarding would require you to add rules to both routers.
        warent 6y ago

deleted 6y ago

[deleted]

nodesocket 6y ago

From a business standpoint, it's hard to justify paying support to be technical enough to diagnose an issue such as this. Let's be honest, even senior network engineers would have a hard time debugging and diagnosing this. AT&T doesn't want to pay support staff six figure salaries and I assume most senior network engineers don't want to be support agents (customer facing). AT&T (though applies to lost of companies) probably need a "unicorn" role of a very technical person that is paid as such, but able to interface with customers on specific highly technical issues.

gamegoblin 6y ago

When I first moved to Seattle there was a great local ISP called CondoInternet that mainly specialized in high-density downtown buildings. I was once having some packet loss issues and called their support line. I assume the company was really small at the time, because the guy who answered the phone was clearly a network engineer who knew the system inside and out. I read him a couple of traceroutes over the phone and we resolved the issue within minutes. I have never experienced such perfect tech support before or after that with any other ISP.

Sebguer 6y ago
They got bought by Wave, who still seem to have pretty solid support? At least over Twitter, it felt like I was talking to someone competent. I haven't had any major issues since installation, though, so maybe I just got lucky.
cpach 6y ago
That sounds amazing!
nitrogen 6y ago
XMission in Salt Lake City had support like that, back when DSL unbundling was a thing (2006).
neelc 6y ago
And when Wave bought them, Wave put incompetent people in charge. I moved to a Seattle suburb and had "Wave G" (post-Wave CondoInternet). I had throughput issues, first on my apartment. That got resolved via Reddit. Then the backbone is slow from time to time which persisted. Sometimes I get a Gigabit, other times I only get 10-20 Mbps. What did Wave say? Let's bring a tech out. I told them "it's a backbone issue" and they didn't understand me. I don't know how I would explain how networking works to a field tech who only knows how to enable a port or run basic diagnostics. Another time, I tripped Port Security, and told them they can re-enable my port. Wave said "we need to bring a tech out" as if that will magically solve everything. This is made worse by my apartment's exclusivity deals: only Comcast and Wave, nothing else. Ziply Fiber (bought Frontier) and Atlas Networks were denied. Bigger companies like Verizon FiOS in NYC had better tech support, I could still get hold of a Level 2 tech for a much smaller issue (and not even on FiOS). Wave G makes AT&T's forced router (but not the flipping bits) seem decent in comparison. I don't want Comcast but I'll happily take AT&T Fiber over Wave G even if it means trading my pfSense box for a crappy AT&T gateway (worse case scenario I could bypass or root the gateway, or pay $15 for static IPs). I fortunately moved (for another reason) and have Google's Webpass. Webpass may not give me a full Gigabit, but it's usually 400-800 Mbps all the time and not 10-20 Mbps 95% of the time with an occasional Gigabit. I wish CondoInternet sold to Webpass instead of Wave (I don't want a merger now, but still). Surprisingly, I may have preferred CenturyLink Fiber if available mainly for GPON over a 60GHz PtP microwave link (less oversubscription!), well unless PPPoE+6rd kills my pfSense. But Webpass works pretty darn well, and I think I'll stay unless CenturyLink suddenly gives me 2 Gbps FTTH or something, or I move.

iandanforth 6y ago

Have you tried saying "shibboleet"?

_whiteCaps_ 6y ago
I think you're being unfairly downvoted by people who haven't seen this xkcd: https://xkcd.com/806/ https://xkcd.com/806/
- closeparen 6y ago
  The real life version of “shibboleet” is your Certified Partner ID number and a serial number with a valid support contract. When I worked for a VAR I could upload logs to Cisco and get experimental patches back. Call up HP, tell them I want an RMA, and they’d just do it. Night and day compared to what consumers get.
robgolding 6y ago
My ISP actually supports this[0], though I haven’t had cause to use it yet - they are also very reliable! [0]: https://www.aa.net.uk/broadband/why-choose-aaisp/ https://www.aa.net.uk/broadband/why-choose-aaisp/

013a 6y ago

Have you seen Parks & Rec, and remember that scene in a Home Depot where an associate walks up to Ron, asks him if he needs help with a project, and Ron responds "I know more than you"? I've pulled a variation of that on CSRs at least once, and surprisingly, it can work. Just be cordial, preempt the typical IT Support stuff they always ask, DO NOT say its intermittent (initially, to the front line CSR; if given a chance to expand the issue after escalation, then add that bit), and get technical ASAP (it doesn't hurt to throw in some parallel industry jargon). Basically, build a case where even the information you're giving them is beyond a first-line CSR playbook, and they have to escalate. "Hi there; I've been observing some erroneous TCP packet bit flipping on HTTP requests which route through one of AT&T's data center in Oakland. I've tried restarting my computer, I'm seeing the same thing on my phone, and I actually swapped my router out for a spare one I have, but its still an issue." (that last sentence exhausts literally every playbook a front-line CSR has. it sounds so easy, right? there are four variables in any front-line CSR diagnostic equation: their network, your router, wifi/ethernet, and the endpoint. you just crossed off three of the four variables in one sentence). (Wait, a data center in Oakland? How do you know this? You can tracert a bad request and geolocate the first IP outside your network, but, lets be realistic: You don't. You're fronting; demonstrating knowledge that a front-line CSR can't disprove. You may think this is misleading to whoever this gets escalated to, but it isn't; their tools are FAR more advanced than yours, and they're used to 99% of customers being incorrect idiots, so they're going to be validating and reconfirming every word you say anyway.) Ron's Parks & Rec example above is crass. But here's the magic bit: frontline CSRs generally look for an excuse to escalate, you just need to give them enough CYA to check their job as done, and the higher tier CSRs/network engineers will love you for actually knowing what you're talking about. Its a win-win; be cordial, be forceful, strut what you know.

zanny 6y ago
I had something like this happen on an even simpler level this last week. I got a Chase credit card but when I initially did the signup called my brother to ask him if he wanted to be on the account and it timed the sign up session out past the account creation but before finalization. I got the card eventually but now I cannot create an online account with it. I called Chase, got transferred 5 times, and then told I would need to go to a physical bank to verify my identity? to create an account. Absolutely not one of them had any clue what "a broken account exists associated with this card in your database, I can guarantee it, forward me to your technical support team" but thats all above a bank reps pay grade. The nearest Chase bank is 1.5 hours away, by the way. Probably just going to cancel the card after cashing out the sign up bonus.

reader_1000 6y ago

Ten years ago, while using ADSL, for some reason captcha images were not loading. I opened a case to ISP support and they called me back. The support guy did not believe it. He said "this is not an analog network, this is digital, you either have it or not". I said I know what the analog and digital mean and also I know this is a digital network since I am a computer engineer and I checked everything so this was a issue with connection. After a couple hours later, he called me back and said that problem was caused by modem and a driver update would fix it and it did. These were good old days, however, when we can reach someone on the phone and talk about problem. Nowadays everything is either automated response or some random person whose whole job to tell you that he/she cannot do anything about the situation.