4 ms·
>We are aware of connectivity issues (…) In order to resolve this faster, your ISP (Internet Service Provider) will need to flush their DNS record for slack.com
by zucked 5y ago
>We are aware of connectivity issues (…) In order to resolve this faster, your ISP (Internet Service Provider) will need to flush their DNS record for slack.com. Please reach out to your networking team to provide them with this information.
Yeah, sure, lemme just pick up the phone and call my ISP and let 'em know to flush their DNS record. I'm sure the T1 rep will absolutely know how to handle that.
- betaby 5y agoThat's a laughably bad and dishonest response from Slack. They are redirecting the blame to ISPs now.
- azundo 5y agoAre they blaming them though? Is this not the actual fastest path to resolving the issue?
- walrus01 5y agounless you are a customer of a very tiny ISP the odds of getting the person on the phone who has root on their recursive caching nameservers provided to DHCP clients are minuscule.
- znpy 5y agoAnd chances are they wouldn't flush that entry for you anyway.
- kreeben 5y agoI wonder how much electricity will be spent flushing then repopulating the caches of each and every DNS server on the globe. Prolly not that much?
- lima 5y agoNo need to flush the full caches. Clearing caches for individual zones is not uncommon.
- wk_end 5y agoThey should at least say something like "we're working with major ISPs to resolve the issue". It really comes off as walking away from a disaster of their own making.
- lima 5y agoNo, the fastest path is putting the signature back. But I guess Route53 won't let them do that.
- alex_c 5y agoMissing part of that quote is "This issue was caused by our own change and not related to any third-party DNS software and services." Seems refreshingly honest to me. Slack screwed up, it's not anyone else's fault, but there's nothing Slack can do about it now, here's a workaround.
- paxys 5y agoIt's more dishonest to read one part of one line and draw conclusions from it without quoting the entire response.
- ziddoap 5y agoWhich part of the full Slack reply is dishonest?
- kaustubhvp 5y agoHave you tried calling Comcast/Xfinity ever?
- sieabah 5y agoWell they aren't necessarily incorrect, TTLs can be a pain. However that's why when you do a migration you lower the TTLs and eat the cost of lookup requests. Wait until your previous TTL lease expires (which is 2x the TTL) then start the migration. After everything is known to be good you up the TTLs again. That way you have a way to "quickly" recover if issues arise.
- walrus01 5y agotranslation: a) we pushed a change that was not well thought out in its ramifications b) we probably didn't lower the TTL on our zones several days in advance of doing this change c) we reverted in a panic! d) by the way, did you know that many ISPs' caching nameservers don't respect low TTLs and will hold onto zones longer than they should? e) please go bother and waste the time of some first tier support rep at RCN or Comcast or CenturyLink, who certainly has the power to administer their DNS servers... oh jeez.
- yjftsjthsd-h 5y ago> d) by the way, did you know that many ISPs' caching nameservers don't respect low TTLs and will hold onto zones longer than they should? In fairness, this part really is a dumb problem on the ISP's end.
- simcop2387 5y agoIt's also partly because so many isps got tired of their caches being basically useless because of so many people setting obscenely low ttls too. There's no winning answer.
- yjftsjthsd-h 5y agoIs DNS truly so high load that they need substantial caching? I mean, DNS responses should be kilobytes on the high end; it seems like they should barely register on overall traffic.
- CaliforniaKarl 5y agoImagine a home with some IoT devices: A smart thermostat, a smart TV, a smart doorbell, and three cameras. The thermostat connects out every 30 seconds to send a temperature report, and to check for settings changes. The smart TV, every 30 seconds, sends a compressed snapshot of the image currently being displayed, for ad generation. The doorbell starts a new connection to stream video whenever movement is detected. The cameras do the same. Each device was built without any thought to DNS caching. As such, every new connection is going to trigger a new DNS lookup. Multiply that by however many homes are in the area, and the load really starts going up. Especially because each lookup from the device causes a series of lookups to happen on the recursive DNS server.
- deleted 5y ago[deleted]
- amelius 5y agoWhy not edit your /etc/hosts file directly?
- deleted 5y ago[deleted]
- hnarn 5y agoProbably because slack doesn’t want a long tail of future hard to pin down issues related to people not reverting their changes to their hosts file.