7 ms·

IO name servers down

102 points by gator-io 9y ago

I'm getting swamped DNS resolution errors for anything .IO. ns-a1.io timesout on every request so do all the others

deleted 9y ago
[deleted]
nodesocket 9y ago
Same here for my startup commando.io. Using AWS Route53. What DNS provider are you using?
- turdnagel 9y ago
  Same here, also using Route 53.
  - nodesocket 9y ago
    I was certain I was getting DDoS'd, but then I inspected the Pingdom down notifications and seeing: DNS error
    - cjsuk 9y ago
      Same here on Route53.
    - jon-wood 9y ago
      This is almost completely irrelevant, but what are you doing where you jump to DDoS before misconfiguration or a service outage?
- dastbe 9y ago
  From the AWS status page: "Some customers have reported intermittent resolution issues with .io domains. We can confirm that Route 53 DNS services are operating normally at this time and these issues seem to be related to the .io top-level domain provider."
- mrideout 9y ago
  I run a DNS monitoring service and have had a number of customers contact me about this in the past couple hours. Most of them were using Route53, but this is an issue with the IO authoritative nameservers, rather than Route53 itself.
tbarbugli 9y ago
These two nameservers: ns-a2.io, ns-a4.io return wrong results consistently
tbarbugli 9y ago
team mate posted on HN already about this: https://news.ycombinator.com/item?id=15293578 https://news.ycombinator.com/item?id=15293578
devnull42 9y ago
It appears that the issue at first impacted all servers in the anycast pool however eventually it only impacted servers ns-a2 and ns-a4. Those servers started returning NXDOMAINs. I am wondering if this was related to the root server key change yesterday. .IO seems to struggle with basic DNS engineering. We are seeing stabilization except for minor issues still on one of the gTLD servers.
- deleted 9y ago
  [deleted]
- steventhedev 9y ago
  The root server key won't actually change until next month. The DNSKEY responses from the root server was increased yesterday. Speculation, but it could be they're running ancient versions of BIND that fail with the larger response size. As per ICANN[0], the timeline is: October 27, 2016: KSK rollover process begins as the new KSK is generated. July 11, 2017: Publication of new KSK in DNS. September 19, 2017: Size increase for DNSKEY response from root name servers. October 11, 2017: New KSK begins to sign the root zone key set (the actual rollover event). January 11, 2018: Revocation of old KSK. March 22, 2018: Last day the old KSK appears in the root zone. August 2018: Old key is deleted from equipment in both ICANN Key Management Facilities. [0]: https://www.icann.org/resources/pages/ksk-rollover https://www.icann.org/resources/pages/ksk-rollover PS - thank you for mentioning this, I wasn't aware it was going to happen until reading your comment.
  - devnull42 9y ago
    Correct the actual key change isn’t until next month however yesterday there was a change in response size from the root servers.
thekemkid 9y ago
Seems like a hijacking: https://www.theregister.co.uk/2017/07/10/io_hijacking_in_transition_cockup/ https://www.theregister.co.uk/2017/07/10/io_hijacking_in_tra...
- SCdF 9y ago
  That is a news report from July.
JelteF 9y ago
For more info check the other post on this topic: https://news.ycombinator.com/item?id=15293578 https://news.ycombinator.com/item?id=15293578 Not sure why that was taken of the front page suddenly.
X-Istence 9y ago
Noticed this issue this morning too... https://twitter.com/bertjwregeer/status/910515512903319552 https://twitter.com/bertjwregeer/status/910515512903319552 Interestingly enough I found that only some of them were returning NXDOMAIN's, so resolution would sometimes work and sometimes it would fail completely.
fenwick67 9y ago
I'm so glad I passed on the .io TLD for my personal page and got .pizza one instead.
- brianwawok 9y ago
  Smart man would get both and load balance the tld
- lzy 9y ago
  I thought you're joking and then discovered there actually really is a .pizza TLD. https://icannwiki.org/.pizza https://icannwiki.org/.pizza
gator-io 9y ago
Everything appears to be functioning at this point
jmkni 9y ago
Seems to be sorted now
riffic 9y ago
Not a surprise; country code top-level domains are run poorly and I sure wouldn’t stick all of my eggs in this single basket.
gator-io 9y ago
I am so done with .io. This is one of many issues they've had in the last year. My problem is that we have scripts all over customer websites hardcoded with api.gator.io We're going to have to have them update the scripts and that is going to be a major pain.
- softawre 9y ago
  Heh, it's even in your phone number > 1-844-GATOR-IO
partycoder 9y ago
Well, the .io domain name is intended for the British Indian Ocean territory. I think the common practice of misusing TLDs (such as registering an .io domain if you are not from the British Indian Ocean territory, or having a Soviet Union domain... a country that no longer exist), is bad. However, I acknowledge this is not the root problem. The root problem is the scarcity of domains under traditional TLDs, and that's in great part due to speculation (e.g: domain parking). It is hard to establish what constitutes placeholder content, therefore rules preventing domain parking are hard if not unfeasible to forbid.
- ptenk 9y ago
  Thankfully built-in parking AdBlock on newer browsers is forcing domain speculators to start dropping their less valuable names, as parking pages are no longer producing any revenue. I see most tech companies coming back to .COM once their given domain is available / priced cheaply as speculators are forced to sell.

102 points by gator-io 9y ago

I'm getting swamped DNS resolution errors for anything .IO. ns-a1.io timesout on every request so do all the others

deleted 9y ago

[deleted]

nodesocket 9y ago

Same here for my startup commando.io. Using AWS Route53. What DNS provider are you using?

turdnagel 9y ago
Same here, also using Route 53.
- nodesocket 9y ago
  I was certain I was getting DDoS'd, but then I inspected the Pingdom down notifications and seeing: DNS error
  - cjsuk 9y ago
    Same here on Route53.
  - jon-wood 9y ago
    This is almost completely irrelevant, but what are you doing where you jump to DDoS before misconfiguration or a service outage?
dastbe 9y ago
From the AWS status page: "Some customers have reported intermittent resolution issues with .io domains. We can confirm that Route 53 DNS services are operating normally at this time and these issues seem to be related to the .io top-level domain provider."
mrideout 9y ago
I run a DNS monitoring service and have had a number of customers contact me about this in the past couple hours. Most of them were using Route53, but this is an issue with the IO authoritative nameservers, rather than Route53 itself.

tbarbugli 9y ago

These two nameservers: ns-a2.io, ns-a4.io return wrong results consistently

tbarbugli 9y ago

team mate posted on HN already about this: https://news.ycombinator.com/item?id=15293578 https://news.ycombinator.com/item?id=15293578

devnull42 9y ago

It appears that the issue at first impacted all servers in the anycast pool however eventually it only impacted servers ns-a2 and ns-a4. Those servers started returning NXDOMAINs. I am wondering if this was related to the root server key change yesterday. .IO seems to struggle with basic DNS engineering. We are seeing stabilization except for minor issues still on one of the gTLD servers.

deleted 9y ago
[deleted]
steventhedev 9y ago
The root server key won't actually change until next month. The DNSKEY responses from the root server was increased yesterday. Speculation, but it could be they're running ancient versions of BIND that fail with the larger response size. As per ICANN[0], the timeline is: October 27, 2016: KSK rollover process begins as the new KSK is generated. July 11, 2017: Publication of new KSK in DNS. September 19, 2017: Size increase for DNSKEY response from root name servers. October 11, 2017: New KSK begins to sign the root zone key set (the actual rollover event). January 11, 2018: Revocation of old KSK. March 22, 2018: Last day the old KSK appears in the root zone. August 2018: Old key is deleted from equipment in both ICANN Key Management Facilities. [0]: https://www.icann.org/resources/pages/ksk-rollover https://www.icann.org/resources/pages/ksk-rollover PS - thank you for mentioning this, I wasn't aware it was going to happen until reading your comment.
- devnull42 9y ago
  Correct the actual key change isn’t until next month however yesterday there was a change in response size from the root servers.

thekemkid 9y ago

Seems like a hijacking: https://www.theregister.co.uk/2017/07/10/io_hijacking_in_transition_cockup/ https://www.theregister.co.uk/2017/07/10/io_hijacking_in_tra...

SCdF 9y ago
That is a news report from July.

JelteF 9y ago

For more info check the other post on this topic: https://news.ycombinator.com/item?id=15293578 https://news.ycombinator.com/item?id=15293578 Not sure why that was taken of the front page suddenly.

X-Istence 9y ago

Noticed this issue this morning too... https://twitter.com/bertjwregeer/status/910515512903319552 https://twitter.com/bertjwregeer/status/910515512903319552 Interestingly enough I found that only some of them were returning NXDOMAIN's, so resolution would sometimes work and sometimes it would fail completely.

fenwick67 9y ago

I'm so glad I passed on the .io TLD for my personal page and got .pizza one instead.

brianwawok 9y ago
Smart man would get both and load balance the tld
lzy 9y ago
I thought you're joking and then discovered there actually really is a .pizza TLD. https://icannwiki.org/.pizza https://icannwiki.org/.pizza

gator-io 9y ago

Everything appears to be functioning at this point

jmkni 9y ago

Seems to be sorted now

riffic 9y ago

Not a surprise; country code top-level domains are run poorly and I sure wouldn’t stick all of my eggs in this single basket.

gator-io 9y ago

I am so done with .io. This is one of many issues they've had in the last year. My problem is that we have scripts all over customer websites hardcoded with api.gator.io We're going to have to have them update the scripts and that is going to be a major pain.

softawre 9y ago
Heh, it's even in your phone number > 1-844-GATOR-IO

partycoder 9y ago

Well, the .io domain name is intended for the British Indian Ocean territory. I think the common practice of misusing TLDs (such as registering an .io domain if you are not from the British Indian Ocean territory, or having a Soviet Union domain... a country that no longer exist), is bad. However, I acknowledge this is not the root problem. The root problem is the scarcity of domains under traditional TLDs, and that's in great part due to speculation (e.g: domain parking). It is hard to establish what constitutes placeholder content, therefore rules preventing domain parking are hard if not unfeasible to forbid.

ptenk 9y ago
Thankfully built-in parking AdBlock on newer browsers is forcing domain speculators to start dropping their less valuable names, as parking pages are no longer producing any revenue. I see most tech companies coming back to .COM once their given domain is available / priced cheaply as speculators are forced to sell.