8 ms·

The Great Puri.sm Outage of 2018

135 points by grinsekatze 9y ago

igor47 9y ago
tl;dr their registrar, 101domains.com, failed to pay their registrar fees to the .sm TLD so all .sm domains registered through them got suspended dns outages are always a nightmare because of ttls, but bringing a registrar and a nontraditional TLD into the picture meant a much more extended outage.
- gtirloni 9y ago
  And in their case, makes it harder to remember their website address.
  - ZenoArrow 9y ago
    Either you remember it or you don't. If you're going to guess it ends with ".com" you may as well save yourself the hassle and search for it using a search engine.
- rconti 9y ago
  I can't believe that 101domains didn't notice that nobody else's .sm domains were working either. You'd think after around a day they would have noticed some correlation in complaints. Or the .sm specialist would have. But yeah, don't use unnecessarily cheap infra for really critical stuff. I don't get the fetish around obscure TLDs with web pages straight out of 1995. http://nic.com.ai http://nic.com.ai
  - bonesss 9y ago
    I'm gonna wager that .sm isn't too active with the registrar, and most of the sites are of a relatively low-traffic nature. If someone from the registrar starts checking the overall health of the TLD, or any domain they weren't directly responsible for, it may have seemed that "everything was ok" even though all their customer records were suspended...
walrus01 9y ago
this is some serious amateur-hour stuff on the part of the registrar. another example of why not to use random ccTLD top level domains. nobody remember when the .ly registry randomly canceled things they didn't like? when .cx canceled things they found objectionable? see also: https://news.ycombinator.com/item?id=5838670 https://news.ycombinator.com/item?id=5838670 https://news.ycombinator.com/item?id=12813065 https://news.ycombinator.com/item?id=12813065 https://www.theregister.co.uk/2017/07/10/io_hijacking_in_transition_cockup/ https://www.theregister.co.uk/2017/07/10/io_hijacking_in_tra... At least if you use .ca, .de, .us or .com/.net (or similar) you can have reasonable confidence that the root zone won't fail entirely. If something goes catastrophically wrong and kills your business that is run by a .CA domain, CIRA knows that businesses in Canada have access to legal recourses... Do you have the same with a .ly domain?
- Illniyar 9y ago
  This case appears to be the registrar's fault, not the top level domain's fault. Unless 1and1 uses a "sub" registrar for .sm domains (which is actually quite likely) then I would say the moral of the story here is to not use shitty 1$ domain registrars who if I remember correctly have very bad support reviews.
  - walrus01 9y ago
    In my opinion and based on past experience, all resellers of small-weird-country ccTLDs are equally shitty and have equally terrible customer service. Margins are very thin. At least if I buy a "new" TLD from a company that resells one of Donuts LLC's huge number of new generic TLDs, I can have some confidence that Donuts actually runs a 24 hour operation. The ICANN vetting process and procedures to set up a new gTLD (let's use .network as an example) are much more thorough than the traditional two letter ccTLD. There are ccTLDs run by countries that edit the zonefiles by hand and don't even have a WHOIS server online. for example using the current issue, nic.sm doesn't even run https on their webserver. Or if I buy a .com from namecheap I can set up 2FA at my registrar, point it at my own authoritative nameservers, and trust that it won't fail.
    - 9y ago
lwhalen 9y ago
And this is why I use gandi.net for all my domain registrations. As their tagline says, "0 bullsh*t". Such has been my experience with them for the last ~20 or so years.
- Operyl 9y ago
  0 Bullshit, except when it isn't. They constantly kept harassing me for information because "ICANN is asking us daily." After being threatened the 7th day in a row, despite providing the same exact information every single god damned day, with the same passport and state identification .. I gave up. That's when I moved to Namecheap, had the glue record problem I mentioned in another comment, moved from them to Google Domains and have been happy since.
oldcynic 9y ago
The surprising omission from this post is that it doesn't conclude with them migrating away from 101domains. Comically bad support and not paying registry fees. I'd have pulled domains the moment it was live again.
- walrus01 9y ago
  If you think customer service is bad for an issue like they had - just wait until you try to transfer a domain away from a shitty registrar. It risks breaking things immediately again. I'm sure they're looking at it but right now they're effectively held hostage.
  - grinsekatze 9y ago
    Is it really so difficult/risky? If you set all dns records up in advance at the new registrar before transferring and also set the desired nameservers for the domain in advance shouldn’t that work? or am I missing something? In any case. I was very surprised that they were using such a registrar in the first place..
    - rconti 9y ago
      Yes, it should. But this outage also should not have happened, but it did. Front-line folks tend to be understandably gunshy about making changes to critical infra.
    - zAy0LfpBZLC8mAC 9y ago
      Really, you should not be using your registrar's nameservers, precisely because that only makes switching registrars more complicated. Run your own or use a separate DNS hosting provider.
      - wink 9y ago
        ...and have the nice experience that your registrar removes your DNS records from their nameserver the same second you click "submit" to change them to somewhere else. Been there, done that. (It's the main reason I won't use 1&1 [yes, the German company, NOT the 101 the OP mentioned] anymore)
Cyberdog 9y ago
Purism makes crapware-free and security-focused phones and laptops. (Dear product blog operators: https://news.ycombinator.com/item?id=16278142 https://news.ycombinator.com/item?id=16278142 )
_nalply 9y ago
Another trap: Once a registrar didn't send renewal reminders and this way we lost a two-letter domain to a domain grabber. What I have learnt: Set up your own reminders to renew domains.
lacker 9y ago
It's pretty early to declare you've already had your greatest outage of 2018 ;-) Hope you aren't jinxing it!
krylon 9y ago
I know this is not a very constructive comment, but I had a very stressful day, being jacked up on caffeine while also being sleep-deprived. My brain wanted to read "OutRage" instead of Outage, and I got kind of confused, ending up thinking, "that is not that outrageous! Why do people get upset so easily?" With that error corrected (what is it called? A "reado"?), the title makes a lot more sense. ;-)
- deleted 9y ago
  [deleted]
fapjacks 9y ago
My favorite part of the story is their sysadmins! Their names are great in this context and make it sound almost like an ancient Greek epic: "Our long-suffering and amazing sysadmins Theodotos and Stelios contacted our puri.sm registrar, 101domain.com, to find out what was going on." You can't go wrong with your two best warriors Theodotos and Stelios at hand!
theossuary 9y ago
This doesn't surprise me, I used 101domains to register a .io domain and only had issues with them. Their support wasn't useful, and when I eventually decided to move my domain away from them it got to the point where I was concerned they'd hold it hostage, because they would not give me a transfer code. Eventually I got it escalated and moved to another registrar, and because of that experience I'll never use them as a registrar again. This story just re-enforces that. Props to the purism guys for going so far as to work with the TLD, that's impressive.
lzy 9y ago
Switch to the .com they just registered permanently and 301 redirect their fanciful domain hack registration to never have to deal with such an issue again?
rdl 9y ago
I hate people abusing ccTLDs for "cute" domain names. It breaks everything about how they are supposed to be assigned/used. The biggest assclownery was .ly, IMO -- an islamofascist rogue state (then; now just a generic failed state) being used for various domains.
- exikyut 9y ago
  Makes me wonder what kinds of interesting stories bit.ly have collected over the years...
a2tech 9y ago
Long story short? Don't use shitty registars. And that cute two letter extension? Don't use that either.
foo101 9y ago
How did it happen that the early internet development community agreed to hand over the governance and registration of a domain name to central authorities like domain name registrars? What is preventing us from making domain name governance and registration completely decentralized?

135 points by grinsekatze 9y ago

tl;dr their registrar, 101domains.com, failed to pay their registrar fees to the .sm TLD so all .sm domains registered through them got suspended dns outages are always a nightmare because of ttls, but bringing a registrar and a nontraditional TLD into the picture meant a much more extended outage.

gtirloni 9y ago
And in their case, makes it harder to remember their website address.
- ZenoArrow 9y ago
  Either you remember it or you don't. If you're going to guess it ends with ".com" you may as well save yourself the hassle and search for it using a search engine.
rconti 9y ago
I can't believe that 101domains didn't notice that nobody else's .sm domains were working either. You'd think after around a day they would have noticed some correlation in complaints. Or the .sm specialist would have. But yeah, don't use unnecessarily cheap infra for really critical stuff. I don't get the fetish around obscure TLDs with web pages straight out of 1995. http://nic.com.ai http://nic.com.ai
- bonesss 9y ago
  I'm gonna wager that .sm isn't too active with the registrar, and most of the sites are of a relatively low-traffic nature. If someone from the registrar starts checking the overall health of the TLD, or any domain they weren't directly responsible for, it may have seemed that "everything was ok" even though all their customer records were suspended...

walrus01 9y ago

this is some serious amateur-hour stuff on the part of the registrar. another example of why not to use random ccTLD top level domains. nobody remember when the .ly registry randomly canceled things they didn't like? when .cx canceled things they found objectionable? see also: https://news.ycombinator.com/item?id=5838670 https://news.ycombinator.com/item?id=5838670 https://news.ycombinator.com/item?id=12813065 https://news.ycombinator.com/item?id=12813065 https://www.theregister.co.uk/2017/07/10/io_hijacking_in_transition_cockup/ https://www.theregister.co.uk/2017/07/10/io_hijacking_in_tra... At least if you use .ca, .de, .us or .com/.net (or similar) you can have reasonable confidence that the root zone won't fail entirely. If something goes catastrophically wrong and kills your business that is run by a .CA domain, CIRA knows that businesses in Canada have access to legal recourses... Do you have the same with a .ly domain?

Illniyar 9y ago
This case appears to be the registrar's fault, not the top level domain's fault. Unless 1and1 uses a "sub" registrar for .sm domains (which is actually quite likely) then I would say the moral of the story here is to not use shitty 1$ domain registrars who if I remember correctly have very bad support reviews.
- walrus01 9y ago
  In my opinion and based on past experience, all resellers of small-weird-country ccTLDs are equally shitty and have equally terrible customer service. Margins are very thin. At least if I buy a "new" TLD from a company that resells one of Donuts LLC's huge number of new generic TLDs, I can have some confidence that Donuts actually runs a 24 hour operation. The ICANN vetting process and procedures to set up a new gTLD (let's use .network as an example) are much more thorough than the traditional two letter ccTLD. There are ccTLDs run by countries that edit the zonefiles by hand and don't even have a WHOIS server online. for example using the current issue, nic.sm doesn't even run https on their webserver. Or if I buy a .com from namecheap I can set up 2FA at my registrar, point it at my own authoritative nameservers, and trust that it won't fail.
  - 9y ago

lwhalen 9y ago

And this is why I use gandi.net for all my domain registrations. As their tagline says, "0 bullsh*t". Such has been my experience with them for the last ~20 or so years.

Operyl 9y ago
0 Bullshit, except when it isn't. They constantly kept harassing me for information because "ICANN is asking us daily." After being threatened the 7th day in a row, despite providing the same exact information every single god damned day, with the same passport and state identification .. I gave up. That's when I moved to Namecheap, had the glue record problem I mentioned in another comment, moved from them to Google Domains and have been happy since.

oldcynic 9y ago

The surprising omission from this post is that it doesn't conclude with them migrating away from 101domains. Comically bad support and not paying registry fees. I'd have pulled domains the moment it was live again.

walrus01 9y ago
If you think customer service is bad for an issue like they had - just wait until you try to transfer a domain away from a shitty registrar. It risks breaking things immediately again. I'm sure they're looking at it but right now they're effectively held hostage.
- grinsekatze 9y ago
  Is it really so difficult/risky? If you set all dns records up in advance at the new registrar before transferring and also set the desired nameservers for the domain in advance shouldn’t that work? or am I missing something? In any case. I was very surprised that they were using such a registrar in the first place..
  - rconti 9y ago
    Yes, it should. But this outage also should not have happened, but it did. Front-line folks tend to be understandably gunshy about making changes to critical infra.
  - zAy0LfpBZLC8mAC 9y ago
    Really, you should not be using your registrar's nameservers, precisely because that only makes switching registrars more complicated. Run your own or use a separate DNS hosting provider.
    - wink 9y ago
      ...and have the nice experience that your registrar removes your DNS records from their nameserver the same second you click "submit" to change them to somewhere else. Been there, done that. (It's the main reason I won't use 1&1 [yes, the German company, NOT the 101 the OP mentioned] anymore)

Cyberdog 9y ago

Purism makes crapware-free and security-focused phones and laptops. (Dear product blog operators: https://news.ycombinator.com/item?id=16278142 https://news.ycombinator.com/item?id=16278142 )

_nalply 9y ago

Another trap: Once a registrar didn't send renewal reminders and this way we lost a two-letter domain to a domain grabber. What I have learnt: Set up your own reminders to renew domains.

lacker 9y ago

It's pretty early to declare you've already had your greatest outage of 2018 ;-) Hope you aren't jinxing it!

krylon 9y ago

I know this is not a very constructive comment, but I had a very stressful day, being jacked up on caffeine while also being sleep-deprived. My brain wanted to read "OutRage" instead of Outage, and I got kind of confused, ending up thinking, "that is not that outrageous! Why do people get upset so easily?" With that error corrected (what is it called? A "reado"?), the title makes a lot more sense. ;-)

deleted 9y ago
[deleted]

fapjacks 9y ago

My favorite part of the story is their sysadmins! Their names are great in this context and make it sound almost like an ancient Greek epic: "Our long-suffering and amazing sysadmins Theodotos and Stelios contacted our puri.sm registrar, 101domain.com, to find out what was going on." You can't go wrong with your two best warriors Theodotos and Stelios at hand!

theossuary 9y ago

This doesn't surprise me, I used 101domains to register a .io domain and only had issues with them. Their support wasn't useful, and when I eventually decided to move my domain away from them it got to the point where I was concerned they'd hold it hostage, because they would not give me a transfer code. Eventually I got it escalated and moved to another registrar, and because of that experience I'll never use them as a registrar again. This story just re-enforces that. Props to the purism guys for going so far as to work with the TLD, that's impressive.

lzy 9y ago

Switch to the .com they just registered permanently and 301 redirect their fanciful domain hack registration to never have to deal with such an issue again?

rdl 9y ago

I hate people abusing ccTLDs for "cute" domain names. It breaks everything about how they are supposed to be assigned/used. The biggest assclownery was .ly, IMO -- an islamofascist rogue state (then; now just a generic failed state) being used for various domains.

exikyut 9y ago
Makes me wonder what kinds of interesting stories bit.ly have collected over the years...

a2tech 9y ago

Long story short? Don't use shitty registars. And that cute two letter extension? Don't use that either.

foo101 9y ago

How did it happen that the early internet development community agreed to hand over the governance and registration of a domain name to central authorities like domain name registrars? What is preventing us from making domain name governance and registration completely decentralized?