28 ms·
Cloudflare silently deleted my DNS records
- whatthesmack 7y agoThis is frightening. I just started the process of moving all ~60 of my domains from Amazon Registrar + Google Cloud DNS to Cloudflare, and will definitely wait until somebody from Cloudflare chimes in here to clarify what's going on.
- Jerry2 7y ago> moving all ~60 of my domains from Amazon Registrar + Google Cloud DNS to Cloudflare You're very brave considering that Cloudflare doesn't even have U2F yet Google and Amazon do.
- whatthesmack 7y agoGreat point! And let's just say that the migration project is now on-hold :)
- ocdtrekkie 7y agoAre you using physical U2F keys for your Google or Amazon accounts? Cloudflare does support standard TOTP-based 2FA like most people use for Amazon and Google. So whether or not the lack of U2F support should matter depends on whether you actually use it elsewhere anyways.
- somehnguy 7y agoThey have TOTP 2 factor however.
- nhoven 7y agoU2F is under active development. My team is actually working on it as we speak
- phyzome 7y agoAmazon doesn't allow multiple U2F. D-:
- iudqnolq 7y agoOP here. I'm considering moving to Amazon Registrar. Why are you leaving?
- whatthesmack 7y agoI was only moving to Cloudflare because they do registrar services at-cost, which would be cheaper than Amazon Registrar.
- ocdtrekkie 7y agoHow many of your sixty domains are business-critical? Cloudflare's Domains service is new, and some of it's management tools are lacking, but I also moved most of my domains to it over the last year for cost savings. I'm thrilled with it, but I'm still keeping a few of my most critical domains with GoDaddy. (Hate them all you want, but GoDaddy hasn't screwed up my domains in well over a decade.) You may be able to save a lot of money without risking your primary domain that you route email through.
- freedomben 7y agoI've been planning too soon, am also now going to wait to see where this goes. DNS is obviously a critical system and I don't know if I can trust Cloudflare now. I'm not a big fish that can make noise. I'm an easy victim.
- flurdy 7y agoDon't put all your eggs in one basket, ie. don't just use one provider. Also for your core domains, do not let the registrar and dns provider be the same entity. Also, don't decide on not migrating just because of one bad experience. None of them are perfect, though vigilance is wise. (I know am probably preaching to the choir :) )
- donmcronald 7y agoThis is what I do. Domains are at Porkbun and domain registration is the ONLY thing I use them for. DNS and caching are at Cloudflare. Hosting is wherever makes sense. Consolidating domain registration, DNS hosting, and site hosting under the same account is a terrible idea and a big risk IMO. Always ask yourself “what if this account gets banned?” All of the big tech companies have automated systems that could turn you into an outlier with no support.
- ocdtrekkie 7y agoThere's a number of Cloudflare folks who are HN regulars, so hopefully you'll get some answers. Hopefully it's something they can reverse. But as a general reminder to everyone (I think this is an unfortunately common problem from a number of companies): If this is how your company handles account issues, you're probably wrong. Whether it's automated or manual, a user should be able to access all of their own information even when you decide to no longer provide them service. And you should test and retest the ability for people who you now deny service to transfer out.
- dvno42 7y agoFunny that this is coming up. I just transferred over from Namecheap to Cloudflare a few days ago and had a similar issue. One of my A records (out of about 20) were missing after the transfer.
- iudqnolq 7y agoI noticed that if you don't unfocus the input field by focusing somewhere else on the page it may not save. That may be what happened to you.
- fernandotakai 7y agoas much as i like cloudflare (and i like them a lot), it's kind of absurd that this kind of thing can happen. a lot of red flags that, if true, would mean that their infrastructure require a lot more care (127.0.0.1 as the source of an audit event? no email when DNS records are deleted? no 1-to-1 message due to this happening?).
- ocdtrekkie 7y agoAt the very least, this sort of lack of good process is definitely what happens when Google decides to cut you off (and another person just commented a similar experience with Amazon), but I suspect it's likely the case for a much larger number of companies and services than people realize. It's fundamental internet architecture, and often little more thought goes into account termination than what you'd do to ban someone from your mid-2000s phpBB forum. So much business focus goes into the onboarding experience, and since you assume all of the people your service terminates are "probably bad people anyways", not a lot of thought goes into offboarding, or ideally, appeals.
- use-net 7y agojust wait until MS revokes certain certs and all Win machines with TPM stop booting LOL!
- thedanbob 7y agoI had an issue with them recently where a SRV record pointing to “.” (meaning “service unavailable”) was being rewritten to the string “false”. It didn’t take them too long to fix it, but it made me wonder how they managed to push a bug like that to production without some sort of automated test catching it.
- ocdtrekkie 7y agoIIRC, if you're on a free plan you get exposed to code changes a little faster than their paying customers.
- daenz 7y agoThis happened to me with AWS somewhat recently[0], and I never found out exactly what happened. I just chalk it up to some dev made a mistake and didn't tell anyone. It's pretty alarming when things like this happen though. 0. https://news.ycombinator.com/item?id=21326014 https://news.ycombinator.com/item?id=21326014
- use-net 7y agocloudns.net does it a bit more customer-friendly way: they e-mailed me saying they deleted some domains not because some entries were broken or had problematic entries, but just because it was "underused", i.e. too few DNS resolve calls. So the tiny data packets in their nameserver caused them unnecessary consumption of electricity or whatever. Very compelling! This is how they do business these days. They bombarded people with all sorts of useless info, but not about this policy of theirs. Makes you feel very much like the proverbial "valued customer". Everything is going downhill in this century, that's a fact.
- jcrites 7y agoI've been involved in using Route 53 to manage thousands of DNS zones, and haven't come across something like that. I'd recommend putting in a support request via the account that was affected to ensure that it gets looked at. If you haven't already, you might consider checking the CloudTrail logs for the account in question to see if there were any API commands related to the zone.
- PetahNZ 7y agoAlthough not DNS related, I have had weird things happen on AWS, such as spikes of 5xx errors reported from CloudFront which was backed by ELB/EB, but the ELB is showing no errors. Even after contacting AWS support they couldn't resolve it, said they required application logs, but there is no logs because the requests never reached the application servers.
- britmob 7y agoThat is... quite scary. Why would you EVER have a way for auto deletion of domains?
- throwawaydns101 7y agoDNS has become frighteningly unreliable. Here are previous stories that show how it is possible to lose access to your domain for no fault of yours: (1) https://news.ycombinator.com/item?id=21700139 https://news.ycombinator.com/item?id=21700139 - Sinkholed (2) https://news.ycombinator.com/item?id=19322966 https://news.ycombinator.com/item?id=19322966 - I lost my domain and everything that goes with it No different than this story where the author's DNS records were deleted because of so called "anomaly". Here are so many more stories: https://news.ycombinator.com/item?id=21710939 https://news.ycombinator.com/item?id=21710939 DNS was a good idea but now there are organizations that have the power to arbitrarily take control and even remove your domain names and records. We really need to come up with a peer-to-peer solution and take back control of the naming system from these authorities.
- nathancahill 7y agoOdd comment to make a throwaway for, not very controversial (unless you work for Cloudflare?)
- throwawaydns101 7y agoI don't work for Cloudflare but I work for another large company that also manages domain names and DNS records. I don't want to risk the possibility that my comment could be interpreted by my employer as conflict the interest.
- Operyl 7y agoProbably should have made a mention to that in the first comment.
- deleted 7y ago[deleted]
- Legogris 7y agoI looked into self-hosting DNS and it doesn't seem like that big of a deal as long as you can ensure uptime to be honest. If you set up the two first on different hosts and possibly have #3/4 being cloud providers I think you're pretty good. Does anyone here have experience with running their own DNS servers for their domains?
- therealmarv 7y agoAlso don't forget: Cloudflare breaks many second and third world countries' Internet with their DNS captchas because they think the good guys live only in first world countries (maybe look up the word discrimination in your dictionary cloudflare) and force them to install extensions like PrivacyPass because they think "we are so big and know what is right for the world".
- input_sh 7y agoThat's CDN captcha, not DNS. If you use Cloudflare solely as a DNS provider, your users don't see the captcha. If you route your traffic through their servers, then they do.
- therealmarv 7y agoyou're right, it's their CDN not their DNS. Nevertheless many site owners choose Cloudflare (paid or not paid) and use Cloudflare's default settings and maybe they also never check their sites from second or third world countries. Result is that the Internet is utterly broken on many Cloudflare hosted sites (and that's a lot of sites) outside of first world countries.
- iudqnolq 7y agoOP here. You're right, and even in the US I still get endless CAPTCHAs because I browse on Firefox on Linux with tracking prevention. My website was down for yak shaving when this happened, but before then I had DDOS protection turned off.
- deleted 7y ago[deleted]
- johnklos 7y agoIs it really all that surprising when a big company that claims to be good but hosts phishing content in the name of free speech does whatever they want, including breaking things and not explaining why? I don't trust Cloudflare one bit, and I think everyone should question whether their attempt to re-centralize everything is beneficial to the planet. There are two major problems here: one, the problem itself, which is the deletion of DNS for apparently no good reason, and two, which is the bigger problem, is that it's incredibly difficult to talk to a human about what happened, so there's no assurance it won't happen again. If people want things to be reliable, we've got to stop using companies with which we cannot communicate.
- djsumdog 7y agoDoes OP have a free account with just DDOS protection. Does a paid account still have the notice to ask in the community forums first?
- iudqnolq 7y agoOP here. I pay them for domain registration, which they offer at cost. I use their free DNS. I disable DDOS protection.
- ocdtrekkie 7y agoIMHO (and I know the parent post includes significant difficulties getting back out of Cloudflare), services like Cloudflare may be crucial to decentralization. I can't deal with something like my blog post being frontpaged on HN if my website is hosted in my house, unless I have a good CDN. As a self-hosting enthusiast, something like Cloudflare is one of the best chances of having a plan that competes with "just hosting it in the cloud".
- luckylion 7y agoI don't know. Sure, maybe not on your barely-broadband DSL connection, but I'm pretty sure you can run most things on a average shared webhosting, even if you're using WP. You just need to make sure that you have a working caching system in place. It's a gigantic difference even on an apache to just read & send a plain file and invoke PHP & run all the costly code. I don't believe HN should bring down any site that can essentially be cached as static HTML. Cloudflare will only help once your server has gone down with their "Always On" thingy, if you have that enabled. They don't cache HTML by default.
- djsumdog 7y agoI really had a strong dislike for Cloudflare after they banned certain customers for political reasons[1]. The CEO mentioned how maybe it wasn't the right thing to do .. and then they did it again. There aren't really any self-hosted solutions for DDoS protection like Cloudflare since it requires things happening in the network layer. Implementing a solution would require access to monitor and reshape the local network, but I'm glad to see companies like Linode and DO offering DDos package. I want to start running my own DNS-over-HTTPS server as well, so I can pump firefox DNS requests to a self-hosted solution and not to Google or Cloudflare. I really don't trust them and am having trouble understanding why so many other people do. [1]: https://battlepenguin.com/politics/the-new-era-of-corporate-censorship/ https://battlepenguin.com/politics/the-new-era-of-corporate-...
- lexicality 7y agoWhile there are a lot of reasons not to trust cloudflare, the fact that they stopped hosting nazis and pedophiles doesn't seem like a good one to open with imo
- djsumdog 7y agoI think it is a problem when you talk about fear of ideas. You can label anyone as a Nazi today, and when you ban people, you kinda give them power. It shows you're afraid of their ideas, and the persecution can embolden them or give them a sense of legitimacy. It can being the Streisand Effect to their cause, taking a no-name site no one knew or cared about and blowing it up into something everyone is deeply aware of.
- claudiawerner 7y ago>and when you ban people, you kinda give them power. The key word being "kinda". You give the perception of power, but there are plenty of things in society we deal with by a form of "banning" (such as incarceration) in which we judge the ban to be good regardless of the power it gives people, or the ideas they represent or practice. Locking up child abusers, for instance, may give the spectre of child abuse power, and highly-publicized instances may fuel the moral panic around "strangers out to get your children", but that doesn't mean they should not be locked up. In civil society, we are justifiably afraid of many ideas - I don't know anyone who wouldn't be afraid of a Nazi-style dictatorship, or its prospect. Fear can be a legitimate way of preventing bad things from happening. I fear dying in a house fire, thus I take certain precautions when cooking. In the same way, a demonstration of power over a person or idea may outweigh the power supposedly given to that idea by banning it. The key is a balance; it may well be that you get an instance of the Streisand Effect, but it has to be shown that the consequences of that outweigh the very material consequences of such ideas coming to fruition in real life. For example, many people don't know who Barbara Streisand even is, and if they do, they likely don't know about the pictures of her house. The very canonical example of the Streisand Effect shows a short-lived controversy about the actual matter and ensuing attention for a few months after the incident. Then people forgot, or simply stopped caring. "The Streisand Effect" is more of a Streisand Effect than the actual incident that created it.
- dana321 7y agoCloudflare. A great solution if you want nobody to be able to easily access your website.
- iudqnolq 7y agoOP here. My website wasn't up when this happened because of some yak shaving, but when it is I disable DDOS protection. I was only using Cloudflare for domain registration and DNS. I don't think I have ethical issues with DDOS protection in general, but as someone who browses using Firefox on Linux with tracking blocking I know how annoying it can get. If I don't need it why bother? Plus I generally like to minimize opaque layers in my "stack".
- tus88 7y agoThe ultimate website blocker.
- dana321 7y agoThat and recaptcha!
- wackget 7y agoWhat are some alternatives which offer DDOS/flood/spam protection?
- superkuh 7y agoIt's very easy to manage incoming bandwidth when you're hosting a tor onion service. The entire Tor ecosystem kind of helps to since there's a limit on the instantaneous amount of data in any circuit. Overall tor is great because I own my domain name (rather than leasing it on the whim of some corp) and it has nice DoS and bandwidth tools built in. And if you were using cloudflare before you should be okay with some people not being able to access your site since that's the norm there.
- ryanlol 7y ago
- Legogris 7y agoI had this exact thing happen to me as well, but wrote it off to having been compromised (fortunately I was only using Cloudflare as secondary DNS servers on a non-production account and am not using them as a registrar, so I only noticed months after the fact). I think a major reason going with someone like Cloudflare for DNS in the first place is reliability and availability and this does not speak to that. Zero communication in my case as well.
- sgnls 7y agoLast week, I have had an issue where a number of domains were purged from the 2nd tier registrar (Claranet) with exactly the same symptoms (domains suspended, zone-files blown away)... and Network Solutions are to blame. An assumption of false-payment led to them suspending "300-500" accounts (mostly UK based). I am still of the opinion something far more sinister is at play... and this doesn't comfort me.
- RcouF1uZ4gsC 7y agoBe wary of being part of something that is a cost center for the company instead of a profit center. CloudFlare is selling domains at cost. That means they are not making any money from being a domain registrar, which means they will do everything to keep the cost of doing it as low possible to themselves. This means lack of customer service and use of ML dragnets for "anomalous" behavior.
- owenmarshall 7y ago.com has a price floor of $7.85. Most registrars seem to target anywhere from the $9.99 - $14.99 range for registration because, as far as I can tell, there is no real differentiation outside of price. Sure, I could spend $lots to get a dedicated account rep from MarkMonitor or CSC but that's not really feasible for my personal site. Are there really any registrars that hit a reasonable price point for individuals and offer service beyond bargain basement? Because if so I'm doing some transfers this weekend.
- ocdtrekkie 7y agoOne of the reasons I've stuck with GoDaddy is 24/7, American phone support. Their .com pricing is closer to $20 for renewal at this point, but I've called them at 2 AM before and gotten help. From previous research, at least, most domain registrars have ticket support at best. I did move all my "less important" domains to Cloudflare for cost savings recently, but they have my most important domains.
- sbarre 7y agoWhat do you consider reasonable price point? I'll speak for myself and say that all my domains have been with Hover for well over a decade now, and the times I've had to deal with their customer service, they've been excellent. In fact, I even had to call them once, and I got a human almost immediately, and that human was able to resolve my issue while I was on the phone.. I don't recall the exact issue and I'm sure it wasn't anything major, but it was still nice. So yeah, Hover. They're nice. And I think their prices are decent?
- randomdude402 7y ago
- judge2020 7y agoCan't think of a reason this domain was touched (I don't work for CF) but I'd recommend reading the threads related to this search: https://community.cloudflare.com/search?q=127.0.0.1%20audit https://community.cloudflare.com/search?q=127.0.0.1%20audit Every related incident seems to be due to either nameservers temporarily/incidentally chanced away from CF (and CF's service not re-checking it perhaps) or the registration billing failing (which doesn't look to be the case since registration expires 2021[0]). The latest change to the domain was about a week ago[0], so if that was when it was transferred to CF, it might be the first scenario. > Because Cloudflare deleted my domain registration I can't change the status from clientTransferProhibited through their dashboard so I don't think I can even leave. Unless something else happened, deleting the zone from your account doesn't affect the registration. Re-adding the domain will instantly allow you to view the registration info and likely transfer away; this would only not work if the zone is banned for some reason. 0: https://who.is/whois/danielzfranklin.org https://who.is/whois/danielzfranklin.org
- crooked-v 7y ago> Re-adding the domain will instantly allow you to view the registration info "Your domain registration configuration depends on your DNS zone configuration" is a very strange way to do things.
- iudqnolq 7y agoOP here. > Every related incident seems to be due to either nameservers temporarily/incidentally chanced away from CF (and CF's service not re-checking it perhaps) or the registration billing failing (which doesn't look to be the case since registration expires 2021[0]). The changes a week ago involves adding and deleting TXT and A records only. Cloudflare manages the nameservers I use as my registrar and I never changed them from the default. I just confirmed all of that in the Cloudflare audit log. > Unless something else happened, deleting the zone from your account doesn't affect the registration. Re-adding the domain will instantly allow you to view the registration info and likely transfer away; this would only not work if the zone is banned for some reason. Thank you so much! Trying that now.
- 7y ago
- Jerry2 7y agoThis is Google-tier lack of support and general 'customer' gaslighting.
- oefrha 7y agoUnrelated issue but sometimes Cloudflare docs/communications are not in sync with their actual system which is immensely frustrating. I was bitten a few times. For instance, a while back I forgot to renew one of my side project domains so it briefly expired for maybe a day or two. Got this email from Cloudflare saying > Your DNS records will be completely removed from our system in 7 days. > ... > Once you have completed this change, click the “Recheck Nameservers” button in your Cloudflare dashboard to ensure your domain stays active on Cloudflare. I promptly renewed, except there's no "Recheck Nameservers" button anywhere, and the dashboard still read "Moved" for maybe a day. Eventually the problem was just gone, but the communication worried me that entire time. (I do appreciate Cloudflare's service, though.)
- outworlder 7y ago> Your DNS records will be completely removed from our system in 7 days. This sounds like a plot of a japanese horror movie.
- gist 7y ago> Does anyone know what might have caused Cloudflare to delete my domain? Any ideas for how I could transfer my domain away from Cloudflare sooner? I don't get the point of 'shoot first ask questions later' type approach. Obviously it would pay to get some kind of affirmative reply from Cloudflare prior to a post which everyone here with incomplete information speculates and wastes time on (like I am doing). Also Cloudflare did not 'delete my (the) domain. It deleted the dns records. There is a difference and no I am not being pedantic either. How would 'the internet' know why this was done there could be any number of good or bad reasons. Lastly the domain is not expired and as such the registrar is required (per ICANN) to supply an auth code so someone can transfer out. Or to allow the customer to change the primary and secondary dns to another dns provider. There is zero (legitimately) that allows cloudflare as either a dns provider or a registrar to lock the domain up pretty much (other than for a legal court order) just for some reason they might decide to do that.
- iudqnolq 7y agoOP here. > Also Cloudflare did not 'delete my (the) domain. It deleted the dns records. There is a difference and no I am not being pedantic either. Thanks. You're absolutely right. I meant delete their record of the domain as it shows up in the UI of their dashboard. > How would 'the internet' know why this was done there could be any number of good or bad reasons. For many reasons luckily HN isn't 'the internet'. I've already gotten some good suggestions. > Lastly the domain is not expired and as such the registrar is required (per ICANN) to supply an auth code so someone can transfer out. Or to allow the customer to change the primary and secondary dns to another dns provider. There is zero (legitimately) that allows cloudflare as either a dns provider or a registrar to lock the domain up pretty much (other than for a legal court order) just for some reason they might decide to do that. I know. Again, I guess I was insufficiently specific. Cloudflare has warned me to expect long wait times before I can talk to a customer support rep. My question was if there's a way to transfer out without needing to wait on a slow support loop.
- johnklos 7y ago> I don't get the point of 'shoot first ask questions later' type approach. At first I thought you were talking about Cloudflare shooting first, but apparently not.
- isclever 7y agoMy takeaway: 1. Setup up monitoring on your critical domains. UptimeRobot and Hetrixtools are good starters with generous free tier. You should know when your website/email/dns isn't working. 2. Don't tie your domain registration with your DNS provider. You lose everything if something goes wrong with your account. 3. Be able to jump ship easily, have backups of your zone, already know where you will transfer to.
- djsumdog 7y ago> UptimeRobot and Hetrixtools are good starters with generous free tier Are there any open source status pages/monitor programs that have build-in checks for HTTPS, DNS records (ipv4/6), arbitrary port checks, etc? I'd rather just setup a status page/alert app on a $5 minimal DO/Vultr node and self-host/support/contribute to a FOSS program than use a commercial provider.
- iudqnolq 7y agoIf you want email or text message alerts I would assume that's a complicated enough system you would want uptime alerts on it, and so on recursively ad infinitum.
- unilynx 7y agoIf you can set up nagios (which one would probably consider an interesting evening challenge if you were already willing to go for your own monitoring droplet) setting up pushover or amazon sns (for sms) should be easy enough.
- iudqnolq 7y agoI would try to set up a completely open source monitoring setup just for fun, but once I'm paying for SNS I personally would rather just pay epsilon more and buy/rent the whole system. I get that may just be personal taste. I absolutely don't trust myself to run my own highly-reliable mailserver to send status alerts.
- hobofan 7y agoHad the same thing happen to me some years ago. Had a (not so important) domain with Gandi, which pointed to the Cloudflare nameservers, and after some time, the domain was gone from the CF dashboard together with all DNS entries. The NS records were still pointing to CF and there also weren't any anomalies with renewal of the domain. I didn't give much thought to it, as I wasn't using CF for anything in production at the time, but sad to see that it also seems to happen to other people.
- 1337n008 7y agoafter they began to turn on their own customers i moved all my domains and closed my account. looks like i have not missed out much. imagine if one day your bank decided to close your entire bank account without telling you...lol.
- potency 7y agoCloudflare lost my support when they started de-platforming people for holding opinions they didn't agree with. Censorship outside of strictly legal bounds should not be tolerated from a company as powerful as Cloudflare.
- J5892 7y agoWhat sites have they de-platformed outside of legal bounds?
- rabite 7y agoDaily Stormer, archive.ph, 8chan
- dependenttypes 7y ago> archive.ph When/why did they remove that one? Have you got any source?
- RL_Quine 7y agoWhy do you think you have a right to host with them? You don’t, you have a privilege that’s extended by them. You’re welcome to host your own thing somewhere else.
- HBKXNCUO 7y ago>You don’t You don't now. Why do successful technology companies have a right to have a proportionally large influence on the public political debate? Is it good for society to allow successful technology companies to have such a large degree of control over something so incredibly vital, merely because they were effective at running a particular type of business?
- mavhc 7y agoIs it censorship if they refuse their money for a service? Pretty sure that's just business. Are they stopping you having a website?
- dariusj18 7y agoCloudflare once deleted one of my domains because the NS records were set in the wrong order.
- LinuxBender 7y agoWhat do you mean by wrong order? Do you mean the NS records in the zone file were after a delegation / referral? What RFC was your zone breaking?
- dariusj18 7y agoIgnore me, I just went to look again, and one of the NS records was misspelled.
- jlgaddis 7y agoWrong order? Since when do NS RRs have to be in any certain order?
- dariusj18 7y agoIgnore me, I just went to look again, and one of the NS records was misspelled.
- paulfurley 7y agoFWIW I recently evaluated a few DNS companies after Namecheap ballsed up our MX records in a similar way. I actively looked for someone we could pay money to, so we are their customer (as opposed to being a free tier user, effectively a cost) The winner was DNSimple[1], who do exactly 1 thing, and they do it extremely well. And they are small enough to not take themselves too seriously[2], which I really appreciate. Oh and their normal support channel is email, and everyone in the company takes a turn. I tested out their support before signing up and quickly heard back from a competent engineer, so they passed that test too. [1] https://dnsimple.com https://dnsimple.com [2] https://dnsimple.com/dnsound https://dnsimple.com/dnsound <— bonkers
- iudqnolq 7y agoThank you. Looks like I'll just have to pay more. Any recommendations for a registrar?
- hedwall 7y agoDnsimple is a registrar as well. I have my personal domains there.
- PopeDotNinja 7y agoHurricane Electric has a pretty solid free DNS offering. I've been using it for like 10 years. https://dns.he.net/ https://dns.he.net/ I haven't needed to talk to them much, but one time I tried to add a .ninja domain, and there backend wouldn't handle it. I emailed them to report the problem at 4:49 p.m. I got an email at 7:09 p.m. the same day (2 hours 20 minutes later later) asking me to try adding it again. [1] When a free service fixes your problem in a few hours, they get +1 gold star from me. [1] I just checked my email to look up the actual times. This was on Mar 15, 2017.
- BenjiWiebe 7y ago+1 for Hurricane Electric. Right now I'm giving CloudFlare's DNS a try, but HE gave me solid service (including dynamic DNS) for years.
- jgrahamc 7y agoThis is being looked into internally and I am involved. Likely won’t post an update here as it pertains to a customer account (unless customer agrees). BTW If you, dear reader, ever find yourself so frustrated with Cloudflare that you feel like your only recourse is a blog post... my email is jgc@cloudflare.com and I’m happy to hear from people.
- diegoperini 7y agoFirst, you are awesome, really :) Second, a bunch of honest questions: Did you consult to your supervisor (or anyone with authority) to be able to bypass the support process (if there is any) like this? If so what was the response? If the response was negative, how did you convince people? After things resolve, can you kindly post how many spam or unrelated emails you receive so that it will be an example to the industry? I'd like to put my skepticism on hold and blindly believe that your post is a reflection of pure concern and not just a PR stunt for damage control.
- wila 7y agoThe CTO at Cloudflare has to ask his supervisor before trying to help someone over at hacker news? :)
- diegoperini 7y agoI didn't know which probably slightly proves some point other people made.
- stedaniels 7y agojgrahamc is the CTO of Cloudflare. His only supervisor is eastdakota/ Matthew Prince, the CEO of Cloudflare.
- diegoperini 7y agoThen I'd like to know the CEO's opinion then.
- Paul-ish 7y ago> However, I'm unable to log in to their community forum. When I click the login button I'm redirected to my dashboard, and when I then click Support on the dashboard I'm redirected back to the forum without being logged in. I suppose it's possibly an issue with Firefox blocking cookies (although I disabled tracking prevention) so it's possible this part is partly a problem on my end. I'm into issues like this more and more, where you run into some strange behavior on a website and you wonder "How did this ever make it into production?", then you open the website in Chrome and the flows work fine. I worry that Firefox is becoming less and less viable.
- _def 7y agoIf a service doesn't function properly without tracking I wouldn't blame it on a privacy respecting browser.
- mark_and_sweep 7y agoThis is not Firefox becoming less and less viable. This is developers caring less and less about supporting older browsers, less capable hardware and, I guess, long-term maintenance in general. Just had a similar case today: My Mom tried to order something online on her old Android tablet - and it didn't work. She blamed the tablet for it, saying "It's just too old, it doesn't work correctly anymore! I used to be able to order stuff on this website". I had to explain to her that her tablet is still working fine, it's just the website that is broken because it's not supporting her device (or browser) anymore. Shockingly, she listed quite a few websites, which she has used for years, which have stopped working for her in the past few months and years; all of these she mentioned as evidence that the problem must be her tablet - not the websites. When I opened two of the sites she mentioned, I wasn't too surprised to find very shiny, very modern single-page applications (with service workers registered and even WebAssembly used on one of them).. So when you are creating a modern web app, please don't just test in Chrome on your new MacBook Pro. Think about your Mom. Ask yourself: "Is this still gonna work on her crappy old device?"
- SahAssar 7y agoWell, it's also a problem of device manufacturers dropping support for devices too quickly. There are still android 4.1 devices sold on amazon, and you really can't expect web developers to support that. The manufacturer should be required to support it for the full lifetime of the device. Especially since your mom uses it to order stuff, which usually includes some pretty security sensitive information. I think you are putting the burden on the wrong party.
- jermaustin1 7y agosame thing happened to me on GoDaddy for multiple domains when I got a call from a client that their emails stopped working. All the zones were factory reset, and no backup of the zones apparently existed at GoDaddy. I was on the call with them for hours refusing to hang up until it was resolved or they would lose the remainder of my business. After 2.5 hours of no valid reason that multiple domains when back to default DNS values and no log of access to my account for moths, I let them go. That's when I moved the couple of handfuls of domains I had left at GoDaddy over to Hover. It's more expensive, but the Hover interface is better, and I trust Hover (Tucows) more (well, I trust GoDaddy less).
- behringer 7y agoThis is why you need name servers from 2 different companies and dns monitoring. It doesn't matter who your provider is. Errors happen and waiting half a week to fix it is insane.
- dergachev 7y agoOut of this very fear, when Evolving Web started using CloudFlare for DNS, we wrote this backup script that runs on cron and pushes our settings to a git repo. https://github.com/evolvingweb/cloudflare-dns-backup-tool https://github.com/evolvingweb/cloudflare-dns-backup-tool
- partiallypro 7y agoI've Cloudflare delete an entire zone before, and I could never get an answer as to what happened. They said it was deleted because the NS were changed on the domain...but they never were.
- matthewmorgan 7y agoCloudflare can suck a fuck
- Mojah 7y agoOccasions are rare where I get to say "hey, I built a thing that might help here!" - so forgive me as I take this opportunity with both hands. Whether this was a bug or a rare protective mechanisme, there will be times when your DNS provider makes a mistake and removes records. You mentioned in your post your DNS isn't hard to reproduce, but how certain are you that _all_ records are restored? How long do you have to fight DNS issues before it's OK? I built DNS Spy [1] for this exact occasion. It monitors your DNS for any changes made, keeps a version of all DNS records (current & former) and allows you to restore/download a BIND9 zone file for your zone. You can easily import this into any commercial DNS provider or in your own BIND9/PowerDNS setup. I would love to hear feedback on how DNS Spy could be improved when DNS disasters like these occur! [1] https://dnsspy.io/ https://dnsspy.io/
- hashhar 7y agoLooks really useful and fulfills a very important purpose. What good is all your backups if you can't get your services back up due to missing DNS configuration.
- iudqnolq 7y ago(OP here). That looks really useful. If I was running a real service I would definitely look into it. Because this is just the personal website and email of a college student I don't think I could justify the expense when using something like Uptime Robot to monitor if a single record points to a web server would probably give me close to the same reliability.
- Mojah 7y agoOh I absolutely agree! If you're a business, whether it's a SaaS or "just" a marketing website for your brick & mortar store, I think it's crucial to have back-ups. Most people think of backups as files, database dumps, previous versions, etc of their website. But the configuration data (in the form of DNS) isn't often considered. You're tech-savvy and can restore your DNS records because you know yoru servers' IP address and your MX records, but who else could do the same?
- parliament32 7y agoCloudflare is pretty trash regardless, but putting all your eggs in one basket (no matter which provider) is just a terrible idea.
- pvtmert 7y agoi am using api to download/backup zone every week (by cron) to gdrive (fuse drive / cheap solution) i do this for all the domains i use/manage this post has been a good reminder to check them :) imho about audit log: since they "delete" everything, nothing is left in the zone/domain. thus, initial log (127.0.0.1/creation) comes up. kind of feature of the bug/logic error.
- n0bel 7y agoWe've just been dealing with this for my company as well. Cloudflare has repeatedly deleted our DNS and cannot provide a reason why it happened. Last time thousands of dollars of PPC Ads were running uselessly.
- Karupan 7y agoStories like these scare the hell out of me. What do you do if one of the big internet corporation deletes some resource or account that is critical to your business? What happens when support isn’t responsive and you don’t have contacts in the company or your HN post doesn’t get visibility? I get it - these are free services. You should factor that into every decision. But the risk is real even if you pay for an account. I’ve been slowly moving away from Gmail to a custom domain, but something like loosing DNS records and not being able to restore them quickly is even worse. Back up everything that can be backed up, don’t rely on a single provider and always have a continuity plan!
- mc3 7y agoFirst time i've seen a https://txti.es https://txti.es posted on here. nice.
- imduffy15 7y agoI hate this paid support model. I don’t want to have to pay to tell a company I think they screwed up.
- rekabis 7y agoCloudflare considered harmful: https://www.devever.net/~hl/cloudflare https://www.devever.net/~hl/cloudflare
- MrStonedOne 7y agoEdit: the dns record export/import functionality is hidden behind the advance search drop down for some reason. Ignore this entire comment. From reading the linked helpdoc, apparently your entire domain can get removed from cloudflare if your register stops reporting cloudflare's servers for the ns records. The mere idea of having to re-enter all hundred or so of our dns records using cloudflares 1.2 second delay at every step of the way add dns record interface because namecheap bugged out for a few seconds is horrifying. There is no way to export all of these, there is no way to import or mass add and i don't think they can lean on the api to save them here. Dns records are data, dns records are sometimes important unbacked up customer data. Cloudflare does not offer a way for customers to back this data up, nor a way to restore or recover from a backup but it acts very callous with this data, deleting it in automated systems based on data from 3rd party providers. Not a good look.
- Hello71 7y agoI googled "cloudflare dns import" and "cloudflare dns export" and the first result both times was an apparently official support article giving step-by-step instructions on how to do so. I myself have used this function about six years ago, so this is not new or untested functionality.
- MrStonedOne 7y agoNever clicked that because i have no reason to look for a dns export/import button in the advance search gui... hmm. good to know about, not sure why they put it there.
- donmcronald 7y agoYou can export a zone file and the auto deletion takes days. You’re site would be offline before the zone gets deleted. I don’t like it a ton, but it’s not even close to what you’re saying.
- MrStonedOne 7y agoI only have the help article to go on and the fact that their export functionality is for some reason hidden in the advance search drop down? as i've just found out... Anywho, the help article does not state it it "takes days", so it does not have to "take days". Otherwise known as listening to the documentation not the observed behavior. Programming 101 here.
- thinkloop 7y ago> I registered for Cloudflare with a Gmail address specifically so that I could receive notifications from them if there were issues with my email setup. good idea
- errrmaybenot 7y agoReminds me of Rackspace - clown IT overrpcied. They provisioned email incorrectly and swore that once we changed nameservers to them it would work. We pointed out it wouldn't using a few basic tools and asked for an escalation, they refused and we were stuck with level 1 tech support. After a few days I found someone on Twitter who esclated, they admitted they fucked it up, but the damage was done, I was frustrated and called it day - I proceeded to cancel, they then had to cheek to say "let's all be professional here"... I can deal with occasional shit support but I can't deal with escalation refusal, or endless cycles of support tennis or being redirected to irrelvant knowledgebae articles.
- johnmarcus 7y agoCloudflare Entrprise sales pissed me off so much the last time I dealt with them that I literally registered "cancelcloudflare.com/net/org". We didn't want to renew there $25k enterprise support for a website that got 50 hits a month. It was embarrassing (for Cloudflare) how incompetent their Enterise Sales rep was. I would rather pay to host my own cache layer then ever have to deal with them again, free or otherwise.
- homero 7y agoSuspending can be a mistake, that's fine. Deleting DNS records and no notification is absurd.
- deleted 7y ago[deleted]
- birkhoff 7y agoMy CS experience with Cloudflare has been extremely awful as well. Exactly the same that I’ve was told to go to community forums. Being a paid customer, I was told that the ticket will be delayed because I’m a free user. My ticket was closed by the system automatically too because I didn’t reply if I need any further assistance that they didn’t offer in the first place all. That happened twice. What. The. Fuck.
- cm2187 7y agoI wonder how many people backup their DNS settings so they can recreate them quickly...
- andreitp1 7y agoI had the same thing happen to me with the same domain - twice. It just disappeared from Cloudflare without any notice. I run multiple domains on the same CF account, yet this has only happened with that specific one, which uses a somewhat unusual TLD (.do).
- PovilasID 7y agoHad the same or very similar issue . My domain registrar dropped all name server settings on an 'update'/whois update requirmet change Luckily I had an old export of DNS records so most were there but not all the settings were retained like DNS only or full proxy. Those were not mission critical, so not that bad but I bet some emails went to the void because there was no real warning from both my domain registration service provider and cf so I found it very 'upsetting' (I was pissed on them both)
- frf37 7y agoso what was the issue?
- peter_d_sherman 7y agoWe need a neutral, third-party service that monitors other service providers. That is, checks periodically for such things as a host being reachable, DNS working, certificates working, etc., which notifies both the service provider company and the end-user when there is a discrepancy, but most importantly, acts as a source of records in the case of a dispute -- in other words, they hold a record of what the service should be and should do. That is, when the service provider creates a service record, they would cryptographically sign it, and forward it to this future third-party service. The same for deletions. Now we can track, record, and audit that service provider's promises... That way, the service provider can't use an all-too-easy excuse like "we can't find that record in our database -- so you must not have ever created one..."