4 ms·
DNSSEC failures are how you get people to disable DNSSEC
- yakkityyak 3y agoThe experience around just setting up DNSSEC when I tried it on Google domains was so incredibly frustrating due to the time delays and pain points surrounding correcting mistakes. Don't know if it has gotten any better.
- silisili 3y agoCorrecting mistakes is a pain point, but really applies to DNS as a whole rather than just DNSSEC. I think a lot of the problem stems from people using really long TTLs for the keys, which used to be the standard advice. That said, as someone who used to manage DNS and DNSSEC at a TLD level, I will admit that documentation and best practices are poor. I remember asking someone why it was this way, and he told me it was because people want to make money contracting, so were less than willing to make it accessible. Once you understand what's happening, you can make a little cheat sheet and it's actually really simple. But it seems like everyone forges ahead their own way, myself included.
- esaym 3y agoShow us your cheat sheet...
- silisili 3y agoI've been out of the business for seven years, and don't have much anymore as I've switched jobs a few times. It was mainly around process and timings. Bind9 has a pretty good guide on it. The easy option is to just add the new key, let everything sign, then later(after at least a TTL period, probably longer to be safe) remove the old. The other way, that we did, is to publish keys before you use them, then retain the old key after the signing key swap, for a TTL period each. That keeps the zone size small. Lastly, don't roll a KSK and ZSK at the same time. It's doable, but not worth the dance in any situation. That said, if you have any specific questions, I'm happy to help.
- cyberax 3y agoOne trick to make it easier: set all relevant TTLs to 1 minute before trying something dangerous. Plenty of resolvers ignore that, but the main ones (1.1.1.1, 8.8.8.8, etc.) respect that.
- tptacek 3y agoThe protocol wasn't designed on purpose to do this, but I think it's worth keeping in mind that this is why centralized DNS providers do DNSSEC features and promote DNSSEC: it's a thing that keeps you dependent on SAAS providers, because the DIY is so difficult. Keeping your domain anchored to a particular platform is a powerful customer retention mechanism; you're likely to do everything you can with that domain with that same platform.
- xvilka 3y agoWe need a truly decentralized alternative, DNSSEC isn't that.
- silisili 3y agoDNS is de facto decentralized. Stand up your own root, it's that simple. DNSSEC has no bearing on how centralized something is - it works the same on the current system as it would on your own new DNS system.
- noman-land 3y agoWait, what? How can I be my own certificate authority?
- silisili 3y agoOpenssl works just fine, if you're asking in earnest. Either way, it has no bearing on DNS.
- cyberax 3y agoCreate your own root certificate, import it into your browser, and into the OS certificate store. Then just sign CSRs with this root certificate's private key. This is pretty much what all large corporations do for their internal infrastructure.
- RandomBK 3y agoThis of course, only works for your own internal use. I feel this disclaimer needs to be called out explicitly. If you're trying to do something that will be trusted by the global internet, you'll need to use an official CA, ergo the complaints of DNSSEC being too 'centralized'.
- IcePic 3y agoThen again, what the "global CAs" are doing is not wholly different in a technical level than what a company with their own internal CA does, except that I hope the global CAs take security very very seriously, and they pass an expensive audit. In the end, it is just some company or organisation that creates a root CA cert and intermediaries, and then jump through hoops to make browsers and OS cert stores add their certs. Not saying it is easy, just that the technical parts are rather similar, the rest being a cumbersome way to prove you are trustworthy by showing off your processes and fences around the signing boxes.
- HaroldBolt78 3y agoI had the power turned off at home about a year ago. I think it was due to maintenance by the utilities company, or maybe I did it for DIY reasons. Anyway, after it came back on, it took me far too long to realise why my internet was down. All DNS responses were being rejected. It turns out, my Pi-Hole DNS server - which had forgotten the time, of course - was trying to resolve the IP of an NTP server so it could fetch the current time, but this name resolution was over DNSSEC which requires the current time. You need the time to get the time. I turned off DNSSEC and haven't looked back since.
- bouke 3y agoI had exactly the same issue happening to me! I’ve since added a Real Time Clock (RTC), so the Pi shouldn’t forget about the current time anymore. That is until the battery runs out of course, for which there is no monitoring in place. So yeah maybe I should disable DNSSEC as well. At least we need to get better at building fault tolerant systems. Maybe Pi-Hole needs to be aware of and handle this scenario better: a special “we’re booting up and we don’t have the current time so don’t do DNSSEC yet”.
- toast0 3y agofake-hwclock[1] is useful for this. I don't know much about Pi-Hole, but if it involves writable storage and has a package manager, fake-hwclock may be available from there. > At least we need to get better at building fault tolerant systems. Security and fault tolerance are at odds here. Failing closed when you can't validate is correct, but not so useful. Properly modeling the clock might give you a way to get bootstrapped, but then an adversary would arrange for a bootstrap situation so they could interfere with your clock and then make use of the incorrect time for their nefarious purposes. [1] https://git.einval.com/cgi-bin/gitweb.cgi?p=fake-hwclock.git https://git.einval.com/cgi-bin/gitweb.cgi?p=fake-hwclock.git
- boringuser2 3y agoDoes pihole even have value anymore with DNS over https? I'd just implement that in 0.2 nanoseconds if I were a Samsung engineer making an intrusive adserver.
- mnd999 3y agoWhen I’ve enabled it in systemd.resolved it breaks every hotel / conference centre / airport WiFi. Maybe that’s a good thing but it’s pretty inconvenient.
- deleted 3y ago[deleted]
- elp 3y agoI run the signing systems for a bunch of TLDs and honestly it scares me. .NZ was just the latest public failure. If multiple large registry operators employing people who wrote the DNS and DNSSEC rfcs have had failures then how can you expect regular mortals to get it right? Transferring your domain and your old ISP had dnssec on but the new one knows nothing about it and didn't check ... Down you go. Mess up the key signing key change over procedure while transferring to a competent DNSSEC provider ... down you go again (although hopefully for a shorter time). We do best practice DNSSEC, using fancy HSMs to do so, but if my fancy, proprietary, badly documented Thales HSM dies and the next person with my job has trouble getting the new box configured... no changes to the zones until its fixed. Take long enough that the RRSIGs expire ... down YOU go again. If at that point they throw out the HSM and roll regular DNSSEC using Knot or something it will probably be at least 24 hours for the zone to work properly even with IANAs emergency procedures. I'm paranoid so we have 3 identical HSMs and detailed restore procedures but there is zero guarantee that if I ever leave my replacement will get it right. You can't hire dnssec staff, they are impossible to find, you have to train on the job. The basic idea is excellent but the implementation in the real world is horrible. If dnssec usage was wide spread we might reach some decent maturity with the tools and protocols, but I can't see that happening any time soon.
- donttellmypeers 3y ago> If dnssec usage was wide spread we might reach some decent maturity with the tools and protocols, but I can't see that happening any time soon. DNSSEC's lack of maturity is a symptom of DNS itself not having a healthy ecosystem. I feel like it'd take as little as a half-percent of the people writing new HTTP tooling to noticeable improve the DNS ecosystem as a whole.
- silisili 3y agoTotally. DNS was always an IETF RFC bound thing. Recently, big players just kinda started making their own decisions. I'm not sure which is better. Having big companies control the internet is definitely bad, but the pedantry and bikeshedding the IETF offers can be equally bad. I'd probably rather see the two cooperate/converge, but that's a pipe dream.
- roadiase 3y agoDnssec is so hard to manage right
- jongjong 3y agoWhy isn't Blockchain tech used to solve the problem of domain name resolution? Why aren't these solutions getting any traction? Tokens would serve as a built-in payment system for purchasing domain names and also as a spam prevention mechanism, it's decentralized and therefore wouldn't rely on any centralized authority, the incentives for running nodes would be built in... They are highly replicated and can replicate without limit which means it would be able to scale to support unlimited reads (which is what matters most when it comes to a DNS system). It would be distributed geographically and therefore could provide the fastest lookup times possible as you could run your own node to get the lowest lookups for existing domains... Also, ownership of domains would be cryptographically secured. WTF is this not getting any traction? Why aren't browser makers adopting this or enabling Blockchain-based domain lookups by default? There are solutions like Unlimited domains but they're not actually being rolled out...
- deleted 3y ago[deleted]
- jongjong 3y agoI forgot the mention the most relevant aspect; that because you could run a node on-premises which syncs up with all the latest domains, may not even need encryption... But if you didn't want to run your node on-premises, you could have an encrypted tunnel to someone (or a company) which does.
- rightbyte 3y agoI think the main problem is the expiration dates. Like, why do they exist? I see no point in having a ticking bomb in your setup.
- mike_hearn 3y agoThe whole issue of expiration in cryptographic systems is extremely fraught. My gut sense is that X.509 certificate expiry is the most common cause of mega-outages these days, where by "mega outage" I mean the sort of thing that takes down entire cellphone networks or major online services for hours at a time. In the past I've tried arguing against this practice of immediately cutting all communication because a deadline expired, but cryptography as a field is highly resistant to business requirements analysis of any kind. No CEO in their right mind would ever agree to having expiry in their infrastructure but the decisions are made by people far away, very deep down in organizational stacks, and it honestly doesn't occur to most business-side people that anyone could be crazy enough to litter miniature time bombs all throughout mission critical HA infrastructure anyway. Part of the issue is misalignment of requirements. Keys don't actually expire, of course. In the old RSA days there was a notion of a sort of slow rusting as factoring attacks tended to get better over time, so every so often you'd want to bump the size of an RSA key to keep it secure. But that wasn't something that had to be done on a deadline. When elliptic curve cryptography went off-patent and started getting deployed it fixed that, and a secp256r1 key made today is as good as one made 15 years ago. So what is the purpose of expiry, then, in today's world? For DNSSEC, it seems to be a kind of encoding of an assumption that keys can leak without anyone being aware of it, whilst the holes being used to do so get sort of ambiently patched anyway again without anyone being aware that they've done so. If you assume a world where keys are constantly being stolen and abused, then regular rotation can help. However, that's not the real world! These keys are stored in HSMs. Has there ever been a leaked DNSSEC key spotted being abused in the wild? Certainly it's questionable whether the juice is worth the squeeze given the outages it causes, and the way these undermine the entire system. For certificates there's that, but the original justification was actually about constraining the size of CRLs. If certificates don't expire then in theory any decommd cert has to be recorded and stuck on a revocation list forever, which would mean they can never shrink. How exactly to handle phaseout of old keys and certificate details has experienced a lot of churn over the years with CRLs, OCSP, OCSP Stapling and nowadays CRLsets all doing similar tasks. Still, this is primarily a logistics and cost management issue rather than an actual security problem. The task could also be done with gradual phase out in which at first expired certs only trigger warnings to developers via the browser console, Google Search console etc. Then later it graduates to a warning in the URL bar. Then eventually TLS stacks start increasing the latency on connections, etc. Then once artificial latency passes a certain threshold that's basically equivalent to an outage anyway and you may as well just return an error immediately. Lately there's another reason for expiry, with short expiry times being enforced by the CA/B Forum so they can change the rules for certificates quicker, not keys. It took a long time to phase out SHA1 and introduce CT, the people who set the certificate rules didn't like that as their job is security, so by making certs constantly expire they hope to push people into setting up automated renewal pipelines which in turn means they can then do things like mandate new algorithms, change the contents of certificates and so on. Constant expiry certainly helps them do their job, but it makes everyone else's job harder as automated renewal isn't always easy especially outside the context of simple web servers that use LetsEncrypt. Summary: If tomorrow I was temporarily made King Of The Internet and told that my only mission in life was to increase availability, I'd start by altering X.509 libraries to implement gradual degradation instead of hard expiry, e.g. with a sleep(time_since_expiry) call somewhere in the stack. I'd also default to simply ignoring expiration outside of TLS context, because X.509 gets used in a lot of places that aren't TLS and they're often hard to reach and fix e.g. certs that are hard-coded into embedded devices. You do NOT want your electricity grid to have an outage caused by certificate expiry. In those contexts it hardly matters because the PKIs are often custom and not very large, so CRL sizes just don't matter. You can easily list every revoked certificate for the entire lifetime of the system, if you even need PKIX revocation support at all, and for those use cases expiry is probably a much bigger business risk than attacks using obsolete algorithms.
- teddyh 3y ago> Exhorting people to not screw up things like DNSSEC KSK rollover clearly hasn't worked, so the only real solution would be better ways to automatically recover from it. No, the solution is to automate the rollovers, so that they can’t be screwed up by manual handling. <https://github.com/DNSSEC-Provisioning/music https://github.com/DNSSEC-Provisioning/music>
- teddyh 3y agoAlso, validation errors is how you get people to ignore XHTML. Strict validation is tricky. With HTTPS, validation was initially very lax, and AFAIK, most SMTP servers still do not validate the certificates of the destination SMTP servers. It is hard to get people to move to stricter validation once habits and working systems have been established which depend on lax – or no – validation. Some try to avoid this by specifying strict validation in advance, like XHTML and DNSSEC, which obviously has its own drawbacks.