11 ms·
Moving our Encrypted DNS servers to run in RAM
- kisamoto 3y agoMullvad encrypted DNS is also available to all - whether paying for VPN services or not. In addition they also support optional content blocking[1] via blocklists, just set the desired TLS/HTTPS DNS server. [1] https://github.com/mullvad/dns-blocklists https://github.com/mullvad/dns-blocklists
- Etheryte 3y agoFYI the standalone encrypted DNS profiles for macOS (.mobileconfig files) do not work if you're using Little Snitch or Lulu to block unwanted network traffic. This is a limitation of macOS that Apple hasn't bothered to fix.
- WirelessGigabit 3y agoDoesn't work how? Little Snitch / Lulu don't use the profile? As in DNS traffic is not encrypted? Little Snitch / Lulu die? Do you have like a ticket for more explanation?
- Etheryte 3y agoYou can have one or the other active at a time, but not both simultaneously. If you have the DNS profile turned on and then proceed to turn on Little Snitch/Lulu, the DNS profile will get turned off by macOS and whatever other DNS configuration you have will take precedence. It's mentioned in passing by Mullvad in the docs [0]: > If you have more than one profile added then macOS seems to use only the last one that you installed. If you remove a profile then it appears that it will not use any of the other profiles. This isn't unique to Little Snitch/Lulu and Mullvad though, you would run into the same problem with any application that creates a network filter by using a profile, see [1] for example. Since the full blown Mullvad VPN app does not use system profiles, it does work in tandem with Little Snitch/Lulu. [0] https://mullvad.net/en/help/dns-over-https-and-dns-over-tls#macOS https://mullvad.net/en/help/dns-over-https-and-dns-over-tls#... [1] https://git.codeproxy.net/paulmillr/encrypted-dns/issues/13 https://git.codeproxy.net/paulmillr/encrypted-dns/issues/13
- TheNewsIsHere 3y agoAs a tangential aside: this limitation on Network Extensions is infuriating. I can understand why it exists, but one of the results is that if you’re using an Endpoint Security extension, it can’t also act as a filter. I’ve never been able to get CrowdStrike Falcon and Little Snitch to work on the same machine because of this. It would be nice if you could have n filters (and a limit on n isn’t unreasonable).
- Erratic6576 3y agoI guess the solution is to use unbound
- quyleanh 3y agoEven when I don’t use any firewall app, the DNS profile doesn’t work on macOS. There is a bug that other apps (and even system) can bypass it. Not like in iOS, DNS profile work very well.
- SV_BubbleTime 3y agoCan you help me out with the threat model for encrypted DNS? As I see it, the model is that I need to contact a site, and use DNS to get the IP - but either my ISP or my VPN provider can see I’m visiting that IP whether I looked it up just now, or knew it already. So unless the model is I’m using a VPN, but am leaking via someone else’s logging DNS, I don’t really get it.
- Etheryte 3y agoPersonally my main motivation is I don't want anyone to snoop what sites I visit and then sell that data, my ISP included, but there are other issues too. To quote Cloudflare [0]: > Queries could be directed to a resolver that performs DNS hijacking. For example, in the UK, Virgin Media and BT return a fake response for domains that do not exist, redirecting users to a search page. This redirection is possible because the computer/phone blindly trusts the DNS resolver that was advertised using DHCP by the ISP-provided gateway router. [0] https://blog.cloudflare.com/dns-encryption-explained/ https://blog.cloudflare.com/dns-encryption-explained/
- _Algernon_ 3y agoCant your ISP reverse lookup the IPs you connect to based on your IP headers anyways?
- jrockway 3y agoThey can, but pretty much everything will reverse lookup to Cloudflare or AWS.
- gruez 3y agoSNI reveals the exact site though
- mmwelt 3y agoAs I understand it, this is now being addressed by ESNI[1] and ECH[2]. Hopefully ECH will continue being more widely deployed. [1] https://www.cloudflare.com/learning/ssl/what-is-encrypted-sni/ https://www.cloudflare.com/learning/ssl/what-is-encrypted-sn... [2] https://blog.cloudflare.com/encrypted-client-hello/ https://blog.cloudflare.com/encrypted-client-hello/
- exxos 3y ago[dead]
- elif 3y agoI would like to see real statistics but my gut feeling from running read/write intensive data applications on SSD and ECC RAM is that both of them fail often enough that this move is somewhat lateral in terms of resiliency. But in terms of clawing raw performance decimals, I applaud the effort. This would be a fun redis project.
- stephen_g 3y agoThat's not the point - they promise to keep no logs, and the best way to demonstrate that they don't is to not even have any fixed storage in the servers at all (they presumably netboot from a read-only image). So they have done this with their VPN servers, now they are rolling out the strategy to DNS servers as well.
- elif 3y agoThat's fine but as long as it is connected to a network you cannot prove a system to be log-free. The majority of systems I've worked on used a message queue of some kind for logging rather than a file on disk. I suppose it could be useful to expedite downtime in the case that they are subpoenaed, but I doubt claiming to be diskless would prevent any subpoena.
- bragr 3y agoWith this kind of security, you're more guarding against your servers being seized or intelligence agency installing an implant. > I doubt claiming to be diskless would prevent any subpoena It does in Sweden. They've repeatedly squashed subpoenas > long as it is connected to a network you cannot prove a system to be log-free. Nothing is perfect. If you read their audits, it's clear admins could get at your traffic if they wanted
- ThePowerOfFuet 3y agoThis is why they have their architecture audited (and release the audit reports). https://mullvad.net/en/blog/2022/6/22/vpn-server-audit-found-no-information-leakage-or-logging-of-customer-data/ https://mullvad.net/en/blog/2022/6/22/vpn-server-audit-found...
- throwaway914 3y agoAs a naive outsider: Does RAM in 2023 encrypt all its contents? Can we reliably "forget" the contents of what's in RAM by powering-off and letting some encryption key somewhere fade from a smaller, more volatile piece of memory? I'm curious what has been done to mitigate cold-boot attacks in RAM. What has changed in hardware? Are the keys to encrypt contents of RAM kept in the TPM or something?
- josephcsible 3y agoIsn't that what Intel TME does?
- throwaway914 3y agoThat does look like it: https://www.intel.com/content/www/us/en/developer/articles/news/runtime-encryption-of-memory-with-intel-tme-mk.html https://www.intel.com/content/www/us/en/developer/articles/n... Looks like AMD has an equivalent: https://www.tomshardware.com/news/intel-mktme-amd-memory-encryption,39467.html https://www.tomshardware.com/news/intel-mktme-amd-memory-enc... I wish there were a sort of checklist that prints out in dmesg saying "Your RAM is encrypted: (check) Your machine has these mitigations against cold-boot attacks: ...some list here... (check, check, check)" In some contexts, I love how far we've come with secure boot, attestation, speculative execution mitigations, etc. but it's very difficult for the average Linux user to understand what they should expect for hardware security, and then work toward enabling all of it. I love what Gnome did in the last year in Settings -> Device Security: https://www.phoronix.com/news/Ubuntu-No-GNOME-Device-Security https://www.phoronix.com/news/Ubuntu-No-GNOME-Device-Securit... There's a detailed list here of what it's looking for: https://forums.debian.net/viewtopic.php?t=152705 https://forums.debian.net/viewtopic.php?t=152705 Not a fan of all the bound-to-Intel technologies. I wish the concept of what's being secured were identified, then made available in dmesg or through some /sys or /proc interface.
- adrian_b 3y agoAMD Epyc servers have offered encrypted memory for many years. Intel is only now catching up.
- LinuxBender 3y agoAre these physical servers or VM's? I ask because some VM types can be frozen/shapshot/cloned or live replicated including memory contents as is done in some VPS providers for lawful requests. From the bare metal host anything can be accessed in a VM or container. Do they have a diagram of their physical setup? [Edit] - Are there any Mullvad architects here that can help us avoid going down theoretical hypothetical rabbit holes and turtle stacks?
- dijit 3y agoMullvad has been pushing system transparency a lot (which is a security system specifically designed for bare-metal), so it's fairly safe to say that it's based on bare-metal. https://www.system-transparency.org https://www.system-transparency.org
- LinuxBender 3y agoMaybe it should be safe to say? but do they actually have a diagram of their physical and logical setup that shows they use bare metal only and not VM's or containers? Is that documented in audited security controls?
- dijit 3y agoSuch a weird discussion because we're talking about a company that is going from the very fundamentals of operating system design and the boot process of bare metal machines in order to get more security. It's very antithetical to that goal to add more layers of abstraction that could be buggy or reduce security. We're talking about motivated and intelligent people purposefully trading off convenience for security. Containers, btw, cannot be snapshotted. So that's a weird thing to put into your question. They've also discussed booting UEFI directly to VPN nodes: https://mullvad.net/en/blog/2022/1/12/diskless-infrastructure-beta-system-transparency-stboot https://mullvad.net/en/blog/2022/1/12/diskless-infrastructur... It's completely weird to argue that they might introduce a layer of insecurity here. Directly goes against everything else they're working on.
- deknos 3y agothey should at least opensource their implementation. they may have errors there. should we just trust everything?
- dewey 3y agoWhy the entitled attitude? They do a lot of open source contributions already so maybe just give them some time instead of immediately disregarding the whole improvement? Small steps into the right direction. https://mullvad.net/en/help/open-source https://mullvad.net/en/help/open-source
- zlg_codes 3y agoMaking a suggestion isn't entitlement. Maybe reconsider the foul void that accusation came from.
- dewey 3y agoIf someone suggests something to me they usually don't say "you should at least". This sounded more like an entitled comment you see every day on open source repositories and making it sometimes not fun to be an open source maintainer.
- k8sToGo 3y agoPretty sure they have been audited recently.
- lopkeny12ko 3y agoI don't understand this. Don't all Linux processes "run in RAM"? It sounds like what they have actually accomplished here is eliminate all disk I/O requirements from their DNS service, i.e. it works against a read-only mounted root filesystem now. That's not really the same as "running in RAM".
- oxygen_crisis 3y agoOnce they have everything running in RAM, they can have racks of servers that contain no persistent storage whatsoever, and are therefore subpoena-proof. > On April 18 [2023] at least six police officers from the National Operations Department (NOA) of the Swedish Police visited the Mullvad VPN office in Gothenburg with a search warrant. They intended to seize computers with customer data. > In line with our policies such customer data did not exist. We argued they had no reason to expect to find what they were looking for and any seizures would therefore be illegal under Swedish law. After demonstrating that this is indeed how our service works and them consulting the prosecutor they left without taking anything and without any customer information. https://mullvad.net/en/blog/2023/4/20/mullvad-vpn-was-subject-to-a-search-warrant-customer-data-not-compromised https://mullvad.net/en/blog/2023/4/20/mullvad-vpn-was-subjec...
- 8organicbits 3y ago> contain no persistent storage whatsoever, and are therefore subpoena-proof. Nonsense. The linked post shows that they didn't have the sought information. If the information was present in RAM, they'd just read it from RAM.
- itslennysfault 3y agoI assume they only hold what the server is doing in any given moment in ram and it is overwritten as new requests come in. By the time the police showed up with the subpoena the data they wanted was long gone.
- theshrike79 3y agoPlease tell me how you would go about reading something that happened two weeks ago from RAM?
- j-bos 3y agoDoes anyone know how Mullvad vpn security compares with Proton vpn?
- drexlspivey 3y ago"In April 2019, upon the order of the Swiss judiciary in a case of clear criminal conduct, we enabled IP logging against a specific user account which is engaged in illegal activities which contravene Swiss law. Pursuant to Swiss law, the user in question will also be notified and afforded the opportunity to defend against this in court before the data can be used in criminal proceedings"
- protonmail 3y agoNote that this applies to Proton Mail. Proton Mail is considered to be a communication service, and in most countries (including Switzerland), communication services are regulated to some extent. All companies need to comply with the local law. That being said, Swiss law is very restrictive, and there are a lot of hurdles that one needs to jump through to get a court order. And even with a court order (and has been proved multiple times in court), there is no way to break Proton Mail's encryption. Privacy is not the same as anonymity, and due to the way the internet works, if anonymity is what you are going after, you have to exercise proper infosec and take preventive measures, such as using Tor or VPN. Under Swiss law, the treatment of VPNs is different, and VPNs can indeed be no-logs, like Proton VPN is. No-logs VPN is also possible in other countries as well. What makes Switzerland different, and possibly unique, is that within the current Swiss legal framework, Proton VPN also does not have forced logging obligations. You can find out more here: https://protonvpn.com/blog/transparency-report/ https://protonvpn.com/blog/transparency-report/.
- ziddoap 3y agoA quote with no context or attribution always makes it fun to figure out what someone intends to say with their comment. This quote is in reference to ProtonMail in 2019, a fairly widely reported on case at the time. This would be a point in favor of Mullvad, as Mullvad has demonstrably squashed a subpoena in the past, thanks to their data-minimization policies (no PII for signup, no PII options for payment, diskless infrastructure, etc.). Note that this is more related to privacy, and not as much security. If the parent comment was specifically talking about security, I would recommend reviewing the software and infrastructure audits made available by either company. I'm not aware of any major fault with either, from a security perspective.
- naet 3y agoI was extremely happy with everything mullvad was doing until they announced they were discontinuing port forwarding, which is important to a niche part of my setup. I understand the reasoning, but if it came back ever I would again happily be a customer.
- vGPU 3y agoVPN port forwarding is basically a “botnet operators welcome” sign from what I understand. The benefits of disabling it are greater than the downsides for Mullvad.
- rbut 3y agoI use Cloudflare's DoH DNS (https://1.1.1.1 https://1.1.1.1) and tunnel that over Mullvad's VPN. That way Mullvad can only see the IPs I visit, Cloudflare only see DNS requests coming from the VPN IP, and my ISP sees nothing. Mullvad's DoH DNS seems nice, but it provides potentially less privacy than the above, so I won't be using it.
- ThePowerOfFuet 3y ago>That way Mullvad can only see the IPs I visit, I hate to break it to you, but if they wanted to look at the SNI on all your TLS connections, they could; ECH is not really a thing yet.
- rbut 3y agoYes you're correct if Mullvad is performing packet inspection then there isn't much I can do about that. My point stands though about not putting all eggs in the one basket and using a different DNS provider than that of the VPN provider.
- tucnak 3y agoNo, I don't think you understand. "That way Mullvad can only see the IPs I visit" is your point and it _doesn't_ stand. There isn't really multiple ways of looking at it; the point you were making is exactly the point that was refuted, and all this "my point stands" is the weakest shit in the whole white world— a compulsion, almost inertial in nature, where by attempting to save face we present ourselves clueless. Now, all this "putting all eggs in the one basket" and common sense knowledge would be very adorable had you not been wrong, and had it not been a bad advice leaving you at a disadvantage. Good luck
- coldblues 3y agoI don't think it's recommended to use another DNS with Mullvad because it makes you more unique.
- herpderperator 3y ago> Today we can announce more steps forward - our Encrypted DNS service has also been converted to run from RAM! Where exactly do services run from if not RAM? The post is very plain with no explanation of what "running from RAM" means. When you execute any binary it gets loaded into RAM as per standard operating system procedures...?
- 10000truths 3y agoThey mean diskless. There's no persistent storage attached to the machines that run the DNS servers, because the entire operating system is loaded into RAM directly from PXE boot. So, if the hardware ever gets seized by law enforcement or rogue actors, there is no risk of exfiltration of private user data.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- UberFly 3y agoVPN provider OVPN (ovpn.com) has been doing this as well.
- plygrnd 3y agoI contacted Mullvad, who confirmed that they only use bare metal - no virtualization or containerization whatsoever. https://ibb.co/XLDQGGt https://ibb.co/XLDQGGt