7 ms·
Tailscale state file encryption no longer enabled by default
- traceroute66 10mo agoTL;DR: Significant u-turn by tailscale. Previously with Tailscale 1.90.2 or later node state storage encrypted by default on all supported platforms. As of yesterday, per changelog, state file encryption and hardware attestation keys are no longer enabled by default. This effectively rolls back history to pre 1.90.2 and you will now have to enable it manually like you did during the public beta period (>= 1.86) of this short-lived new feature.
- shepherdjerred 10mo agowhat's the implication?
- usefulposter 10mo agoHelp center - https://tailscale.com/kb/1596/secure-node-state-storage https://tailscale.com/kb/1596/secure-node-state-storage: >Secure node state storage can help protect against a malicious actor copying node state from one device to another, effectively cloning the node. By using platform-specific capabilities, Tailscale ensures node state encrypts at rest, making theft from disk and node cloning more difficult. Marketing blogpost - https://tailscale.com/blog/encrypting-data-at-rest https://tailscale.com/blog/encrypting-data-at-rest: >What we really care about here are those private keys stored in the state file, since those are used to identify your node to the coordination server and to other nodes. We need to protect them from exfiltration. >If the Tailscale state file is unencrypted, an attacker with that kind of root access could use the file’s contents from a different machine and impersonate your node. From the perspective of the Tailscale coordination server, it’s as if your device switched to a different network and got a new IP address. We call this attack “node cloning”.
- nottorp 10mo agoSo it protects you from an attacker who already has local root?
- cronos 10mo agoNot even that. An attacker with local root can just extract the wireguard keys from process memory, or use the TPM to decrypt the state file like Tailscale would. The only scenario where it helps is a local attacker who can read the state file on disk, but is not full root. Kinda unlikely on Linux, but could happen on Windows.
- nottorp 10mo ago> An attacker with local root can just extract the wireguard keys from process memory, or use the TPM to decrypt the state file like Tailscale would. That was my point :)
- traceroute66 10mo agoHistorical blog post from tailscale (August 2025) saying how awesome and important this feature was[1]. TL;DR If you care about the stuff mentioned in that blog post (which most sensible sysadmins would) then the implication is that you are no longer protected against those threat scenarios UNLESS you manually apply the flag at install time. Which means for people using deployment scripts/tools you now need to update those to put the flag in during installation. Because previously you could rely on the feature being "on by default", which is no longer the case. [1]https://tailscale.com/blog/encrypting-data-at-rest https://tailscale.com/blog/encrypting-data-at-rest
- Thaxll 10mo agoTPM is really badly implemented. When you upgrade your firmware, OS, everything can go south. Just upgrading your firmware with bitlocker enabled can brick your PC.
- traceroute66 10mo ago> TPM is really badly implemented. When you upgrade your firmware, OS, everything can go south. Could you elaborate ? Firmware/OS should not affect TPM contents ? Otherwise e.g. TPM-reliant Windows installs would break ? In addition there are cloud scenarios where your VM has a TPM and you want to e.g .stop a malicious actor poaching your VM and running it elsewhere. Having the tailscale TPM tied to your cloud hypervisor prevents the "lift and shift" attack.
- Thaxll 10mo agoEverytime I have to upgrade my MB firmware it breaks bitlocker and I have to either use restoring keys from microsoft website or disable bitlocker encryption before the upgrade. https://www.reddit.com/r/MSI_Gaming/comments/15w8wgj/psa_tpm_is_cleared_when_updating_biosuefi_all/ https://www.reddit.com/r/MSI_Gaming/comments/15w8wgj/psa_tpm...
- stanac 10mo agoYou cant reliably store secrets in tpm and expect it to work after an os update. Windows is using workarounds during windows update to avoid breaking bitlocker. https://learn.microsoft.com/en-us/windows/security/hardware-security/tpm/switch-pcr-banks-on-tpm-2-0-devices https://learn.microsoft.com/en-us/windows/security/hardware-...
- db48x 10mo agoYou are correct. Updating the firmware or the OS does not actually erase the TPM. What is really going on is that the TPM register holds a value that is like a hash. Each time you measure the system state you update the register with a hash of the previous value and the measurement. When you ask the TPM to hold a key you specify which register value is used to encrypt the key. Later when you use the key it will fail if the TPM cannot decrypt the key. This can only happen if the TPM register has the wrong value, which can only happen if someone has tampered with the system. But voluntarily upgrading the BIOS or the OS looks exactly like tampering. The correct procedure is to unlock the keys, copy them out of the TPM, perform the upgrade, reboot to remeasure the system state, then finally store the keys back into the TPM.
- asgeirn 10mo agoFrom what I can deduce from the release notes and the linked documentation, it can still be enabled? And it relates to Windows and Linux only, and using the TPM. My guess is that unreliable TPMs made it risky to have this enabled by default.
- traceroute66 10mo ago> it can still be enabled? Yes, just like >= 1.86, you set a flag during install. But that's not the point. The point is that >= 1.90.2 it became enabled by default. The point is that most people would expect that "by default" to be a permanent fixture, i.e. a sane secure-by-default config. This means that people with automated deployments based on >= 1.90.2 can no longer rely on the "by default" and this now needs to be flagged.
- esseph 10mo agoIf your threat profile has you worried about tailscale + tpm, you probably shouldn't be running talescale unless you're also running headscale... Just a thought.
- snailmailman 10mo agoNot sure if its a "significant" u-turn, when its a relatively new feature. Its only been out for a few months, and seems to be getting rolled back because it was breaking things. Its annoying that a security benefit is being turned off, but it can be turned back on if you are confident it will not break your setup.
- traceroute66 10mo ago> Not sure if its a "significant" u-turn I would say it is because they made a big marketing blog post about it at the time[1] (August 2025). So clearly they considered it a significant new feature. The blog post ended with the words "If we don’t spot any major regressions with 1.86, the next stable release will likely turn on state encryption by default for all new nodes". It was then enabled by default 1.90.2 onwards (October 2025). That is why I would consider it a significant u-turn. [1]https://tailscale.com/blog/encrypting-data-at-rest https://tailscale.com/blog/encrypting-data-at-rest
- hug 10mo agoI don't get it. It seems like they're doing largely what they said they would. They wanted to push a feature, and they said they would if they didn't see any major regressions. Then they did see a major regression, so they pulled the feature. Exact version numbers, timelines, and builds are pretty irrelevant to that process. Or are you actually saying you would prefer they had just left their product broken for a significant portion of users, just to keep aligned with the version numbers they mentioned in a blog post?
- jsiepkes 10mo agoGuessing it was too support intensive? Caused too many issues for people who then reached out to support?
- jkaplowitz 10mo agoFrom the changelog, it seems like this may have been due to issues caused by the on-by-default setting, although I don’t work for Tailscale and am speculating here with no inside info. I wonder, would Tailscale be willing to confirm that they plan to fix whatever the issues are and re-enable this default within a short-ish timeframe? I currently have plenty of trust in the good intentions of the people running Tailscale, but with geopolitics as it currently is, I’d love to have a concrete reason even beyond that positive track record to believe that this change isn’t attempting to satisfy ease-of-surveillance concerns expressed by government agencies in whichever country.
- aiiane 10mo agoSeems like the issues in question are not within Tailscale's span of control (basically, the devices themselves with TPMs are too unreliable in the general population, so the feature is more appropriate for controlled environments that opt in to its usage).
- db48x 10mo agoThe TPM devices themselves are reliable, but using them comes with a lot of caveats. 99% of users have never heard of the TPM, and 99% of the ones who have won’t have realized that upgrading the BIOS clears¹ the TPM. Add in the fact that Tailscale users didn’t _know_ that tailscale was using the TPM and you have a recipe for users breaking things without realizing it. In an enterprise environment where you can afford to hire people specifically to care about these thing, using TPMs for additional security is a great idea. ¹: and very few of those can explain that it doesn’t actually clear the TPM. Instead it causes a different state to be measured by the TPM, and in that new state the TPM cannot unlock the keys that were previously stored in it. This is a great way to protect the computer against someone who can pull the hard drive out of the computer and try to read the data off of it, or who can substitute a different BIOS chip to get around a BIOS password, but not so great for ordinary users who want the occasional upgrade to go smoothly.
- rstat1 10mo agoHere’s the PR explaining why they disabled this function https://github.com/tailscale/tailscale/pull/18336 https://github.com/tailscale/tailscale/pull/18336 Seems like it caused tons of problems due to the variability of TPM quality among other things
- xyzzy_plugh 10mo agoThis never should have been on by default. The end user (read: administrator) needs to know they want to use the TPM. This is a huge foot gun for many devices. The accompanying changelog note hints at why: > Failure to load hardware attestation keys no longer prevents the client from starting. This could happen when the TPM device is reset or replaced. This is unfortunate as for many, many deployments, you absolutely want this on. But because it's a time bomb for certain device/OS combinations that Tailscale can't control or predict, and folks install Tailscale on pretty much everything, then the incidence of borked installs can only rise.
- candiddevmike 10mo agoAs someone with a passing interest in using TPM for crypto things, everytime I think deeply about the implementation details like this, I come back to needing some kind of recovery password/backup key setup that entirely negates the point of the TPM in the first place. They seem really neat, but I struggle to see the benefit they have for doing crypto when a tiny slip up means your users' keys are poof, gone. And the tiny slip up may not even be with your software, but some edge case in the OS/TPM stack.
- XorNot 10mo agoThe benefit is that you don't enter the recovery password most of the time. And when you do it should be rare and lead to a password reset.
- nottorp 10mo agoSame with passkeys actually.
- SchemaLoad 10mo agoPasskeys get synced between your devices so they aren't any more fragile than passwords in a password manager.
- 10mo ago
- cronos 10mo agoI'm one of the Tailscale engineers who built node state encryption initially (@awly on Github), and who made the call to turn it off by default in 1.92.5. Another comment in this thread guessed right - this feature is too support intensive. Our original thinking was that a TPM being reset or replaced is always sign of tampering and should result in the client refusing to start or connect. But turns out there are many situations where TPMs are not reliable for non-malicious reasons. Some examples: * https://github.com/tailscale/tailscale/issues/17654 https://github.com/tailscale/tailscale/issues/17654 * https://github.com/tailscale/tailscale/issues/18288 https://github.com/tailscale/tailscale/issues/18288 * https://github.com/tailscale/tailscale/issues/18302 https://github.com/tailscale/tailscale/issues/18302 * plus a number of support tickets TPMs are a great tool for organizations that have good control of their devices. But the very heterogeneous fleet of devices that Tailscale users have is very difficult to support out of the box. So for now we leave it to security-conscious users and admins to enable, while avoiding unexpected breakage for the broader user base. We should've provided more of this context in the changelog, apologies!
- Thaxll 10mo agoDid you rely on the Google go tpm lib for that?
- cronos 10mo agoYes, we use github.com/google/go-tpm/tpm2
- sydbarrett74 10mo agoThat's an eminently reasonable and logical policy. Thanks for the context.
- traceroute66 10mo ago@cronos Question: You link to https://github.com/tailscale/tailscale/issues/17654 https://github.com/tailscale/tailscale/issues/17654 where a user states[1]: "Previous workaround from some comments (TS_ENCRYPT_STATE=false, FLAGS="--encrypt-state=false") didn't help on this problematic Debian 13 host" And the same user states "I confirm this issue is NOT found anymore with tailscale version 1.92.1". Could you provide a little extra context to clarify those types of comments which seem to suggest it wasn't state encryption after all ? [1] https://github.com/tailscale/tailscale/issues/17654#issuecomment-3652622607 https://github.com/tailscale/tailscale/issues/17654#issuecom...
- deleted 10mo ago[deleted]
- lubasara 10mo ago[dead]
- lubasara 10mo ago[dead]
- zero0529 10mo agoThank god, I was running Tailscale on a nixos machine on some really old hardware and I couldn’t figure out why it kept crashing. It was because of this but it just failed silently.
- sp_c 10mo agoSo in linux it looks like we just update /etc/default/tailscaled with: FLAGS="--encrypt-state" ...and hope for the best? edit: I see this in my logs, I guess it is working: migrated "/var/lib/tailscale/tailscaled.state" from plaintext to TPM-sealed format
- 486sx33 10mo ago[dead]
- maximgeorge 10mo ago[dead]
- l34zgl45y 10mo ago[flagged]
- quotemstr 10mo agoAnd this is why in computing we can't have nice things. Any mass market profit can't, in a business realistic evaluation, ship something that breaks even 1% of users. Consequently, we're stuck with lowest common denominator everything and have a hard time delaying software fixes for what ails us. Instead of fixing things, we are best encapsulate the damage. If I were running Tailscale, I'd say "Fuck the people with broken TPMs. Fix your computers. We're going to be secure by default." I guess there's a reason Avery and not I call the shots there
- maufl 10mo agoOh, I got bitten by that! I have my work Linux installation on an USB stick so I can boot it on either my desktop or laptop and one day tailscale stopped working. I thought that might be a rare situation, but it looks like TPM based encryption failed for other reasons too.
- unit149 10mo ago[dead]
- issung 10mo agoMy Tailscale was broken for the past month and I only just fixed it yesterday, and today this patch is released that would have made it a non-issue. Updating my BIOS caused the issue. The main problem was that Tailscale's behaviour was very poor in this case. It simply got stuck "Starting" and never provided any error information.