22 ms·

Tailscale SSH

759 points by ignoramous 4y ago

jamiegreen 4y ago
Forgive my ignorance, but what is the benefit for an individual to run this? I currently just use 1.1.1.1 by Cloudflare on my two main devices....not realloy sure I understand what the advantage of this is?
newfonewhodis 4y ago
What happens if I use Tailscale SSH and Google (or whatever IDP) decides to ban my account? Is there a break-glass or something that would let me either change IDPs or re-enable openssh-based access without losing my servers?
- leohonexus 4y ago
  Totally, I’m a happy customer, but completely relying on Tailscale for out-of-band SSH doesn’t sit right with me, especially for personal machines. One ban and poof, you can’t access your own servers. And to be honest, I understand the appeal of not having to muck around with the system, but SSH isn’t that cumbersome once you’ve set up your /etc/hosts, and encrypt your id_rsa files. Couldn’t get any easier than `ssh <machine>`.
- bluehatbrit 4y ago
  This bothers me as well as a personal user. I'd love to go all-in on tailscale but having companies like Google running the identity is really off-putting. For now I think I'll be leaving the tailscale SSH functionality and keeping my own setup. I also have a static IP at home which is allowed access to my remote dedicated box. If they enabled some other identity provider that I could either self-host, or use their own with an email + password then I could close off that extra hole but for now it feels too risky for me. On a corporate side I don't mind so much, SSO is so standard and I'd feel perfectly comfortable using Okta or Google SSO because there seem to be far fewer stories of an entire GSuite being banned with no recourse. Tailscale is great and has solved a bunch of problems for me, but the idea of having a seperate IDP that have so many horror stories around them freaks me out enough to slightly lower my security to account for it. Thankfully there are fewer stories like that about GitHub so I'm using them for now.
mountainriver 4y ago
Tailscale is my absolute dream networking solution, I would go as far as to say it will ultimately change how we develop applications in the future
- birdyrooster 4y ago
  If the auth flow was as good as Touch ID and no window switching, yeah it would be acceptable but this flow would give me a headache with all the flashing.
  - dave_universetf 4y ago
    TouchID and related are on the list. Hooking into the existing auth flow was the easiest to get this out the door (and more desirable for some companies who want the audit event in their SSO stack, arguably), vs. figuring out when to nudge people to enroll physical factors and so forth. But I definitely also want "tap your security key please" as an option :)
sandstrom 4y ago
Good! Boundary (https://www.boundaryproject.io/ https://www.boundaryproject.io/) by Hashicorp needs some healthy competition. Teleport is also a tool in this space, for those looking for alternatives.
- sandstrom 4y ago
  And for anyone looking at Tailscale, I should also mention ZeroTier (https://www.zerotier.com/ https://www.zerotier.com/). In my opinion they have better tech, but they are pretty bad at packaging it, and bad at making it work for actual use-cases. Tailscale seems to be much more clever around building out stuff (like this one, SSH) that actually goes all the way for a particular use-case. ZeroTier feels more like a building block, where you need to bring more stuff yourself. Either way, both are awesome pieces of technology, and really useful!
  - philosopher1234 4y ago
    Why is the tech better?
  - glitchcrab 4y ago
    I had no end of problems with zerotier, connections would randomly drop between machines even when they were on the same LAN. No such problems with tailscale
infogulch 4y ago
I'm very interested in Tailscale for both personal and business use-cases, but I'm rather put off by the stark centralization of offered identity providers: Microsoft, Github (Microsoft), Google, okta (?). What are the chances that Tailscale would offer authentication using decentralized/self-hosted identity providers like Ory ( https://www.ory.sh/ https://www.ory.sh/ )?
- mdeeks 4y ago
  They offer custom SSO providers using SAML or OIDC https://tailscale.com/kb/1119/sso-saml-oidc/ https://tailscale.com/kb/1119/sso-saml-oidc/ Unfortunately they are locked behind Enterprise pricing because of the extra help and debugging needed to get them working. Maybe at some point this will be offered standard though.
  - grinich 4y ago
    Small plug for my startup (hope that's ok!) - if anyone else is looking for an easy way to set up SSO, you should check out the WorkOS Admin Portal feature[0]. It's essentially an onboarding wizard that works for any identity provider. This makes the SAML/OIDC configuration self-serve, which in turn allows you to easily provide SSO to anyone who wants it. The UI can also be branded with your logo/colors and run on your own custom domain. [0] https://workos.com/admin-portal https://workos.com/admin-portal
- doublepg23 4y ago
  There is also the self-hostable Headscale implementation.
  - sandstrom 4y ago
    In case anyone is looking for the URL: https://github.com/juanfont/headscale https://github.com/juanfont/headscale 5k stars on Github, and lots of activity. Seems very interesting!
    - jsmith45 4y ago
l30n4da5 4y ago
my first thought is that this seems less secure than using a private ssh key and locking your machine down to only that ssh key. you're essentially using google as your machine login, which seems like weaker security, imo. edit: I'll caveat this and say, I think Tailscale is fantastic! I've been using it personally on my machines for a few months now, and it is awesome.
aspyct 4y ago
I started using tailscale a few days ago, and I absolutely love it. However, one thing is still nagging me: technically, they can add devices to my network without telling me, right? Or is there something I'm missing?
- samb1729 4y ago
  Are you asking whether the owners and operators of the Tailscale control plane can theoretically add devices to your network without your authorisation? If so then yes, definitely. Perhaps a terrible analogy, but to me the question reads like "can the bank just spend my savings?" How might you expect a fresh node to join your existing Tailnet without Tailscale having a means to add a node?
  - dx034 4y ago
    Most code is open source, I guess they could include a feature (not enabled by default) that sends a warning whenever it sees a previously unseen device on the network. Would be noisy and useless for most, but prevent tailscale from adding a new device secretly. But then again, I'm not sure there are many people who'd worry about that.
  - dvzk 4y ago
    Requiring an administrator or other device to pre-authorize or manually approve a new device, by signing the new device key with a client signature key. Why would you expect anything else? That’s like saying Wireguard or SSH servers should just accept any client. The purpose of mesh VPN controllers is to automate redundant key management, not to subvert the original security model.
jasonlotito 4y ago
If I have to use a browser to make use of this (which the demo shows), I never want to use it. It's like the abomination of Okta and Luminate. Absolutely horrible UX. Nope. Will fight very hard to avoid ever having to use this. Antagonistic toward developers at best.
- raggi 4y ago
  Tailscalar here, you can also fetch, validate and edit your ACL's using the API https://github.com/tailscale/tailscale/blob/main/api.md#acl https://github.com/tailscale/tailscale/blob/main/api.md#acl
atonse 4y ago
Another question, can this be used to create SSO-enabled SFTP? Isn't SFTP just ftp over SSH?
- mfincham 4y ago
  SFTP isn't really FTP over SSH, it's just a collision of naming.
- rollcat 4y ago
  > Isn't SFTP just ftp over SSH? FTP is a wholly different protocol, that has aged rather poorly. You might be thinking of FTPS, which is FTP with TLS. SFTP is its own thing and is actually decent/sane. > can this be used to create SSO-enabled SFTP? From another reply somewhere else in the comments, apparently yes.
- dave_universetf 4y ago
  SFTP is a sub-protocol of SSH (technically a "subsystem" in the RFC-speak), which implements features similar to "legacy" FTP. Anyway, our ssh server knows about sftp, so `sftp <host>` should just work.
  - atonse 4y ago
    Wonderful! How would it work with gui based clients for the check prompts? Just launches the browser? Can’t wait to try this.
mfsch 4y ago
Looks interesting, but it seems that this doesn’t work well for servers where every user has a personal account. It appears that this use case would require a separate ACL entry for every user, which a) can get slightly annoying to manage and b) requires a business plan. It would be nice if something like `"users": ["autogroup:emailuser"]` was supported to allow `alice@example.com` to connect as the user `alice`, but that would probably cause issues with e.g. Github organizations, where email addresses can have different TLDs.
quartzic 4y ago
But you still can't have multiple tailnets. The strategy of "have hobbyists try out the software themselves, like it, then implement it at their work" seems incompatible with this fact.
- titanomachy 4y ago
  Do you use the same Google/Github/Microsoft/whatever account for both work and personal stuff?
  - enneff 4y ago
    A lot of people do just use one account for everything. Many smaller companies don’t bother giving people corporate accounts.
    - dx034 4y ago
      That sounds extremely risky. Apart from the fact that it makes it much harder to restrict access for leaving employees, mixing personal and work identities sounds like a recipe for disaster. What happens if a personal account gets banned? How do you enforce security rules? I guess companies where there's not even any identity management, securing your network via tailscale is not your primary concern.
  - structural 4y ago
    It's more than just a work/personal split. Even at work, having "development" and "production" tailnets so that things like testing complex ACLs, inhouse apps that use tailscale via its API, etc. are possible without having everyone on the devops team create an unmanaged/non-company email so they can create their own development tailnet, and then deploy a bunch of company IP using this rogue account. It's a pain point.
- mdeeks 4y ago
  Agreed this is a big limitation. The only way to do it is if you have secondary email address domains. Say mdeeks@company.com and mdeeks@company.team. You can create a separate tailnet for company.team but you also have to roll out additional subnet routers (if you use them) that are authed on that second tailnet. Also you wont be able to easily write rules that interact with things that are not authed onto the second tailnet. They need a first class concept of "canary" or "beta" that applies to ACLs, DNS configs, client versions, and all sorts of other toggles in the UI. It's a hard product problem and I'm not even sure how some of it should work. I just know I need a way to test changes before I roll it out to everyone at the company. Right now there aren't good options for that.
NoraCodes 4y ago
I know this opinion comes up every time Tailscale is mentioned, but requiring SSO _and_ only supporting companies like Google and Microsoft on the free tier means a lot of people can't use it without being exposed to a ton of risk in the form of automated moderation/deletion decisions. I want to be excited about this stuff, but it just won't fit into my risk profile until that changes. Hell, I'd be happy to pay $5/mo or whatever if that meant I could roll my own SSO, or even just use a cheap-per-user, low-volume provider.
- cstejerean 4y ago
  I can see some risk with say Google as the surface area is much larger (maybe they don’t like some play store activity or some ads activity or YouTube activity, etc). But I use my GitHub account and I’m really not worried about automated moderation locking me out of my account.
  - cynix 4y ago
    If only they allowed switching from Google to GitHub...
- skybrian 4y ago
  This is a workaround, but could you minimize that risk by signing up for both Microsoft and Google? It should help for any policy / moderation / "computer says no" decisions that are vendor-specific. (On the other hand, for security risks, it means that a security hole in either one would be a problem.)
  - NoraCodes 4y ago
    That is not how Tailscale SSO works.
jgeralnik 4y ago
So @bradfitz when are you releasing https://tailscale.com/connect/ https://tailscale.com/connect/ for real? :) Context for the uninitiated - as a crazy idea on the podcast Security Cryptography Whatever (hosted by tptacek and others less well known on HN) Avery and Brad of tailscale imagined an ssh client in the browser with QR code authentication to SSO to allow you to connect to your tailscale network (over tailscale SSH) from untrusted computers such as internet cafes. (Or mostly untrusted - safe from keyloggers but maybe not from a dedicated active malware that injects into your browser and tries to inject secret commands into your ssh session). I created a silly PoC here (video instead of link because don't try it for real) https://twitter.com/jgeralnik/status/1487913797155233798 https://twitter.com/jgeralnik/status/1487913797155233798 back when tailscale ssh was a secret binary in the tailscale github repo
- bradfitz 4y ago
  Actually pretty soon, probably. It's working. It needs some UI love first and some docs so people know how it works and don't immediately freak out. :)
- bradfitz 4y ago
  > when tailscale ssh was a secret binary in the tailscale github repo That makes it sound like we put a binary in our git repo :) It was its own Go package main that people could run. It was never a secret. We just didn't advertise it a ton! :)
  - jgeralnik 4y ago
    I didn't mean binary as in "binary blob" but as in "standalone program separate from the tailscale program". Sorry for any confusion :)
midislack 4y ago
What's Tailscale? Some kind of VPN?
alex_dev 4y ago
I've been having trouble adopting Tailscale. As so many others say, relying on another identity provider is unfortunate - I, too, worry what happens when Google decides to lock me out because some algorithm decided my account is fishy. The biggest blocker has been the issues with the Android client. I'm either hitting https://github.com/tailscale/tailscale/issues/915 https://github.com/tailscale/tailscale/issues/915 or https://github.com/tailscale/tailscale/issues/4611 https://github.com/tailscale/tailscale/issues/4611, but neither issue appears to have a fix coming soon. Whenever I am on my carrier's network, my phone's internet stops until I disable Tailscale - that's just a show stopper from using TailScale. So instead of developing this SSH feature, I would have preferred to seen them work on their bug backlog. In the meantime, I'm experimenting with ZeroTier. While it doesn't have the ease and cool magicDNS+LetsEncrypt feature, I think I'll survive with something more reliable.
- viraptor 4y ago
  Are you after the LE part specifically? If not, I'm quite happy with mdns and the seems to be a unicast version available too: https://www.zerotier.com/2021/05/06/zeronsd-unicast-dns-resolution-for-zerotier-now-in-alpha/ https://www.zerotier.com/2021/05/06/zeronsd-unicast-dns-reso... For public domains, I've got a quick script which mirrors what appears in avahi to route53, so that's one way to deal with certs.
  - alex_dev 4y ago
    I appreciate that Tailscale runs the DNS server so it's one less thing for me to manage. Similarly, the built-in LE is just icing on the cake as it's one less thing to think about. Once https://github.com/hassio-addons/addon-tailscale/pull/89 https://github.com/hassio-addons/addon-tailscale/pull/89 is merged, running Home Assistant on a VPN with a LE certificate, would be such a quick setup for anyone. Indeed, you can do all that yourself as you point out. Just last night I manually created a public domain to point to a ZeroTier address and ran the Lets Encrypt addon in Home Assistant to generate a certificate via the DNS challenge. Didn't take long, but there were many steps involved (creating a Google Cloud service account and configuring everything).
ngcc_hk 4y ago
There is a reason why in a corp you need to install certain kind of ai network sniffer to get this underlying network traffic to surface. Be worked on network security and it is just hard to work in a network which you cannot see I think. The bypass is a success and it is not even free (price wise it seems). Crazy.
- tptacek 4y ago
  Tailscale is the network infrastructure for this feature. This is like being concerned that Cisco can see your ICMP headers in iOS.
googleide 4y ago
Slightly OT, How do you do "re-authenticate before connecting" (https://tailscale.com/kb/1193/tailscale-ssh/ https://tailscale.com/kb/1193/tailscale-ssh/) when using Google identity, we are using Google Oauth2 and latest identity SDK but can't see how to force a user to re-authenticate if logged in, do you just make a random unique claim? Related: https://stackoverflow.com/questions/32433378/google-login-api-force-to-type-password https://stackoverflow.com/questions/32433378/google-login-ap...
dx034 4y ago
I love that feature but I'd be a bit scared to just switch off all other ssh, in case the tailscale service ever crashes. I know, machines can just be set up again, but if the problem reproduces there's no way to debug it. So what's the recommendation here to stay safe but still have a failover? Keep ssh enabled for only one user (with sudo rights) and a key that's stored at some secure location?
pilif 4y ago
I don't understand Tailscale's pricing structure: On one end, the features they are adding make the most sense if every machine that should be accessible is running tailscale. Both the fine-grained ACL support and now this SSH thing don't make sense with shared subnets. However, their pricing ties number of servers to number of users. In our case, we have potentially 3 admins who would administer about 50 machines, plus some ephemeral ones. Assuming that each admin has two Macs and an iPhone just on their client side, I don't see how this can ever work within the limits in their pricing plans (except if I'd use subnet sharing, but that would cause me to miss out on many additional features that only make sense if Tailscale is running on each machine). Is there no way to buy additional devices? And my other gripe is with their API: The fine grained ACL support is perfect to, say, issue temporary access to some machines for some users and the API does allow that. But why the hell are API keys only valid for 60 days? I don't want to build a solution on top of a piece of infrastructure that requires me to manually log into a site every 60 days.
- ithkuil 4y ago
  1. api keys != auth keys 2. you can disable key expiry for devices where it makes sense, see https://tailscale.com/kb/1028/key-expiry/#disabling-key-expiry https://tailscale.com/kb/1028/key-expiry/#disabling-key-expi...
  - pilif 4y ago
    1. I know. I'm talking about API keys though - they do expire after 90 days. See https://tailscale.com/kb/1101/api/ https://tailscale.com/kb/1101/api/ 2. For API keys, expiration cannot be disabled.
    - ithkuil 4y ago
      Out of curiosity, what are you using the API for?
SuperSandro2000 4y ago
You still need to manage some amount (possibly smaller than right now) of ssh keys because if not then you are totally reliant on tailscale being up all the time or you can't access your infrastructure of they have an outage.
nmiculinic 4y ago
This is pretty awesome! At my workplace we're using tailscale, and it's been mostly good experience. There were some hickups (like tokens expiring without sending any notification email), though all in all much better then alternatives.
ratg13 4y ago
Is anyone using tailscale on an organizational level? I'm curious to hear about some of the use cases, and whether some companies and organizations are attempting to adopt this instead of traditional VPN.
- RL_Quine 4y ago
  I would have loved this at a company a couple of years ago which was massively all in on Google Auth for literally everything. If you're fine with that being your concrete level of authentication for everything; internal tools, external tools, etc, then tailscale sort of just slots right in, and SSH makes it even more so. I would be very hesitant to build around this personally, hijacking Google accounts is already something pretty high value to a companies adversary, and using Tailscale and SSH like this turns it from a compromise of accounts indirectly through password resets into access to unlimited production machines, internal services, etc. It feels almost layer violating to have a soft social login through Google, that gets persisted in every Chrome browser and logged into on every employees phones also directly control SSH, but maybe that's just me.
  - zikduruqe 4y ago
    Then host your own Auth server - https://github.com/juanfont/headscale https://github.com/juanfont/headscale
- soSadm4n 4y ago
  One of my clients, an industrial/commercial property realtor (to contextualize the environment; we’re not talking military secrets here), uses it. Day to day I interact with it like any other VPN client except I auth via the Google workspace account they gave me. It’s Tailscale, or hosted OpenVPN and cross your fingers they’re not snooping, or DIY Wireguard or OpenVPN and all the usual ups and downs of DIY. Software based infra is out of the unknown unknowns era these days and years of rising usability expectations means Oracle level nightmares to deal with do not gain enough momentum to survive anymore. Tailscale is plenty easy to deal with. The only consideration is do you believe your traffic is really secure? Otherwise “it just works” like anything else these days. That said, my project for them is deprecating the infra accessed via Tailscale (24/7 EC2 running web dashboards). The already Dockerized dashboards will run locally now and use an API to retrieve the data. Real people directly in your infra is probably best avoided.
RL_Quine 4y ago
I'm not entirely convinced I want a feature that adds even more exposure to the sort of goofy login flow Tailscale has.
- tptacek 4y ago
  It's helpful for people to know, from context later in the thread, that one of the core concerns behind this comment is the idea of using SSO at all, and thus giving "the keys to the kingdom" to Google. Of course, it's also worth knowing that SSO is basically a universal best-practice for security teams, and while it's not de jure required by SOC2, it's almost de facto required. For once, I think the best-practices and compliance people have this one right: you are extraordinarily unlikely to get burnt for trusting Google in this instance, and the security track record of ad-hoc authentication is worse than abysmal (ad-hoc authentication is probably implicated in a plurality of all major incidents).
  - mdeeks 4y ago
    Probably the main concern around Google is for personal accounts where it sounds like people often find no way of recovering their accounts when Google thinks you did something bad. For business users that isn't as much the case since there are good support options and it isn't likely you'll get locked out. Great for businesses, potentially risky for individuals.
    - tptacek 4y ago
      This seems entirely fair to me. I mean, if you're scared of Google, you can use something like Okta. If it's me choosing, it's not a hard decision: I want centralized authentication, so I can quickly do blanket interventions like enabling/disabling apps, requiring phishing-proof MFA without having to do any implementation work, and linking everything to onboarding/offboarding/access review policies. And if I'm going to trust anyone's security team with this, it's going to be Google's.
  - mike_d 4y ago
    > it's also worth knowing that SSO is basically a universal best-practice for security teams It is also best practice that some things explicitly don't sit behind SSO for defense in depth. If for example you leave CrowdStrike and JAMF behind Okta, and Okta gets popped an attacker can disable your endpoint protection and push ransomware to every machine.
deleted 4y ago
[deleted]
edf13 4y ago
Never login as root… even over secured links!
- e12e 4y ago
  Indeed, "root" is not a person - only persons should have authorization (to log in, to elevate via su/sudo). Ed: although in this case the binding between a system user and a person happens at the tailscale level.
  - megous 4y ago
    "bob" UNIX account is not a person either. If you ssh in, no matter to what account, your key ID is logged and that's what matters. Anything can happen afterwards, unless you have a really tight grip on your system, since local privilege escallations are not that hard or uncommon.
- lfmunoz4 4y ago
- alerighi 4y ago
  Why? To me it doesn’t make sense not to do so. What kind of security gives you logging it as non root user and then using sudo (maybe even without a password) to become root? Root makes everything more easier, for example to copy a file on a server with scp if you don’t have root login you first has to copy it to /tmp and the copy it in the right directory by logging in as normal user and elevating with su/sudo. Also you have to remember which username was chosen when the server was installed, was it user, admin, or pi, or some default? I get not using root for everyday usage on a desktop, but for a server having a non root user is not that useful. Of course you have to know what you are doing, but sudo doesn’t protect you from stupid mistakes anyway (especially if configured with NOPASSWD as I always see doing because having to continuously type the password is annoying and password tends to be forgotten)
  - kevin_nisbet 4y ago
    It probably depends a bit on your security posture and how you manage the server. The recommendation for avoiding root tends to be central to the idea of least privilege. By default, grant/allow the least privileges necessary, helps eliminate alot of security surface area, while also being a layer of protection against mistakes. It's easy to be careless with a superuser account, and have a bad day. And this is naturally in conflict with being productive. As you mentioned, it's easier to be productive if you can just do all of the things all of the time. And operating in environments where a mistake may not be that devastating, or compromise or vulnerability this might be a reasonable tradeoff. But I've worked in environments where this is too risky. For example: 1. Engineer accidentally pastes the wrong buffer into a terminal. They had accidentally copied some other piece of text. 2. The text happens to contain \nhostname set\n. 3. The terminal as it's spitting out errors, does see a valid command, to change the hostname. 4. That particular system was an HA system, and the process monitor in use, grepping the running processes for the command + args, of which hostname was an argument. And decided the process was no longer running. 5. The cluster seeing the failure, decides to boot another process. But at this time in history, that process was could only handle a single instance. Both instances now decided to conflict with each other. 6. Some part I don't remember about the failover site. 7. A million cell phones can no longer get an IP address. So it's a question of tradeoffs, but is a generally recommended practice to not login directly to root, and operate with less privileges when not required. And then escalate if granted / required.
Sytten 4y ago
This seems like the perfect complement to replace the SSM Agent / bastion instance currently used to access AWS VPC (it is super clunky to use). This should allow an easier time to do reverse tunnelling to databases without having to manage SSH keys.
- ryanisnan 4y ago
  I'm curious, what's really clunky about SSM? Other than ensuring the pre-requisites are met, and knowing the instance-id, SSM works pretty flawlessly. You can easily write a wrapper that looks up the instance-id from the hostname, if you prefer to use it that way.
  - fladrif 4y ago
    Haha I'm not sure if you were being serious, but the workflow you just outlined is the clunky part of SSM. The pre-requisites are getting all the IAM roles and permissions setup (no mean feat), installing the agent, configuring it with keys generated by another user, and getting the connection information back from the aws console. This promises to be a lot easier to setup and authenticate, install tailscale, login.
    - LilBytes 4y ago
      Installing the agent client side is no more or less tedious than installing the Tailscale client, IMO anyway. I made two scripts, one in .Net with a GUI for non-devs to grep a server hostname or tag:name in AWS that resolves to an instance ID for SSH or RDP. And another python script doing the same but without the GUI for the dev team. Works a treat. But you've already explained why it's a little tedious and now I've documented and understood why. Tailscale MagicDNS does all this nonsense for you. Yeah ok thanks for rubber ducking me I see your point now. :)
    - viraptor 4y ago
      It really seems to depend on the point of view. If you're already using AWS seriously, your hosts will have the default agent anyway, IAM is already managed in a reasonable way (iamy, tf or similar), etc. so the setup is not that hard. I'm not sure what you mean by information from the AWS console - it's usable in the terminal.
pphysch 4y ago
> (SSH certificates are better, but have you tried running your own enterprise CA?) For a small business, what is so hard about keeping a file (CA private key) secure and changing it when required?
- loriverkutya 4y ago
  Changing it when it is required.
- lloeki 4y ago
  > For a small business, what is so hard about keeping a file (CA private key) secure and changing it when required? For a small business? Well, keeping a file secure and changing it when required ^^' I mean, it's not out of this world hard to generate your private CA but there are a thousand footguns, the experience isn't exactly friendly, and it's Yet Another Thing To Do And Keep Track Off, i.e even if there's someone who has the technical chops, they may not have the bandwidth, and also, lottery factor. Let alone keeping it properly secure. There's a whole framework/procedure to create to set that up properly. (been there done that, I was exactly in the situation above)
  - pphysch 4y ago
    Can you be more specific on some of those main footguns? I need to rotate the CA for some rare reason. Boom, I do it. All the old SSH certs are invalidated, but users can get a new one through the usual automated flow.
danousna 4y ago
What would be the advantages of this compared to say Teleport ? Teleport is working fine for us, but I wonder if the network based approach (+ wireguard) of Tailscale would be better in terms of network redundancy ?
- tptacek 4y ago
  The big thing you get with Teleport that you don't yet get with Tailscale --- apart from entirely owning the source of truth for SSH authentication on your own infra, which is a very minor issue for almost everyone but is a major issue for some people --- is that Teleport gives you transcript-level audit logs of your SSH sessions. Teleport also has that web-based SSH console (it's one of the better web-based consoles) and the ability to do joint SSH connections. But the audit log is the big one. Obviously, the flip side of this is that Tailscale's SSH is built in; if you're already using Tailscale, and you're not already using Teleport, you should enable Tailscale's SSH right away; it is hugely better than managing your own SSH service ad-hoc.
  - dave_universetf 4y ago
    Session recording's actually already in the network engine for SSH, we just haven't plumbed the whole "push recordings somewhere and surface them" yet. Soon :)
    - tptacek 4y ago
      It's an extremely valuable feature, in that it can knock out a bunch of different SOC2 DRL line items with a single screenshot.
      - alipitch 4y ago
        For those who are not familiar with the term DRL in "SOC2 DRL line item", it is document request list (DRL).
  - outworlder 4y ago
    > is that Teleport gives you transcript-level audit logs of your SSH sessions That is extremely valuable. Just in case 'transcript-level audit' didn't sink in, it's a session recording – not only you can see the all keystrokes typed but you can see all the outputs, the whole state. Someone doing a TOP command for an hour? You can watch the same thing later. Think asciinema (https://asciinema.org/ https://asciinema.org/).
riobard 4y ago
I'll have to ask this since it's bothering me for quite a while… If I connect to a server via WireGuard, would it make more sense to run simpler & unencrypted `rsh` instead of `ssh`? It's kinda pointless to double encrypt.
- Helitico 4y ago
  Stay secure by default. CPU overhead for encryption is basically non existend.
- dsr_ 4y ago
  Yes and no. You shouldn't have rsh on your system at all -- there's a case for telnet to test connections (though netcat is better), but there's no case for rsh. ssh used to allow setting cipher=none, but that's not available anymore. Think of it this way: you're paying the small overhead of double encryption, but you're gaining not fatfingering your way to a password compromise.
  - throwaway894345 4y ago
    I'm not following. How does double encryption help to avoid a password compromise if everything is authed with tailscale in the first place?
    - xgbi 4y ago
      Somebody listening on local connections can sniff your password.
      - throwaway894345 4y ago
        How does that work? First of all, I would expect that rsh-over-tailscale doesn't use password authentication, and secondly can a local user sniff traffic before it hits tailscaled?
    - 8organicbits 4y ago
      I suspect they mean that if you have rsh installed for tailscale you need to be very careful with how you run it. If you accidentally let rsh listen on 0.0.0.0 and don't firewall it then you've given attackers a way to guess passwords. Forgetting to firewall services or accidentally exposing services to the internet is pretty common. ssh is more hardened than rsh, especially with key based auth, so the risk is lower.
dimitar 4y ago
How Tailscale different from other VPN solutions?
- matthewmacleod 4y ago
  They have a relatively detailed series of comparisons with other solutions in their documentation: https://tailscale.com/kb/comparisons/vpns/ https://tailscale.com/kb/comparisons/vpns/
bradfitz 4y ago
I'm one of the authors of this. Happy to answer any questions. One of the fun technical details is that, when enabled on a machine (tailscale up --ssh), the userspace tailscaled process takes over all TCP port 22 packets after the WireGuard decryption and doesn't even feed them into the kernel over TUN. We use gVisor's netstack to handle the TCP connections in-process. So it doesn't matter whether you have other processes (or iptables rules, etc) that would prevent the Tailscale SSH server from binding to port 22. This lets people gradually use Tailscale SSH over time without messing with their system one. The Tailscale SSH server currently only runs on Linux but there's support in git main for macOS too but it's not super well tested yet and not included in the sandboxed GUI builds currently.
- jdadj 4y ago
  This is neat. I've used Cloudflare's Zero-Trust SSH, but I've been frustrated that it interacts poorly with sftp and scp because of the client-side changes that they make to ~/.ssh/config Does tailscale have the same issue?
  - bradfitz 4y ago
    We don't modify or require changes to your SSH client. You can use any SSH client you want.
  - xena 4y ago
    Tailscale employee here. Tailscale SSH works at the target side by listening on the SSH port on that machine. Client changes aren't needed for this to work, all that is required is to use your SSH client as normal. This should allow you to use sftp and scp without issues.
  - jgeralnik 4y ago
    For what it's worth I encountered the same issue and came up with a solution: https://github.com/cloudflare/cloudflared/issues/574 https://github.com/cloudflare/cloudflared/issues/574 Cloudflare have ignored the github issue (which includes a solution) but at least 3 other people seem to have found my solution helpful.

759 points by ignoramous 4y ago

Forgive my ignorance, but what is the benefit for an individual to run this? I currently just use 1.1.1.1 by Cloudflare on my two main devices....not realloy sure I understand what the advantage of this is?

newfonewhodis 4y ago

What happens if I use Tailscale SSH and Google (or whatever IDP) decides to ban my account? Is there a break-glass or something that would let me either change IDPs or re-enable openssh-based access without losing my servers?

leohonexus 4y ago
Totally, I’m a happy customer, but completely relying on Tailscale for out-of-band SSH doesn’t sit right with me, especially for personal machines. One ban and poof, you can’t access your own servers. And to be honest, I understand the appeal of not having to muck around with the system, but SSH isn’t that cumbersome once you’ve set up your /etc/hosts, and encrypt your id_rsa files. Couldn’t get any easier than `ssh <machine>`.
bluehatbrit 4y ago
This bothers me as well as a personal user. I'd love to go all-in on tailscale but having companies like Google running the identity is really off-putting. For now I think I'll be leaving the tailscale SSH functionality and keeping my own setup. I also have a static IP at home which is allowed access to my remote dedicated box. If they enabled some other identity provider that I could either self-host, or use their own with an email + password then I could close off that extra hole but for now it feels too risky for me. On a corporate side I don't mind so much, SSO is so standard and I'd feel perfectly comfortable using Okta or Google SSO because there seem to be far fewer stories of an entire GSuite being banned with no recourse. Tailscale is great and has solved a bunch of problems for me, but the idea of having a seperate IDP that have so many horror stories around them freaks me out enough to slightly lower my security to account for it. Thankfully there are fewer stories like that about GitHub so I'm using them for now.

mountainriver 4y ago

Tailscale is my absolute dream networking solution, I would go as far as to say it will ultimately change how we develop applications in the future

birdyrooster 4y ago
If the auth flow was as good as Touch ID and no window switching, yeah it would be acceptable but this flow would give me a headache with all the flashing.
- dave_universetf 4y ago
  TouchID and related are on the list. Hooking into the existing auth flow was the easiest to get this out the door (and more desirable for some companies who want the audit event in their SSO stack, arguably), vs. figuring out when to nudge people to enroll physical factors and so forth. But I definitely also want "tap your security key please" as an option :)

sandstrom 4y ago

Good! Boundary (https://www.boundaryproject.io/ https://www.boundaryproject.io/) by Hashicorp needs some healthy competition. Teleport is also a tool in this space, for those looking for alternatives.

sandstrom 4y ago
And for anyone looking at Tailscale, I should also mention ZeroTier (https://www.zerotier.com/ https://www.zerotier.com/). In my opinion they have better tech, but they are pretty bad at packaging it, and bad at making it work for actual use-cases. Tailscale seems to be much more clever around building out stuff (like this one, SSH) that actually goes all the way for a particular use-case. ZeroTier feels more like a building block, where you need to bring more stuff yourself. Either way, both are awesome pieces of technology, and really useful!
- philosopher1234 4y ago
  Why is the tech better?
- glitchcrab 4y ago
  I had no end of problems with zerotier, connections would randomly drop between machines even when they were on the same LAN. No such problems with tailscale

infogulch 4y ago

I'm very interested in Tailscale for both personal and business use-cases, but I'm rather put off by the stark centralization of offered identity providers: Microsoft, Github (Microsoft), Google, okta (?). What are the chances that Tailscale would offer authentication using decentralized/self-hosted identity providers like Ory ( https://www.ory.sh/ https://www.ory.sh/ )?

mdeeks 4y ago
They offer custom SSO providers using SAML or OIDC https://tailscale.com/kb/1119/sso-saml-oidc/ https://tailscale.com/kb/1119/sso-saml-oidc/ Unfortunately they are locked behind Enterprise pricing because of the extra help and debugging needed to get them working. Maybe at some point this will be offered standard though.
- grinich 4y ago
  Small plug for my startup (hope that's ok!) - if anyone else is looking for an easy way to set up SSO, you should check out the WorkOS Admin Portal feature[0]. It's essentially an onboarding wizard that works for any identity provider. This makes the SAML/OIDC configuration self-serve, which in turn allows you to easily provide SSO to anyone who wants it. The UI can also be branded with your logo/colors and run on your own custom domain. [0] https://workos.com/admin-portal https://workos.com/admin-portal
doublepg23 4y ago
There is also the self-hostable Headscale implementation.
- sandstrom 4y ago
  In case anyone is looking for the URL: https://github.com/juanfont/headscale https://github.com/juanfont/headscale 5k stars on Github, and lots of activity. Seems very interesting!
  - jsmith45 4y ago

l30n4da5 4y ago

my first thought is that this seems less secure than using a private ssh key and locking your machine down to only that ssh key. you're essentially using google as your machine login, which seems like weaker security, imo. edit: I'll caveat this and say, I think Tailscale is fantastic! I've been using it personally on my machines for a few months now, and it is awesome.

aspyct 4y ago

I started using tailscale a few days ago, and I absolutely love it. However, one thing is still nagging me: technically, they can add devices to my network without telling me, right? Or is there something I'm missing?

samb1729 4y ago
Are you asking whether the owners and operators of the Tailscale control plane can theoretically add devices to your network without your authorisation? If so then yes, definitely. Perhaps a terrible analogy, but to me the question reads like "can the bank just spend my savings?" How might you expect a fresh node to join your existing Tailnet without Tailscale having a means to add a node?
- dx034 4y ago
  Most code is open source, I guess they could include a feature (not enabled by default) that sends a warning whenever it sees a previously unseen device on the network. Would be noisy and useless for most, but prevent tailscale from adding a new device secretly. But then again, I'm not sure there are many people who'd worry about that.
- dvzk 4y ago
  Requiring an administrator or other device to pre-authorize or manually approve a new device, by signing the new device key with a client signature key. Why would you expect anything else? That’s like saying Wireguard or SSH servers should just accept any client. The purpose of mesh VPN controllers is to automate redundant key management, not to subvert the original security model.

jasonlotito 4y ago

If I have to use a browser to make use of this (which the demo shows), I never want to use it. It's like the abomination of Okta and Luminate. Absolutely horrible UX. Nope. Will fight very hard to avoid ever having to use this. Antagonistic toward developers at best.

raggi 4y ago
Tailscalar here, you can also fetch, validate and edit your ACL's using the API https://github.com/tailscale/tailscale/blob/main/api.md#acl https://github.com/tailscale/tailscale/blob/main/api.md#acl

atonse 4y ago

Another question, can this be used to create SSO-enabled SFTP? Isn't SFTP just ftp over SSH?

mfincham 4y ago
SFTP isn't really FTP over SSH, it's just a collision of naming.
rollcat 4y ago
> Isn't SFTP just ftp over SSH? FTP is a wholly different protocol, that has aged rather poorly. You might be thinking of FTPS, which is FTP with TLS. SFTP is its own thing and is actually decent/sane. > can this be used to create SSO-enabled SFTP? From another reply somewhere else in the comments, apparently yes.
dave_universetf 4y ago
SFTP is a sub-protocol of SSH (technically a "subsystem" in the RFC-speak), which implements features similar to "legacy" FTP. Anyway, our ssh server knows about sftp, so `sftp <host>` should just work.
- atonse 4y ago
  Wonderful! How would it work with gui based clients for the check prompts? Just launches the browser? Can’t wait to try this.

mfsch 4y ago

Looks interesting, but it seems that this doesn’t work well for servers where every user has a personal account. It appears that this use case would require a separate ACL entry for every user, which a) can get slightly annoying to manage and b) requires a business plan. It would be nice if something like `"users": ["autogroup:emailuser"]` was supported to allow `alice@example.com` to connect as the user `alice`, but that would probably cause issues with e.g. Github organizations, where email addresses can have different TLDs.

quartzic 4y ago

But you still can't have multiple tailnets. The strategy of "have hobbyists try out the software themselves, like it, then implement it at their work" seems incompatible with this fact.

titanomachy 4y ago
Do you use the same Google/Github/Microsoft/whatever account for both work and personal stuff?
- enneff 4y ago
  A lot of people do just use one account for everything. Many smaller companies don’t bother giving people corporate accounts.
  - dx034 4y ago
    That sounds extremely risky. Apart from the fact that it makes it much harder to restrict access for leaving employees, mixing personal and work identities sounds like a recipe for disaster. What happens if a personal account gets banned? How do you enforce security rules? I guess companies where there's not even any identity management, securing your network via tailscale is not your primary concern.
- structural 4y ago
  It's more than just a work/personal split. Even at work, having "development" and "production" tailnets so that things like testing complex ACLs, inhouse apps that use tailscale via its API, etc. are possible without having everyone on the devops team create an unmanaged/non-company email so they can create their own development tailnet, and then deploy a bunch of company IP using this rogue account. It's a pain point.
mdeeks 4y ago
Agreed this is a big limitation. The only way to do it is if you have secondary email address domains. Say mdeeks@company.com and mdeeks@company.team. You can create a separate tailnet for company.team but you also have to roll out additional subnet routers (if you use them) that are authed on that second tailnet. Also you wont be able to easily write rules that interact with things that are not authed onto the second tailnet. They need a first class concept of "canary" or "beta" that applies to ACLs, DNS configs, client versions, and all sorts of other toggles in the UI. It's a hard product problem and I'm not even sure how some of it should work. I just know I need a way to test changes before I roll it out to everyone at the company. Right now there aren't good options for that.

NoraCodes 4y ago

I know this opinion comes up every time Tailscale is mentioned, but requiring SSO _and_ only supporting companies like Google and Microsoft on the free tier means a lot of people can't use it without being exposed to a ton of risk in the form of automated moderation/deletion decisions. I want to be excited about this stuff, but it just won't fit into my risk profile until that changes. Hell, I'd be happy to pay $5/mo or whatever if that meant I could roll my own SSO, or even just use a cheap-per-user, low-volume provider.

cstejerean 4y ago
I can see some risk with say Google as the surface area is much larger (maybe they don’t like some play store activity or some ads activity or YouTube activity, etc). But I use my GitHub account and I’m really not worried about automated moderation locking me out of my account.
- cynix 4y ago
  If only they allowed switching from Google to GitHub...
skybrian 4y ago
This is a workaround, but could you minimize that risk by signing up for both Microsoft and Google? It should help for any policy / moderation / "computer says no" decisions that are vendor-specific. (On the other hand, for security risks, it means that a security hole in either one would be a problem.)
- NoraCodes 4y ago
  That is not how Tailscale SSO works.

jgeralnik 4y ago

So @bradfitz when are you releasing https://tailscale.com/connect/ https://tailscale.com/connect/ for real? :) Context for the uninitiated - as a crazy idea on the podcast Security Cryptography Whatever (hosted by tptacek and others less well known on HN) Avery and Brad of tailscale imagined an ssh client in the browser with QR code authentication to SSO to allow you to connect to your tailscale network (over tailscale SSH) from untrusted computers such as internet cafes. (Or mostly untrusted - safe from keyloggers but maybe not from a dedicated active malware that injects into your browser and tries to inject secret commands into your ssh session). I created a silly PoC here (video instead of link because don't try it for real) https://twitter.com/jgeralnik/status/1487913797155233798 https://twitter.com/jgeralnik/status/1487913797155233798 back when tailscale ssh was a secret binary in the tailscale github repo

bradfitz 4y ago
Actually pretty soon, probably. It's working. It needs some UI love first and some docs so people know how it works and don't immediately freak out. :)
bradfitz 4y ago
> when tailscale ssh was a secret binary in the tailscale github repo That makes it sound like we put a binary in our git repo :) It was its own Go package main that people could run. It was never a secret. We just didn't advertise it a ton! :)
- jgeralnik 4y ago
  I didn't mean binary as in "binary blob" but as in "standalone program separate from the tailscale program". Sorry for any confusion :)

midislack 4y ago

What's Tailscale? Some kind of VPN?

alex_dev 4y ago

I've been having trouble adopting Tailscale. As so many others say, relying on another identity provider is unfortunate - I, too, worry what happens when Google decides to lock me out because some algorithm decided my account is fishy. The biggest blocker has been the issues with the Android client. I'm either hitting https://github.com/tailscale/tailscale/issues/915 https://github.com/tailscale/tailscale/issues/915 or https://github.com/tailscale/tailscale/issues/4611 https://github.com/tailscale/tailscale/issues/4611, but neither issue appears to have a fix coming soon. Whenever I am on my carrier's network, my phone's internet stops until I disable Tailscale - that's just a show stopper from using TailScale. So instead of developing this SSH feature, I would have preferred to seen them work on their bug backlog. In the meantime, I'm experimenting with ZeroTier. While it doesn't have the ease and cool magicDNS+LetsEncrypt feature, I think I'll survive with something more reliable.

viraptor 4y ago
Are you after the LE part specifically? If not, I'm quite happy with mdns and the seems to be a unicast version available too: https://www.zerotier.com/2021/05/06/zeronsd-unicast-dns-resolution-for-zerotier-now-in-alpha/ https://www.zerotier.com/2021/05/06/zeronsd-unicast-dns-reso... For public domains, I've got a quick script which mirrors what appears in avahi to route53, so that's one way to deal with certs.
- alex_dev 4y ago
  I appreciate that Tailscale runs the DNS server so it's one less thing for me to manage. Similarly, the built-in LE is just icing on the cake as it's one less thing to think about. Once https://github.com/hassio-addons/addon-tailscale/pull/89 https://github.com/hassio-addons/addon-tailscale/pull/89 is merged, running Home Assistant on a VPN with a LE certificate, would be such a quick setup for anyone. Indeed, you can do all that yourself as you point out. Just last night I manually created a public domain to point to a ZeroTier address and ran the Lets Encrypt addon in Home Assistant to generate a certificate via the DNS challenge. Didn't take long, but there were many steps involved (creating a Google Cloud service account and configuring everything).

ngcc_hk 4y ago

There is a reason why in a corp you need to install certain kind of ai network sniffer to get this underlying network traffic to surface. Be worked on network security and it is just hard to work in a network which you cannot see I think. The bypass is a success and it is not even free (price wise it seems). Crazy.

tptacek 4y ago
Tailscale is the network infrastructure for this feature. This is like being concerned that Cisco can see your ICMP headers in iOS.

googleide 4y ago

Slightly OT, How do you do "re-authenticate before connecting" (https://tailscale.com/kb/1193/tailscale-ssh/ https://tailscale.com/kb/1193/tailscale-ssh/) when using Google identity, we are using Google Oauth2 and latest identity SDK but can't see how to force a user to re-authenticate if logged in, do you just make a random unique claim? Related: https://stackoverflow.com/questions/32433378/google-login-api-force-to-type-password https://stackoverflow.com/questions/32433378/google-login-ap...

dx034 4y ago

I love that feature but I'd be a bit scared to just switch off all other ssh, in case the tailscale service ever crashes. I know, machines can just be set up again, but if the problem reproduces there's no way to debug it. So what's the recommendation here to stay safe but still have a failover? Keep ssh enabled for only one user (with sudo rights) and a key that's stored at some secure location?

pilif 4y ago

I don't understand Tailscale's pricing structure: On one end, the features they are adding make the most sense if every machine that should be accessible is running tailscale. Both the fine-grained ACL support and now this SSH thing don't make sense with shared subnets. However, their pricing ties number of servers to number of users. In our case, we have potentially 3 admins who would administer about 50 machines, plus some ephemeral ones. Assuming that each admin has two Macs and an iPhone just on their client side, I don't see how this can ever work within the limits in their pricing plans (except if I'd use subnet sharing, but that would cause me to miss out on many additional features that only make sense if Tailscale is running on each machine). Is there no way to buy additional devices? And my other gripe is with their API: The fine grained ACL support is perfect to, say, issue temporary access to some machines for some users and the API does allow that. But why the hell are API keys only valid for 60 days? I don't want to build a solution on top of a piece of infrastructure that requires me to manually log into a site every 60 days.

ithkuil 4y ago
1. api keys != auth keys 2. you can disable key expiry for devices where it makes sense, see https://tailscale.com/kb/1028/key-expiry/#disabling-key-expiry https://tailscale.com/kb/1028/key-expiry/#disabling-key-expi...
- pilif 4y ago
  1. I know. I'm talking about API keys though - they do expire after 90 days. See https://tailscale.com/kb/1101/api/ https://tailscale.com/kb/1101/api/ 2. For API keys, expiration cannot be disabled.
  - ithkuil 4y ago
    Out of curiosity, what are you using the API for?

SuperSandro2000 4y ago

You still need to manage some amount (possibly smaller than right now) of ssh keys because if not then you are totally reliant on tailscale being up all the time or you can't access your infrastructure of they have an outage.

nmiculinic 4y ago

This is pretty awesome! At my workplace we're using tailscale, and it's been mostly good experience. There were some hickups (like tokens expiring without sending any notification email), though all in all much better then alternatives.

ratg13 4y ago

Is anyone using tailscale on an organizational level? I'm curious to hear about some of the use cases, and whether some companies and organizations are attempting to adopt this instead of traditional VPN.

RL_Quine 4y ago
I would have loved this at a company a couple of years ago which was massively all in on Google Auth for literally everything. If you're fine with that being your concrete level of authentication for everything; internal tools, external tools, etc, then tailscale sort of just slots right in, and SSH makes it even more so. I would be very hesitant to build around this personally, hijacking Google accounts is already something pretty high value to a companies adversary, and using Tailscale and SSH like this turns it from a compromise of accounts indirectly through password resets into access to unlimited production machines, internal services, etc. It feels almost layer violating to have a soft social login through Google, that gets persisted in every Chrome browser and logged into on every employees phones also directly control SSH, but maybe that's just me.
- zikduruqe 4y ago
  Then host your own Auth server - https://github.com/juanfont/headscale https://github.com/juanfont/headscale
soSadm4n 4y ago
One of my clients, an industrial/commercial property realtor (to contextualize the environment; we’re not talking military secrets here), uses it. Day to day I interact with it like any other VPN client except I auth via the Google workspace account they gave me. It’s Tailscale, or hosted OpenVPN and cross your fingers they’re not snooping, or DIY Wireguard or OpenVPN and all the usual ups and downs of DIY. Software based infra is out of the unknown unknowns era these days and years of rising usability expectations means Oracle level nightmares to deal with do not gain enough momentum to survive anymore. Tailscale is plenty easy to deal with. The only consideration is do you believe your traffic is really secure? Otherwise “it just works” like anything else these days. That said, my project for them is deprecating the infra accessed via Tailscale (24/7 EC2 running web dashboards). The already Dockerized dashboards will run locally now and use an API to retrieve the data. Real people directly in your infra is probably best avoided.

RL_Quine 4y ago

I'm not entirely convinced I want a feature that adds even more exposure to the sort of goofy login flow Tailscale has.

tptacek 4y ago
It's helpful for people to know, from context later in the thread, that one of the core concerns behind this comment is the idea of using SSO at all, and thus giving "the keys to the kingdom" to Google. Of course, it's also worth knowing that SSO is basically a universal best-practice for security teams, and while it's not de jure required by SOC2, it's almost de facto required. For once, I think the best-practices and compliance people have this one right: you are extraordinarily unlikely to get burnt for trusting Google in this instance, and the security track record of ad-hoc authentication is worse than abysmal (ad-hoc authentication is probably implicated in a plurality of all major incidents).
- mdeeks 4y ago
  Probably the main concern around Google is for personal accounts where it sounds like people often find no way of recovering their accounts when Google thinks you did something bad. For business users that isn't as much the case since there are good support options and it isn't likely you'll get locked out. Great for businesses, potentially risky for individuals.
  - tptacek 4y ago
    This seems entirely fair to me. I mean, if you're scared of Google, you can use something like Okta. If it's me choosing, it's not a hard decision: I want centralized authentication, so I can quickly do blanket interventions like enabling/disabling apps, requiring phishing-proof MFA without having to do any implementation work, and linking everything to onboarding/offboarding/access review policies. And if I'm going to trust anyone's security team with this, it's going to be Google's.
- mike_d 4y ago
  > it's also worth knowing that SSO is basically a universal best-practice for security teams It is also best practice that some things explicitly don't sit behind SSO for defense in depth. If for example you leave CrowdStrike and JAMF behind Okta, and Okta gets popped an attacker can disable your endpoint protection and push ransomware to every machine.

deleted 4y ago

[deleted]

edf13 4y ago

Never login as root… even over secured links!

e12e 4y ago
Indeed, "root" is not a person - only persons should have authorization (to log in, to elevate via su/sudo). Ed: although in this case the binding between a system user and a person happens at the tailscale level.
- megous 4y ago
  "bob" UNIX account is not a person either. If you ssh in, no matter to what account, your key ID is logged and that's what matters. Anything can happen afterwards, unless you have a really tight grip on your system, since local privilege escallations are not that hard or uncommon.
lfmunoz4 4y ago
alerighi 4y ago
Why? To me it doesn’t make sense not to do so. What kind of security gives you logging it as non root user and then using sudo (maybe even without a password) to become root? Root makes everything more easier, for example to copy a file on a server with scp if you don’t have root login you first has to copy it to /tmp and the copy it in the right directory by logging in as normal user and elevating with su/sudo. Also you have to remember which username was chosen when the server was installed, was it user, admin, or pi, or some default? I get not using root for everyday usage on a desktop, but for a server having a non root user is not that useful. Of course you have to know what you are doing, but sudo doesn’t protect you from stupid mistakes anyway (especially if configured with NOPASSWD as I always see doing because having to continuously type the password is annoying and password tends to be forgotten)
- kevin_nisbet 4y ago
  It probably depends a bit on your security posture and how you manage the server. The recommendation for avoiding root tends to be central to the idea of least privilege. By default, grant/allow the least privileges necessary, helps eliminate alot of security surface area, while also being a layer of protection against mistakes. It's easy to be careless with a superuser account, and have a bad day. And this is naturally in conflict with being productive. As you mentioned, it's easier to be productive if you can just do all of the things all of the time. And operating in environments where a mistake may not be that devastating, or compromise or vulnerability this might be a reasonable tradeoff. But I've worked in environments where this is too risky. For example: 1. Engineer accidentally pastes the wrong buffer into a terminal. They had accidentally copied some other piece of text. 2. The text happens to contain \nhostname set\n. 3. The terminal as it's spitting out errors, does see a valid command, to change the hostname. 4. That particular system was an HA system, and the process monitor in use, grepping the running processes for the command + args, of which hostname was an argument. And decided the process was no longer running. 5. The cluster seeing the failure, decides to boot another process. But at this time in history, that process was could only handle a single instance. Both instances now decided to conflict with each other. 6. Some part I don't remember about the failover site. 7. A million cell phones can no longer get an IP address. So it's a question of tradeoffs, but is a generally recommended practice to not login directly to root, and operate with less privileges when not required. And then escalate if granted / required.

Sytten 4y ago

This seems like the perfect complement to replace the SSM Agent / bastion instance currently used to access AWS VPC (it is super clunky to use). This should allow an easier time to do reverse tunnelling to databases without having to manage SSH keys.

ryanisnan 4y ago
I'm curious, what's really clunky about SSM? Other than ensuring the pre-requisites are met, and knowing the instance-id, SSM works pretty flawlessly. You can easily write a wrapper that looks up the instance-id from the hostname, if you prefer to use it that way.
- fladrif 4y ago
  Haha I'm not sure if you were being serious, but the workflow you just outlined is the clunky part of SSM. The pre-requisites are getting all the IAM roles and permissions setup (no mean feat), installing the agent, configuring it with keys generated by another user, and getting the connection information back from the aws console. This promises to be a lot easier to setup and authenticate, install tailscale, login.
  - LilBytes 4y ago
    Installing the agent client side is no more or less tedious than installing the Tailscale client, IMO anyway. I made two scripts, one in .Net with a GUI for non-devs to grep a server hostname or tag:name in AWS that resolves to an instance ID for SSH or RDP. And another python script doing the same but without the GUI for the dev team. Works a treat. But you've already explained why it's a little tedious and now I've documented and understood why. Tailscale MagicDNS does all this nonsense for you. Yeah ok thanks for rubber ducking me I see your point now. :)
  - viraptor 4y ago
    It really seems to depend on the point of view. If you're already using AWS seriously, your hosts will have the default agent anyway, IAM is already managed in a reasonable way (iamy, tf or similar), etc. so the setup is not that hard. I'm not sure what you mean by information from the AWS console - it's usable in the terminal.

pphysch 4y ago

> (SSH certificates are better, but have you tried running your own enterprise CA?) For a small business, what is so hard about keeping a file (CA private key) secure and changing it when required?

loriverkutya 4y ago
Changing it when it is required.
lloeki 4y ago
> For a small business, what is so hard about keeping a file (CA private key) secure and changing it when required? For a small business? Well, keeping a file secure and changing it when required ^^' I mean, it's not out of this world hard to generate your private CA but there are a thousand footguns, the experience isn't exactly friendly, and it's Yet Another Thing To Do And Keep Track Off, i.e even if there's someone who has the technical chops, they may not have the bandwidth, and also, lottery factor. Let alone keeping it properly secure. There's a whole framework/procedure to create to set that up properly. (been there done that, I was exactly in the situation above)
- pphysch 4y ago
  Can you be more specific on some of those main footguns? I need to rotate the CA for some rare reason. Boom, I do it. All the old SSH certs are invalidated, but users can get a new one through the usual automated flow.

danousna 4y ago

What would be the advantages of this compared to say Teleport ? Teleport is working fine for us, but I wonder if the network based approach (+ wireguard) of Tailscale would be better in terms of network redundancy ?

tptacek 4y ago
The big thing you get with Teleport that you don't yet get with Tailscale --- apart from entirely owning the source of truth for SSH authentication on your own infra, which is a very minor issue for almost everyone but is a major issue for some people --- is that Teleport gives you transcript-level audit logs of your SSH sessions. Teleport also has that web-based SSH console (it's one of the better web-based consoles) and the ability to do joint SSH connections. But the audit log is the big one. Obviously, the flip side of this is that Tailscale's SSH is built in; if you're already using Tailscale, and you're not already using Teleport, you should enable Tailscale's SSH right away; it is hugely better than managing your own SSH service ad-hoc.
- dave_universetf 4y ago
  Session recording's actually already in the network engine for SSH, we just haven't plumbed the whole "push recordings somewhere and surface them" yet. Soon :)
  - tptacek 4y ago
    It's an extremely valuable feature, in that it can knock out a bunch of different SOC2 DRL line items with a single screenshot.
    - alipitch 4y ago
      For those who are not familiar with the term DRL in "SOC2 DRL line item", it is document request list (DRL).
- outworlder 4y ago
  > is that Teleport gives you transcript-level audit logs of your SSH sessions That is extremely valuable. Just in case 'transcript-level audit' didn't sink in, it's a session recording – not only you can see the all keystrokes typed but you can see all the outputs, the whole state. Someone doing a TOP command for an hour? You can watch the same thing later. Think asciinema (https://asciinema.org/ https://asciinema.org/).

riobard 4y ago

I'll have to ask this since it's bothering me for quite a while… If I connect to a server via WireGuard, would it make more sense to run simpler & unencrypted `rsh` instead of `ssh`? It's kinda pointless to double encrypt.

Helitico 4y ago
Stay secure by default. CPU overhead for encryption is basically non existend.
dsr_ 4y ago
Yes and no. You shouldn't have rsh on your system at all -- there's a case for telnet to test connections (though netcat is better), but there's no case for rsh. ssh used to allow setting cipher=none, but that's not available anymore. Think of it this way: you're paying the small overhead of double encryption, but you're gaining not fatfingering your way to a password compromise.
- throwaway894345 4y ago
  I'm not following. How does double encryption help to avoid a password compromise if everything is authed with tailscale in the first place?
  - xgbi 4y ago
    Somebody listening on local connections can sniff your password.
    - throwaway894345 4y ago
      How does that work? First of all, I would expect that rsh-over-tailscale doesn't use password authentication, and secondly can a local user sniff traffic before it hits tailscaled?
  - 8organicbits 4y ago
    I suspect they mean that if you have rsh installed for tailscale you need to be very careful with how you run it. If you accidentally let rsh listen on 0.0.0.0 and don't firewall it then you've given attackers a way to guess passwords. Forgetting to firewall services or accidentally exposing services to the internet is pretty common. ssh is more hardened than rsh, especially with key based auth, so the risk is lower.

dimitar 4y ago

How Tailscale different from other VPN solutions?

matthewmacleod 4y ago
They have a relatively detailed series of comparisons with other solutions in their documentation: https://tailscale.com/kb/comparisons/vpns/ https://tailscale.com/kb/comparisons/vpns/

bradfitz 4y ago

I'm one of the authors of this. Happy to answer any questions. One of the fun technical details is that, when enabled on a machine (tailscale up --ssh), the userspace tailscaled process takes over all TCP port 22 packets after the WireGuard decryption and doesn't even feed them into the kernel over TUN. We use gVisor's netstack to handle the TCP connections in-process. So it doesn't matter whether you have other processes (or iptables rules, etc) that would prevent the Tailscale SSH server from binding to port 22. This lets people gradually use Tailscale SSH over time without messing with their system one. The Tailscale SSH server currently only runs on Linux but there's support in git main for macOS too but it's not super well tested yet and not included in the sandboxed GUI builds currently.

jdadj 4y ago
This is neat. I've used Cloudflare's Zero-Trust SSH, but I've been frustrated that it interacts poorly with sftp and scp because of the client-side changes that they make to ~/.ssh/config Does tailscale have the same issue?
- bradfitz 4y ago
  We don't modify or require changes to your SSH client. You can use any SSH client you want.
- xena 4y ago
  Tailscale employee here. Tailscale SSH works at the target side by listening on the SSH port on that machine. Client changes aren't needed for this to work, all that is required is to use your SSH client as normal. This should allow you to use sftp and scp without issues.
- jgeralnik 4y ago
  For what it's worth I encountered the same issue and came up with a solution: https://github.com/cloudflare/cloudflared/issues/574 https://github.com/cloudflare/cloudflared/issues/574 Cloudflare have ignored the github issue (which includes a solution) but at least 3 other people seem to have found my solution helpful.