12 ms·

Making an SSH client the hard way

299 points by darthShadow 4y ago

dekhn 4y ago
Most of the products from tailscale just seem to be "look at the inner platforms we can build that replace the outer platforms". Having an SSH client in your browser join your VPN violates all the principles of modern computing.
- deleted 4y ago
  [deleted]
- bradfitz 4y ago
  > Having an SSH client in your browser join your VPN violates all the principles of modern computing. I think that's a compliment? You're welcome? :)
  - convolvatron 4y ago
    absolutely. the lines weren't bad, but they were off in places and now they are set in stone for no good reason. edit: _especially_ when it comes to security
  - dekhn 4y ago
    It's not a compliment. Or rather, within my understanding of how things should be architected, it's not. I certainly wouldn't claim that my own beliefs about network architecture should trump others, and I work in a different domain from most of the people with your use case. Whether the disruptive work you're doing is good for the world in the long run is still a very open question in my mind. I used to think that everything in the world should move to the browser (in my case, that would be high performance molecular graphics and microscope control) and ChromeOS was the logical extension. Web Assembly to handle the existing C++ codebases, a collection of standard web tech to handle the user interface. Big fan of SSH extension because it meant that machines that only ran a browser could be useful command line programming terminals. But didn't like that SSH extension was written in a dead-end container technology (NaCl) and the CHrome folks sort of messed up multiple ways for the extension to integrate better. But after working with web tech for enough time I came to conclude that putting things in the browser like this is an antipattern, in particular increasing the surface complexity of the software space while not actually replacing existing systems (openssh continues to exist, OS-level VPNs continue to exist, even after you port the VPN and client to web assembly and run it in a browser). In my mental model, it makes more sense to put the browser in a VM and then wire the VM's networking to a VPN handled by the OS, rather than putting what is more or less a significant fraction of a virtual machine manager's capabilities into the browser. If you're going to do that, why not go whole-hog and add a VM to chrome so that it can run linux with a full networking stack and then host an SSH client inside that? So you can run linux in your chrome in your linux. My compliment is: I am impressed at how well you parlayed several technical projects into a thought leadership position, but we have fundamentally different architectural principles and work in different domains. Your work disrupts mine, but mine doesn't disrupt yours. My enterprise actually disallows me from visiting your company's website on my work computer because users installing their own VPNs is considered a security risk (fwiw, I bought into BeyondCorp, which eschews VPNs, a long time ago, and would prefer my enterprise eliminate VPNs, as they don't really protect our users).
    - 0xbadcafebee 4y ago
      SSH in the web browser is actually the best practice today. Here are some examples of why SSH in your browser actually compliments modern computing: - An SSO-authenticated web interface, integrated with a host agent on your instances, means you don't have to manage SSH keys. - If you just need a disposable CLI that inherits permissions from your SSO-authenticated user role, you can do that from a disposable box in a web interface after authenticating via the web interface. Google Cloud Shell is a good example. - Cloud-native development is easier if developers can just start working on a unified environment, without having to set up & maintain a local environment. Utilizing the web browser avoids the need to consider separate tools and separate methods of network connection. - SSH'ing to "private" instances is impossible without going through a bastion or VPN. The bastion then becomes a single point of attack, and is hard to maintain and secure. Similarly the VPN is an additional attack vector, maintenance headache, and requires client-side software, configuration, troubleshooting. Instead of deploying a bunch of bastions or setting up a VPN, if you can use the backend control plane through an SSO-authenticated API gateway, along with a backend proxy to internal networks, you can avoid bastions altogether. This is the best practice for Zero-Trust. Google Cloud IAP Proxy is a good example. The implementation of it, with WebAssembly/WebSockets/WireGuard/DERP, may be lamentable for several reasons. But it probably (I assume?) solves problems that other SSH Web Interfaces didn't. I hate that the web browser has monopolized computing interfaces :) But in this case it seems to solve many problems.
      - dekhn 4y ago
        I'm fine with ssh in a browser. I used Chrome SSH Extension for many years to connect to a VM running tmux. And I use RDP if I truly need a remote desktop. However, it (browser SSH) not a replacement for, it's an augmentation of, the OS-level ssh client. Turning this around. Let's take the idea of using WASM to put a full environment in the user's browser. This is a logical idea, after all- WASM exists to make it possible to write applications in Not-Javascript and deploy them in a browser. IE, don't stop with SSH: you should have a web server, a shell, multiprocessing, scripting languages, everything necessary to host VSCode server and a self-hosted compilation toolchain in a browser. Full linux user space in a browser, enough to compile ChromeOS and boot into a browser running linux What have you achieved? A very expensive (in terms of porting cost, CPU usage, and deployment size) inner platform that does what an OS does already. But it's inside the browser, with a patched version of code (because WASM always trails native apps), with each sub-application maybe linking in its own TCP stack. So it will always trail innovations in desktops, since it's not a full replacement for the existing system. So it makes the world more complicated and exposes more surface areas for security management.
- xaduha 4y ago
  From the article > Web-based SSH clients aren’t new. Nearly every VPS and cloud provider already lets you connect to your VMs from the web — so how is this different? This is clearly isn't for everyone, but if you need or already use something like that, then I think this has a chance to be more secure than some other options.
  - dekhn 4y ago
    I really liked Chrome SSH extension but now I've returned to using the ssh command line client on all three platforms. The issue is wiring up the browser-hosted application with a custom network inside the browser. It's a truly interesting but highly disruptive concept and I'm curious how it will play out. Perhaps in the future every program will statically compile its own TCP stack and talk over RAW sockets, but... that's sort of throwing away everything BSD and Linux and Windows achieved over the last few decades in terms of OS abstractions.
    - seabrookmx 4y ago
      Funnily enough, the Chrome SSH extension uses pNaCl which is wasm-like in a roundabout way (non-ISA dependent bytecode running in the browser). So it's really not so different from what tailscale is doing. > Perhaps in the future every program will statically compile its own TCP stack and talk over RAW sockets A surprising number of common applications already do this. I ran into a recent bug[1] caused by a Windows update where TLS handshakes would randomly fail. But this only presented in a few apps. Browsers and .NET apps were all completely unaffected because they don't use the OS level functionality to handle TLS. [1]: This is a link to the KB that fixed the issue, which was introduced in the 22H2 cumulative update (search for "SEC_E_ILLEGAL_MESSAGE"): https://support.microsoft.com/en-us/topic/october-25-2022-kb5018496-os-build-22621-755-preview-64040bea-1e02-4b6d-bad1-b036200c2cb3 https://support.microsoft.com/en-us/topic/october-25-2022-kb...
      - dekhn 4y ago
        Yes, I know pnacl very well- I used it as a sandbox for an idle cycle harvester that ran in the background of Google servers. It was never a workable technology (even when we did heroics to make it work) although many of the ideas were good. TLS is different from TCP. TLS support might be provided by an OS, but it's certainly something an application can link in since it's really just a byte translator with some additional complex logic. TCP is an OS-level protocol for all the reasons that history chose it (having your network device and network protocol in the same ring).
vngzs 4y ago
This is really cool and fun, but is this a safe way to run SSH clients? If, say, the adblock Chrome extension you're using gets bought by a malware operator and backdoored[0], now it also has SSH and VPN access. [0]: https://www.wired.co.uk/article/fake-chrome-extensions-malware https://www.wired.co.uk/article/fake-chrome-extensions-malwa...
- dgentry 4y ago
  The Tailscale VPN client, the same one which runs on other devices, is compiled to WASM. It handles all of the key exchanges to connect to the tailnet. The SSH session is running as a WASM Tailscale client. The browser, opening connections from within the browser engine, doesn't have the keys for SSH or VPN access.
  - ikiris 4y ago
    citation needed
  - shp0ngle 4y ago
    It doesn’t have the keys, but it can inject any javascript and do whatever the user can do.
    - Spivak 4y ago
      To me it seems they've taken all precautions they can reasonably take -- "what if the user installs a keylogger" isn't fixable by anyone.
      - gunapologist99 4y ago
        "Installing a keylogger" is a vast oversimplification, even if it is outside of their threat model. Installing almost anything in your browser is usually a matter of a couple of clicks.
    - ithkuil 4y ago
      Like intercepting your oauth token next time you login into SSO and then use that to access your tailnet. This was true even before this new feature. The new threat model is entirely psychological.
amluto 4y ago
I can't shake the feeling that Tailscale's SSH authentication mechanism is at the wrong layer of the stack. It appears to work by looking at the (source, dest) IP address pair and mapping that to a Tailscale identity. But this may mean that any user or anyone who can initiate TCP connections from an authenticated user's IP can authenticate to the destination over Tailscale SSH. If Tailscale's client was a userspace construct bound to a specific user SSH program, maybe fine. But Tailscale's client is a regular VPN client. What happens if you connect to the Tailscale VPN, open a malicious but sandboxed app of some sort, and that app connects to the target on TCP port 22. For all that it's a seriously unfinished product, Cloudflare's SSH offering seems better thought out. Perhaps Tailscale should find a way to issue a short-lived certificate and use that in addition? (It looks like regular sshd could almost be convinced to handle this. If the SSH_CONNECTION environment variable were passed to the AuthorizedPrincipalsCommand helper or if the source and destination were available as '%' tokens, then AuthorizedPrincipalsCommand could do the Tailscale tuple lookup and use it as a second factor in addition to a short-lived certificate (or regular SSH key or whatever). I bet openssh would accept a patch for this.)
- mihaip 4y ago
  Tailscale SSH's check mode (https://tailscale.com/kb/1193/tailscale-ssh/#configure-tailscale-ssh-with-check-mode https://tailscale.com/kb/1193/tailscale-ssh/#configure-tails...) is meant to address the issue of "rogue process starts an SSH connection". For truly sensitive applications, you can set the check period to be "1s" to always require it.
  - amluto 4y ago
    Hmm. If the problem is that Tailscale SSH doesn’t strongly associate the person authenticating with the connection being authenticated, asking the person to reauthenticate seems like a pretty weak solution.
    - Jarwain 4y ago
      Unless I'm misunderstanding something, the check solution creates that strong association. Logging in gives you a link you have to go to and auth, authing let's your session connect. Disconnect, and you have to do this again. No check mode reuses the auth of the tailscale client, check mode authenticates the ssh connection itself
easton 4y ago
Could the Tailscale client be packaged as an extension so I can visit sites on my Tailnet without having to install a client? Sometimes I want to visit a "internal" site without having to install the client, if I'm using a temporary box for something. I'm not sure how much more work would have to be done, might have to dig into the open source pieces of this.
- capdeck 4y ago
  Putting my vote in for this feature as well... Also, why not compile VNC into WASM and get full remote desktop experience for graphical apps. It seems that hard work has already been done!
  - bradfitz 4y ago
    > Also, why not compile VNC into WASM and get full remote desktop experience for graphical apps. It seems that hard work has already been done! Yup. :) In fact, that's mentioned in the original public bug: https://github.com/tailscale/tailscale/issues/3157 https://github.com/tailscale/tailscale/issues/3157
heliophobicdude 4y ago
Hi Mihai! Great work! I would love to see where this goes! Forgive my ignorance but is there any sort of native client besides the browser running in the background to help with websocket to tcp? Or a tunnel to a cloud service to help there?
- mihaip 4y ago
  No native client is running. The browser makes a WebSocket connection to our relay server, and we run the WireGuard tunnel over that.
  - ignoramous 4y ago
    You mean WireGuard (in userspace w/ netstack) tunneled in WebSockets to DERPServers, correct? What sorcery is this. (:
Scarbutt 4y ago
But can it run emacs?
- xena 4y ago
  Yes
skybrian 4y ago
In the old days, people said you shouldn't write crypto in JavaScript because it was somehow insecure. Have those concerns gone away with WebAssembly and https everywhere?
- rany_ 4y ago
  I don't think the argument was ever that "JavaScript was insecure." It was that the websites hosting it may be compromised or may change the script at any time without any indication (or the FBI forcing a site to backdoor the JS for some investigation)
  - ignoramous 4y ago
    Yeah, but see also Code Verify (and sub-resource integrity): https://blog.cloudflare.com/cloudflare-verifies-code-whatsapp-web-serves-users/ https://blog.cloudflare.com/cloudflare-verifies-code-whatsap...
- kevin_thibedeau 4y ago
  It will never go away because you can't guarantee constant time algorithms will be implemented as such when transformed by a JIT.
  - lxgr 4y ago
    If machine code can issue the necessary hints to the hardware to skip all time-variant optimizations, why couldn‘t the same work for a WASM runtime?
    - jerf 4y ago
      It could, but you need a lot of work on a lot of layers, possibly routed through an entire standards committee, to get it done. If it ever happens it's going to be a long time.
- e12e 4y ago
  I think the consensus is still that you can't write side-channel/timing proof crypto in (most) Javascript (runtimes) - but that with webcrypto(?) most runtimes will provide the secure crypto primitives you need in order to do (secure) crypto with Javascript?
Spivak 4y ago
> To make this possible, we ported the following to WebAssembly: the Tailscale client, WireGuard®, a complete userspace network stack (from gVisor), and an SSH client. I love that they were clearly inspired by fly.io. Warms my heart that a random blog post with a good idea can spread like this.
- bradfitz 4y ago
  Which blog post are you referring to? But yes, we love Fly and use them (and they use us) and we share a slack channel between our two companies for casual banter.
  - Spivak 4y ago
    This one! https://fly.io/blog/ssh-and-user-mode-ip-wireguard/ https://fly.io/blog/ssh-and-user-mode-ip-wireguard/
    - matthewaveryusa 4y ago
      I mean if you want to go back, this is the original post from the horse’s mouth: https://lists.zx2c4.com/pipermail/wireguard/2021-January/006323.html https://lists.zx2c4.com/pipermail/wireguard/2021-January/006...
chatmasta 4y ago
Cool feature! I was just looking at boringtun last night and wondering if it could compile to WASM, to get a virtualized network interface in the browser. Did you experiment with the new WebTransport API [0] at all? It's only supported in Chromium browsers, but seems promising for this kind of use case. [0] https://chromestatus.com/feature/4854144902889472 https://chromestatus.com/feature/4854144902889472
PaulWaldman 4y ago
> To make this possible, we ported the following to WebAssembly: the Tailscale client, WireGuard®, a complete userspace network stack (from gVisor), and an SSH client. Would it be possible to bundle the same into a portable application allowing you to use Tailscale without installing it? My understanding is that currently if you can't install Tailscale on a client you need to use Subnet Router. https://tailscale.com/kb/1109/devices-without-tailscale/ https://tailscale.com/kb/1109/devices-without-tailscale/
- lxgr 4y ago
  You‘d have to redirect all network usage (i.e. the sockets API or your platform‘s equivalent) through the custom stack, which is possible if you can rebuild the source or by using something like LD_PRELOAD for binaries, but can get very tricky in the general case. There‘s an utility called SSHuttle that does something similar for SSH instead of Tailscale/Wireguard: It redirects all sockets usage to go through an SSH connection, to allow usage of SSH port forwarding without explicit SOCKS support on the app‘s side.
  - Serow225 4y ago
    SSHuttle was created by the CEO/co-founder of Tailscale :)
    - lxgr 4y ago
      Indeed, both the name and the very clever combination of existing components sounded familiar when I first heard about Tailscale :)
    - jillesvangurp 4y ago
      Interesting. Sshuttle is great. I've used that a couple of times to create a simple vpn without having to deal with openvpn. Simple and effective.
    - blibble 4y ago
      that's a bit worrying there's a rather large misunderstanding on its github page: > You can't use openssh's PermitTunnel feature because it's disabled by default on openssh servers; plus it does TCP-over-TCP, which has terrible performance. it doesn't do TCP over TCP, it's a bytestream over TCP (exactly the same as shuttle) something like OpenVPN running in TCP mode would be TCP over TCP
gunapologist99 4y ago
This significantly increases the threat model for your remote servers to include all sorts of remote attacks through the web, including: * garden-variety web attacks (i.e., XSS, CRSF, etc) * attacks that might become viable against the browser (for example, Mobile Safari has a history of vulnerabilities) * various attacks against the backend web server (API attacks) * attacks against the WASM layer * CDN injections * Tailscale's backend (various types of injections, timing attacks, or deeper attacks on Tailscale's infrastructure like the nightmares of HeartBleed, Shellshock, Meltdown, etc) That's probably a very incomplete list. Realistically, this essentially (actually, literally) opens a remote root shell into your entire infrastructure through a web page, with apparently nothing more than matching an IP address pair (https://news.ycombinator.com/item?id=33361837 https://news.ycombinator.com/item?id=33361837) to authenticate. What could go wrong? This design with its loose coupling between authenticated user and IP addresses for high-value targets makes me view Tailscale's security model in a whole new light.
- sedatk 4y ago
  That's unnecessary sensationalism. Most of those vectors are behind an SSO login and are not exposed to Internet at all (from the article: "your browser becomes a Tailscale client, and joins your tailnet in the same way as any other device that you run Tailscale on"). Or, did you mean attacks on SSO? If that's the case, then SSH web wouldn't make any difference. Someone authenticating themselves could use regular SSH or whatever. Similarly, Tailscale backend is already subject to the vectors you mentioned (API, side-channel attacks). This feature doesn't add any new attack vectors. Again, attacks on browser means end of game already. Someone can use that vector to access to your local network in other ways. They don't need Tailscale's SSH web client for that.
  - gunapologist99 4y ago
    > Again, attacks on browser means end of game already. A bad Chrome extension does not allow the bad guys to open a terminal on my machine, load my ssh keys, launch an authenticated SSH connection, and launch an authenticated SSH connection into an enumerated list of remote servers.
cynix 4y ago
Will Tailscale SSH support FreeBSD any time soon?
aliqot 4y ago
I wouldn't be the target market for this, however more power to them. I understand that to an extent we should never roll our own, so to speak, however, I think that we should not put all of our eggs in one basket. In that regard, I think that once the beginning kinks are ironed out it will be a better thing and we should ultimately embrace these types of endeavors. Let me say again, though I admire it, I'd never use this. I like to sleep soundly, as irrational as that may be.
ice3 4y ago
Interesting, I could see this as a nice replacement for Gravitational Teleport. One of the things that Teleport lacks (IMO) is Wireguard
kybernetyk 4y ago
>To make this possible, we ported the following to WebAssembly: the Tailscale client, WireGuard®, a complete userspace network stack (from gVisor), and an SSH client. Wow, and they're proud of it.
- zeroxfe 4y ago
  It's a pretty good approach and pretty hard to pull off. I'd be proud of it too.
dekhn 4y ago
I just learned there's a POSIX kernel in Go that can actually run apps. https://www.usenix.org/conference/osdi18/presentation/cutler https://www.usenix.org/conference/osdi18/presentation/cutler I think this should be compiled with WASM and deployed via web page. Then we can explore the idea of browser-hosted POSIX kernels.

299 points by darthShadow 4y ago

Most of the products from tailscale just seem to be "look at the inner platforms we can build that replace the outer platforms". Having an SSH client in your browser join your VPN violates all the principles of modern computing.

deleted 4y ago
[deleted]
bradfitz 4y ago
> Having an SSH client in your browser join your VPN violates all the principles of modern computing. I think that's a compliment? You're welcome? :)
- convolvatron 4y ago
  absolutely. the lines weren't bad, but they were off in places and now they are set in stone for no good reason. edit: _especially_ when it comes to security
- dekhn 4y ago
  It's not a compliment. Or rather, within my understanding of how things should be architected, it's not. I certainly wouldn't claim that my own beliefs about network architecture should trump others, and I work in a different domain from most of the people with your use case. Whether the disruptive work you're doing is good for the world in the long run is still a very open question in my mind. I used to think that everything in the world should move to the browser (in my case, that would be high performance molecular graphics and microscope control) and ChromeOS was the logical extension. Web Assembly to handle the existing C++ codebases, a collection of standard web tech to handle the user interface. Big fan of SSH extension because it meant that machines that only ran a browser could be useful command line programming terminals. But didn't like that SSH extension was written in a dead-end container technology (NaCl) and the CHrome folks sort of messed up multiple ways for the extension to integrate better. But after working with web tech for enough time I came to conclude that putting things in the browser like this is an antipattern, in particular increasing the surface complexity of the software space while not actually replacing existing systems (openssh continues to exist, OS-level VPNs continue to exist, even after you port the VPN and client to web assembly and run it in a browser). In my mental model, it makes more sense to put the browser in a VM and then wire the VM's networking to a VPN handled by the OS, rather than putting what is more or less a significant fraction of a virtual machine manager's capabilities into the browser. If you're going to do that, why not go whole-hog and add a VM to chrome so that it can run linux with a full networking stack and then host an SSH client inside that? So you can run linux in your chrome in your linux. My compliment is: I am impressed at how well you parlayed several technical projects into a thought leadership position, but we have fundamentally different architectural principles and work in different domains. Your work disrupts mine, but mine doesn't disrupt yours. My enterprise actually disallows me from visiting your company's website on my work computer because users installing their own VPNs is considered a security risk (fwiw, I bought into BeyondCorp, which eschews VPNs, a long time ago, and would prefer my enterprise eliminate VPNs, as they don't really protect our users).
  - 0xbadcafebee 4y ago
    SSH in the web browser is actually the best practice today. Here are some examples of why SSH in your browser actually compliments modern computing: - An SSO-authenticated web interface, integrated with a host agent on your instances, means you don't have to manage SSH keys. - If you just need a disposable CLI that inherits permissions from your SSO-authenticated user role, you can do that from a disposable box in a web interface after authenticating via the web interface. Google Cloud Shell is a good example. - Cloud-native development is easier if developers can just start working on a unified environment, without having to set up & maintain a local environment. Utilizing the web browser avoids the need to consider separate tools and separate methods of network connection. - SSH'ing to "private" instances is impossible without going through a bastion or VPN. The bastion then becomes a single point of attack, and is hard to maintain and secure. Similarly the VPN is an additional attack vector, maintenance headache, and requires client-side software, configuration, troubleshooting. Instead of deploying a bunch of bastions or setting up a VPN, if you can use the backend control plane through an SSO-authenticated API gateway, along with a backend proxy to internal networks, you can avoid bastions altogether. This is the best practice for Zero-Trust. Google Cloud IAP Proxy is a good example. The implementation of it, with WebAssembly/WebSockets/WireGuard/DERP, may be lamentable for several reasons. But it probably (I assume?) solves problems that other SSH Web Interfaces didn't. I hate that the web browser has monopolized computing interfaces :) But in this case it seems to solve many problems.
    - dekhn 4y ago
      I'm fine with ssh in a browser. I used Chrome SSH Extension for many years to connect to a VM running tmux. And I use RDP if I truly need a remote desktop. However, it (browser SSH) not a replacement for, it's an augmentation of, the OS-level ssh client. Turning this around. Let's take the idea of using WASM to put a full environment in the user's browser. This is a logical idea, after all- WASM exists to make it possible to write applications in Not-Javascript and deploy them in a browser. IE, don't stop with SSH: you should have a web server, a shell, multiprocessing, scripting languages, everything necessary to host VSCode server and a self-hosted compilation toolchain in a browser. Full linux user space in a browser, enough to compile ChromeOS and boot into a browser running linux What have you achieved? A very expensive (in terms of porting cost, CPU usage, and deployment size) inner platform that does what an OS does already. But it's inside the browser, with a patched version of code (because WASM always trails native apps), with each sub-application maybe linking in its own TCP stack. So it will always trail innovations in desktops, since it's not a full replacement for the existing system. So it makes the world more complicated and exposes more surface areas for security management.
xaduha 4y ago
From the article > Web-based SSH clients aren’t new. Nearly every VPS and cloud provider already lets you connect to your VMs from the web — so how is this different? This is clearly isn't for everyone, but if you need or already use something like that, then I think this has a chance to be more secure than some other options.
- dekhn 4y ago
  I really liked Chrome SSH extension but now I've returned to using the ssh command line client on all three platforms. The issue is wiring up the browser-hosted application with a custom network inside the browser. It's a truly interesting but highly disruptive concept and I'm curious how it will play out. Perhaps in the future every program will statically compile its own TCP stack and talk over RAW sockets, but... that's sort of throwing away everything BSD and Linux and Windows achieved over the last few decades in terms of OS abstractions.
  - seabrookmx 4y ago
    Funnily enough, the Chrome SSH extension uses pNaCl which is wasm-like in a roundabout way (non-ISA dependent bytecode running in the browser). So it's really not so different from what tailscale is doing. > Perhaps in the future every program will statically compile its own TCP stack and talk over RAW sockets A surprising number of common applications already do this. I ran into a recent bug[1] caused by a Windows update where TLS handshakes would randomly fail. But this only presented in a few apps. Browsers and .NET apps were all completely unaffected because they don't use the OS level functionality to handle TLS. [1]: This is a link to the KB that fixed the issue, which was introduced in the 22H2 cumulative update (search for "SEC_E_ILLEGAL_MESSAGE"): https://support.microsoft.com/en-us/topic/october-25-2022-kb5018496-os-build-22621-755-preview-64040bea-1e02-4b6d-bad1-b036200c2cb3 https://support.microsoft.com/en-us/topic/october-25-2022-kb...
    - dekhn 4y ago
      Yes, I know pnacl very well- I used it as a sandbox for an idle cycle harvester that ran in the background of Google servers. It was never a workable technology (even when we did heroics to make it work) although many of the ideas were good. TLS is different from TCP. TLS support might be provided by an OS, but it's certainly something an application can link in since it's really just a byte translator with some additional complex logic. TCP is an OS-level protocol for all the reasons that history chose it (having your network device and network protocol in the same ring).

vngzs 4y ago

This is really cool and fun, but is this a safe way to run SSH clients? If, say, the adblock Chrome extension you're using gets bought by a malware operator and backdoored[0], now it also has SSH and VPN access. [0]: https://www.wired.co.uk/article/fake-chrome-extensions-malware https://www.wired.co.uk/article/fake-chrome-extensions-malwa...

dgentry 4y ago
The Tailscale VPN client, the same one which runs on other devices, is compiled to WASM. It handles all of the key exchanges to connect to the tailnet. The SSH session is running as a WASM Tailscale client. The browser, opening connections from within the browser engine, doesn't have the keys for SSH or VPN access.
- ikiris 4y ago
  citation needed
- shp0ngle 4y ago
  It doesn’t have the keys, but it can inject any javascript and do whatever the user can do.
  - Spivak 4y ago
    To me it seems they've taken all precautions they can reasonably take -- "what if the user installs a keylogger" isn't fixable by anyone.
    - gunapologist99 4y ago
      "Installing a keylogger" is a vast oversimplification, even if it is outside of their threat model. Installing almost anything in your browser is usually a matter of a couple of clicks.
  - ithkuil 4y ago
    Like intercepting your oauth token next time you login into SSO and then use that to access your tailnet. This was true even before this new feature. The new threat model is entirely psychological.

amluto 4y ago

I can't shake the feeling that Tailscale's SSH authentication mechanism is at the wrong layer of the stack. It appears to work by looking at the (source, dest) IP address pair and mapping that to a Tailscale identity. But this may mean that any user or anyone who can initiate TCP connections from an authenticated user's IP can authenticate to the destination over Tailscale SSH. If Tailscale's client was a userspace construct bound to a specific user SSH program, maybe fine. But Tailscale's client is a regular VPN client. What happens if you connect to the Tailscale VPN, open a malicious but sandboxed app of some sort, and that app connects to the target on TCP port 22. For all that it's a seriously unfinished product, Cloudflare's SSH offering seems better thought out. Perhaps Tailscale should find a way to issue a short-lived certificate and use that in addition? (It looks like regular sshd could almost be convinced to handle this. If the SSH_CONNECTION environment variable were passed to the AuthorizedPrincipalsCommand helper or if the source and destination were available as '%' tokens, then AuthorizedPrincipalsCommand could do the Tailscale tuple lookup and use it as a second factor in addition to a short-lived certificate (or regular SSH key or whatever). I bet openssh would accept a patch for this.)

mihaip 4y ago
Tailscale SSH's check mode (https://tailscale.com/kb/1193/tailscale-ssh/#configure-tailscale-ssh-with-check-mode https://tailscale.com/kb/1193/tailscale-ssh/#configure-tails...) is meant to address the issue of "rogue process starts an SSH connection". For truly sensitive applications, you can set the check period to be "1s" to always require it.
- amluto 4y ago
  Hmm. If the problem is that Tailscale SSH doesn’t strongly associate the person authenticating with the connection being authenticated, asking the person to reauthenticate seems like a pretty weak solution.
  - Jarwain 4y ago
    Unless I'm misunderstanding something, the check solution creates that strong association. Logging in gives you a link you have to go to and auth, authing let's your session connect. Disconnect, and you have to do this again. No check mode reuses the auth of the tailscale client, check mode authenticates the ssh connection itself

easton 4y ago

Could the Tailscale client be packaged as an extension so I can visit sites on my Tailnet without having to install a client? Sometimes I want to visit a "internal" site without having to install the client, if I'm using a temporary box for something. I'm not sure how much more work would have to be done, might have to dig into the open source pieces of this.

capdeck 4y ago
Putting my vote in for this feature as well... Also, why not compile VNC into WASM and get full remote desktop experience for graphical apps. It seems that hard work has already been done!
- bradfitz 4y ago
  > Also, why not compile VNC into WASM and get full remote desktop experience for graphical apps. It seems that hard work has already been done! Yup. :) In fact, that's mentioned in the original public bug: https://github.com/tailscale/tailscale/issues/3157 https://github.com/tailscale/tailscale/issues/3157

heliophobicdude 4y ago

Hi Mihai! Great work! I would love to see where this goes! Forgive my ignorance but is there any sort of native client besides the browser running in the background to help with websocket to tcp? Or a tunnel to a cloud service to help there?

mihaip 4y ago
No native client is running. The browser makes a WebSocket connection to our relay server, and we run the WireGuard tunnel over that.
- ignoramous 4y ago
  You mean WireGuard (in userspace w/ netstack) tunneled in WebSockets to DERPServers, correct? What sorcery is this. (:

Scarbutt 4y ago

But can it run emacs?

xena 4y ago
Yes

skybrian 4y ago

In the old days, people said you shouldn't write crypto in JavaScript because it was somehow insecure. Have those concerns gone away with WebAssembly and https everywhere?

rany_ 4y ago
I don't think the argument was ever that "JavaScript was insecure." It was that the websites hosting it may be compromised or may change the script at any time without any indication (or the FBI forcing a site to backdoor the JS for some investigation)
- ignoramous 4y ago
  Yeah, but see also Code Verify (and sub-resource integrity): https://blog.cloudflare.com/cloudflare-verifies-code-whatsapp-web-serves-users/ https://blog.cloudflare.com/cloudflare-verifies-code-whatsap...
kevin_thibedeau 4y ago
It will never go away because you can't guarantee constant time algorithms will be implemented as such when transformed by a JIT.
- lxgr 4y ago
  If machine code can issue the necessary hints to the hardware to skip all time-variant optimizations, why couldn‘t the same work for a WASM runtime?
  - jerf 4y ago
    It could, but you need a lot of work on a lot of layers, possibly routed through an entire standards committee, to get it done. If it ever happens it's going to be a long time.
e12e 4y ago
I think the consensus is still that you can't write side-channel/timing proof crypto in (most) Javascript (runtimes) - but that with webcrypto(?) most runtimes will provide the secure crypto primitives you need in order to do (secure) crypto with Javascript?

Spivak 4y ago

> To make this possible, we ported the following to WebAssembly: the Tailscale client, WireGuard®, a complete userspace network stack (from gVisor), and an SSH client. I love that they were clearly inspired by fly.io. Warms my heart that a random blog post with a good idea can spread like this.

bradfitz 4y ago
Which blog post are you referring to? But yes, we love Fly and use them (and they use us) and we share a slack channel between our two companies for casual banter.
- Spivak 4y ago
  This one! https://fly.io/blog/ssh-and-user-mode-ip-wireguard/ https://fly.io/blog/ssh-and-user-mode-ip-wireguard/
  - matthewaveryusa 4y ago
    I mean if you want to go back, this is the original post from the horse’s mouth: https://lists.zx2c4.com/pipermail/wireguard/2021-January/006323.html https://lists.zx2c4.com/pipermail/wireguard/2021-January/006...

chatmasta 4y ago

Cool feature! I was just looking at boringtun last night and wondering if it could compile to WASM, to get a virtualized network interface in the browser. Did you experiment with the new WebTransport API [0] at all? It's only supported in Chromium browsers, but seems promising for this kind of use case. [0] https://chromestatus.com/feature/4854144902889472 https://chromestatus.com/feature/4854144902889472

PaulWaldman 4y ago

> To make this possible, we ported the following to WebAssembly: the Tailscale client, WireGuard®, a complete userspace network stack (from gVisor), and an SSH client. Would it be possible to bundle the same into a portable application allowing you to use Tailscale without installing it? My understanding is that currently if you can't install Tailscale on a client you need to use Subnet Router. https://tailscale.com/kb/1109/devices-without-tailscale/ https://tailscale.com/kb/1109/devices-without-tailscale/

lxgr 4y ago
You‘d have to redirect all network usage (i.e. the sockets API or your platform‘s equivalent) through the custom stack, which is possible if you can rebuild the source or by using something like LD_PRELOAD for binaries, but can get very tricky in the general case. There‘s an utility called SSHuttle that does something similar for SSH instead of Tailscale/Wireguard: It redirects all sockets usage to go through an SSH connection, to allow usage of SSH port forwarding without explicit SOCKS support on the app‘s side.
- Serow225 4y ago
  SSHuttle was created by the CEO/co-founder of Tailscale :)
  - lxgr 4y ago
    Indeed, both the name and the very clever combination of existing components sounded familiar when I first heard about Tailscale :)
  - jillesvangurp 4y ago
    Interesting. Sshuttle is great. I've used that a couple of times to create a simple vpn without having to deal with openvpn. Simple and effective.
  - blibble 4y ago
    that's a bit worrying there's a rather large misunderstanding on its github page: > You can't use openssh's PermitTunnel feature because it's disabled by default on openssh servers; plus it does TCP-over-TCP, which has terrible performance. it doesn't do TCP over TCP, it's a bytestream over TCP (exactly the same as shuttle) something like OpenVPN running in TCP mode would be TCP over TCP

gunapologist99 4y ago

This significantly increases the threat model for your remote servers to include all sorts of remote attacks through the web, including: * garden-variety web attacks (i.e., XSS, CRSF, etc) * attacks that might become viable against the browser (for example, Mobile Safari has a history of vulnerabilities) * various attacks against the backend web server (API attacks) * attacks against the WASM layer * CDN injections * Tailscale's backend (various types of injections, timing attacks, or deeper attacks on Tailscale's infrastructure like the nightmares of HeartBleed, Shellshock, Meltdown, etc) That's probably a very incomplete list. Realistically, this essentially (actually, literally) opens a remote root shell into your entire infrastructure through a web page, with apparently nothing more than matching an IP address pair (https://news.ycombinator.com/item?id=33361837 https://news.ycombinator.com/item?id=33361837) to authenticate. What could go wrong? This design with its loose coupling between authenticated user and IP addresses for high-value targets makes me view Tailscale's security model in a whole new light.

sedatk 4y ago
That's unnecessary sensationalism. Most of those vectors are behind an SSO login and are not exposed to Internet at all (from the article: "your browser becomes a Tailscale client, and joins your tailnet in the same way as any other device that you run Tailscale on"). Or, did you mean attacks on SSO? If that's the case, then SSH web wouldn't make any difference. Someone authenticating themselves could use regular SSH or whatever. Similarly, Tailscale backend is already subject to the vectors you mentioned (API, side-channel attacks). This feature doesn't add any new attack vectors. Again, attacks on browser means end of game already. Someone can use that vector to access to your local network in other ways. They don't need Tailscale's SSH web client for that.
- gunapologist99 4y ago
  > Again, attacks on browser means end of game already. A bad Chrome extension does not allow the bad guys to open a terminal on my machine, load my ssh keys, launch an authenticated SSH connection, and launch an authenticated SSH connection into an enumerated list of remote servers.

cynix 4y ago

Will Tailscale SSH support FreeBSD any time soon?

aliqot 4y ago

I wouldn't be the target market for this, however more power to them. I understand that to an extent we should never roll our own, so to speak, however, I think that we should not put all of our eggs in one basket. In that regard, I think that once the beginning kinks are ironed out it will be a better thing and we should ultimately embrace these types of endeavors. Let me say again, though I admire it, I'd never use this. I like to sleep soundly, as irrational as that may be.

ice3 4y ago

Interesting, I could see this as a nice replacement for Gravitational Teleport. One of the things that Teleport lacks (IMO) is Wireguard

kybernetyk 4y ago

>To make this possible, we ported the following to WebAssembly: the Tailscale client, WireGuard®, a complete userspace network stack (from gVisor), and an SSH client. Wow, and they're proud of it.

zeroxfe 4y ago
It's a pretty good approach and pretty hard to pull off. I'd be proud of it too.

dekhn 4y ago

I just learned there's a POSIX kernel in Go that can actually run apps. https://www.usenix.org/conference/osdi18/presentation/cutler https://www.usenix.org/conference/osdi18/presentation/cutler I think this should be compiled with WASM and deployed via web page. Then we can explore the idea of browser-hosted POSIX kernels.