7 ms·

This looks great, nice work! I have a dream of taking a userspace Wireguard project like this and gluing PAKE over a relay in front of it to exchange Wireguard

This looks great, nice work! I have a dream of taking a userspace Wireguard project like this and gluing PAKE over a relay in front of it to exchange Wireguard keys, followed by holepunching and establishing a direct tunnel. Basically Magic Wormhole for arbitrary tunnels - and hopefully vastly improved performance for the file transfer case as well, rather than crapping out at 20-30 MB/s over long fat networks.

xyzzy_plugh 3y ago
You're basically describing Tailscale.
- mbreese 3y ago
  Maybe, but I read it as wormholes at the application level. Which would be awesome for me. I do a lot of work on remote servers for heavy data processing (large files). Sometimes I need to get these files back to my local computer, or just view figures I’ve generated using remote data. I have a small reverse tunnel daemon setup where I can (in my remote terminal) send arbitrary files or data back to my local computer. $ rtun send file.txt $ rtun view data.pdf It is a daemon that listens locally and when I setup SSH, a reverse tunnel is configured from one Unix socket to another. These are multiuser servers and Unix sockets handle authorization easily. The same setup could be handled similarly with a zmodem like process and aware terminal. However, this setup is a bit cumbersome and I can’t just run `ssh host`. If I could have the same setup with an in-app wireguard tunnel, with no other setup, it would be amazing.
  - dpeckett 3y ago
    Hmm interesting usecase, I know a lot of datascience folks are fans of SSHFS but it sounds like you have kind of got an inversion of control thing going on. Is there a reason you aren't mounting the remote filesystem on your local machine and manipulating files/data via that? I've been looking for interesting applications for noisysockets, and I'm obsessed with network attached storage so this is a fun one for me.
    - mbreese 3y ago
      I have issues trusting SSHFS. It's never been stable enough for me. Maybe it's because I have to go through at least one ssh proxy, in addition to a VPN. Maybe it's that the remote filesystem is slow enough, so trying to do anything remotely is very slow. But really, it think it's that I'm already in a terminal connected to a remote system. I don't want to have to go to a different terminal to try and transfer data that I'm already looking at. And trying to use a Finder window (or explorer) to navigate a complex remote filesystem hierarchy isn't fun. Occasionally I can do my work locally, but usually the data is large enough that I have to do my work on a remote server/cluster. When I generate figures describing my data, I want to see those locally. This particular use-case could be solved by using something like Xpdf, but it's easier to send the figure back to my local machine and view it with Preview.app. If I could view a PDF figure in my terminal -- I'd probably just do that. I also sometimes do need to send datafiles back to my local computer. In these cases, I could use sshfs (but don't like the duelling terminals) or scp (but my file paths can be long and complicated, so typing out paths is a pain). I used to actually just handle this with Dropbox. I'd have a program that would send files to a specific Dropbox folder and that would then sync to my local computer. That worked well, but the delay between syncing was an issue. I've also tried using a remote tunnel back to an SSH server running on my laptop. That was equally cumbersome. The two other limits I would like to avoid: 1) no ports, unix sockets only, and 2) no coordinating servers. I try to use only unix sockets so that I can use normal unix permissions to protect the socket. I'd rather not also include an authentication layer if I can avoid it. I'd also like to not need a third-party server to coordinate the connection between the server and my local computer. My remote servers are behind VPNs and while they can access the internet, it's heavily firewalled. If I already have an SSH connection to the remote server, I'd like to tunnel through this if at all possible. Here's the code/project I wrote to manage this: https://github.com/mbreese/rtun https://github.com/mbreese/rtun
  - apitman 3y ago
    I maintain this list: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling Your use case sounds interesting and there may be a tool out there that will do it, but I can't quite wrap my head around your description of how everything is connected and what runs where with your current setup. I agree with sibling that my main question is what prevents you from using SSHFS or similar?
- ikiris 3y ago
  I for one would appreciate a tailscale that actually does proper networking instead of their weird static routes but not thing.

xyzzy_plugh 3y ago

You're basically describing Tailscale.

mbreese 3y ago
Maybe, but I read it as wormholes at the application level. Which would be awesome for me. I do a lot of work on remote servers for heavy data processing (large files). Sometimes I need to get these files back to my local computer, or just view figures I’ve generated using remote data. I have a small reverse tunnel daemon setup where I can (in my remote terminal) send arbitrary files or data back to my local computer. $ rtun send file.txt $ rtun view data.pdf It is a daemon that listens locally and when I setup SSH, a reverse tunnel is configured from one Unix socket to another. These are multiuser servers and Unix sockets handle authorization easily. The same setup could be handled similarly with a zmodem like process and aware terminal. However, this setup is a bit cumbersome and I can’t just run `ssh host`. If I could have the same setup with an in-app wireguard tunnel, with no other setup, it would be amazing.
- dpeckett 3y ago
  Hmm interesting usecase, I know a lot of datascience folks are fans of SSHFS but it sounds like you have kind of got an inversion of control thing going on. Is there a reason you aren't mounting the remote filesystem on your local machine and manipulating files/data via that? I've been looking for interesting applications for noisysockets, and I'm obsessed with network attached storage so this is a fun one for me.
  - mbreese 3y ago
    I have issues trusting SSHFS. It's never been stable enough for me. Maybe it's because I have to go through at least one ssh proxy, in addition to a VPN. Maybe it's that the remote filesystem is slow enough, so trying to do anything remotely is very slow. But really, it think it's that I'm already in a terminal connected to a remote system. I don't want to have to go to a different terminal to try and transfer data that I'm already looking at. And trying to use a Finder window (or explorer) to navigate a complex remote filesystem hierarchy isn't fun. Occasionally I can do my work locally, but usually the data is large enough that I have to do my work on a remote server/cluster. When I generate figures describing my data, I want to see those locally. This particular use-case could be solved by using something like Xpdf, but it's easier to send the figure back to my local machine and view it with Preview.app. If I could view a PDF figure in my terminal -- I'd probably just do that. I also sometimes do need to send datafiles back to my local computer. In these cases, I could use sshfs (but don't like the duelling terminals) or scp (but my file paths can be long and complicated, so typing out paths is a pain). I used to actually just handle this with Dropbox. I'd have a program that would send files to a specific Dropbox folder and that would then sync to my local computer. That worked well, but the delay between syncing was an issue. I've also tried using a remote tunnel back to an SSH server running on my laptop. That was equally cumbersome. The two other limits I would like to avoid: 1) no ports, unix sockets only, and 2) no coordinating servers. I try to use only unix sockets so that I can use normal unix permissions to protect the socket. I'd rather not also include an authentication layer if I can avoid it. I'd also like to not need a third-party server to coordinate the connection between the server and my local computer. My remote servers are behind VPNs and while they can access the internet, it's heavily firewalled. If I already have an SSH connection to the remote server, I'd like to tunnel through this if at all possible. Here's the code/project I wrote to manage this: https://github.com/mbreese/rtun https://github.com/mbreese/rtun
- apitman 3y ago
  I maintain this list: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling Your use case sounds interesting and there may be a tool out there that will do it, but I can't quite wrap my head around your description of how everything is connected and what runs where with your current setup. I agree with sibling that my main question is what prevents you from using SSHFS or similar?
ikiris 3y ago
I for one would appreciate a tailscale that actually does proper networking instead of their weird static routes but not thing.