4 ms·

Hmm interesting usecase, I know a lot of datascience folks are fans of SSHFS but it sounds like you have kind of got an inversion of control thing going on. Is

Hmm interesting usecase, I know a lot of datascience folks are fans of SSHFS but it sounds like you have kind of got an inversion of control thing going on. Is there a reason you aren't mounting the remote filesystem on your local machine and manipulating files/data via that? I've been looking for interesting applications for noisysockets, and I'm obsessed with network attached storage so this is a fun one for me.

mbreese 3y ago
I have issues trusting SSHFS. It's never been stable enough for me. Maybe it's because I have to go through at least one ssh proxy, in addition to a VPN. Maybe it's that the remote filesystem is slow enough, so trying to do anything remotely is very slow. But really, it think it's that I'm already in a terminal connected to a remote system. I don't want to have to go to a different terminal to try and transfer data that I'm already looking at. And trying to use a Finder window (or explorer) to navigate a complex remote filesystem hierarchy isn't fun. Occasionally I can do my work locally, but usually the data is large enough that I have to do my work on a remote server/cluster. When I generate figures describing my data, I want to see those locally. This particular use-case could be solved by using something like Xpdf, but it's easier to send the figure back to my local machine and view it with Preview.app. If I could view a PDF figure in my terminal -- I'd probably just do that. I also sometimes do need to send datafiles back to my local computer. In these cases, I could use sshfs (but don't like the duelling terminals) or scp (but my file paths can be long and complicated, so typing out paths is a pain). I used to actually just handle this with Dropbox. I'd have a program that would send files to a specific Dropbox folder and that would then sync to my local computer. That worked well, but the delay between syncing was an issue. I've also tried using a remote tunnel back to an SSH server running on my laptop. That was equally cumbersome. The two other limits I would like to avoid: 1) no ports, unix sockets only, and 2) no coordinating servers. I try to use only unix sockets so that I can use normal unix permissions to protect the socket. I'd rather not also include an authentication layer if I can avoid it. I'd also like to not need a third-party server to coordinate the connection between the server and my local computer. My remote servers are behind VPNs and while they can access the internet, it's heavily firewalled. If I already have an SSH connection to the remote server, I'd like to tunnel through this if at all possible. Here's the code/project I wrote to manage this: https://github.com/mbreese/rtun https://github.com/mbreese/rtun

mbreese 3y ago

I have issues trusting SSHFS. It's never been stable enough for me. Maybe it's because I have to go through at least one ssh proxy, in addition to a VPN. Maybe it's that the remote filesystem is slow enough, so trying to do anything remotely is very slow. But really, it think it's that I'm already in a terminal connected to a remote system. I don't want to have to go to a different terminal to try and transfer data that I'm already looking at. And trying to use a Finder window (or explorer) to navigate a complex remote filesystem hierarchy isn't fun. Occasionally I can do my work locally, but usually the data is large enough that I have to do my work on a remote server/cluster. When I generate figures describing my data, I want to see those locally. This particular use-case could be solved by using something like Xpdf, but it's easier to send the figure back to my local machine and view it with Preview.app. If I could view a PDF figure in my terminal -- I'd probably just do that. I also sometimes do need to send datafiles back to my local computer. In these cases, I could use sshfs (but don't like the duelling terminals) or scp (but my file paths can be long and complicated, so typing out paths is a pain). I used to actually just handle this with Dropbox. I'd have a program that would send files to a specific Dropbox folder and that would then sync to my local computer. That worked well, but the delay between syncing was an issue. I've also tried using a remote tunnel back to an SSH server running on my laptop. That was equally cumbersome. The two other limits I would like to avoid: 1) no ports, unix sockets only, and 2) no coordinating servers. I try to use only unix sockets so that I can use normal unix permissions to protect the socket. I'd rather not also include an authentication layer if I can avoid it. I'd also like to not need a third-party server to coordinate the connection between the server and my local computer. My remote servers are behind VPNs and while they can access the internet, it's heavily firewalled. If I already have an SSH connection to the remote server, I'd like to tunnel through this if at all possible. Here's the code/project I wrote to manage this: https://github.com/mbreese/rtun https://github.com/mbreese/rtun