6 ms·

Discovered that you can replace sshfs with rclone. And the project appear to be way more active : https://github.com/rclone/rclone https://github.com/rclone/rcl

Discovered that you can replace sshfs with rclone. And the project appear to be way more active : https://github.com/rclone/rclone https://github.com/rclone/rclone Edit: cf: https://rclone.org/commands/rclone_mount/ https://rclone.org/commands/rclone_mount/

synergy20 3y ago
how so? rclone is more like scp to me only.
- Timshel 3y ago
  Just edited a link to the doc ^^
- mrshadowgoose 3y ago
  rclone has a "mount" command.
  - ape4 3y ago
    Sorry if this is documented (I did take a quick look). Does rclone mounted do whole file compression like rsync? I don't see how the file API would support this. eg fopen, write, close, etc
    - tredre3 3y ago
      In this scenario, rclone mount uses sftp in the background which compresses the entire stream (you can control that by passing args to the ssh command spawned by rclone) but it doesn't do it per-file. In practice I don't know if there is a difference.
- lillecarl 3y ago
  https://rclone.org/commands/rclone_mount/ https://rclone.org/commands/rclone_mount/
- elkhadiy 3y ago
  Mix up rclone's mount with the Crypt and maybe even the Union backend to cloud storage for a very interesting proposition. I was using a similar setup to get unlimited storage on VPSes when google was offering unlimited gsuite storage. You just need to play around caching values and options on the Crypt side if you plan on streaming data from it.
jhvkjhk 3y ago
rclone looks promising, but last time I tried it, it was very slow compared with sshfs.
- nickcw 3y ago
  Interesting. I haven't tried a speed comparison but I know we sped up the sftp backend recently.
  - Filligree 3y ago
    Does the FUSE mount work on OSX?
    - nickcw 3y ago
      It does. You can use it with macfuse or with fuse-t. See: https://rclone.org/commands/rclone_mount/#mounting-on-macos https://rclone.org/commands/rclone_mount/#mounting-on-macos
    - duped 3y ago
      VFS on MacOS is a minefield. You either need to use a kext (bad option, for many reasons), a network file system (NFS or SMB) and pretend your VFS is a remote server, or create a FileProvider system extension (which cannot actually function as a VFS). If your workflow relies on a VFS that isn't NFS/SMB then don't use MacOS. fuse-t is kind of clever in that it spins up a TCP server that transpiles NFS requests into FUSE requests, but it comes with a bit of a cost and eats a TCP port. The one benefit is that you can actually mount and use a file system entirely in userspace this way, which you can't do on Linux without sandboxing (fusermount3 is SUID to get around this).
      - callalex 3y ago
        >but it comes with a bit of a cost and eats a TCP port I have never heard of someone running out of TCP ports on a personal computer since, well, the invention of TCP on personal computers.
dpatterbee 3y ago
I love rclone, I'm currently using `rclone mount` to mount a Backblaze B2 bucket to use as backing storage for Jellyfin and it works a treat. There are plenty of dials to tune things like cache size and duration to minimize unnecessary downloads from B2, and with my usage patterns it ends up astoundingly cost effective (although at some point I might move to Hetzner storage boxes for even cheaper storage).
- jetbalsa 3y ago
  Wasabi might been a good choice for you on that front
  - FredFS456 3y ago
    Backblaze B2 is $5/TB/month, Hetzner is $4.08/TB/month for 1TB, Wasabi is $6.99/TB/month Wasabi seems like the most expensive here, with the caveat that Hetzner requires ordering discrete steps of storage rather than the 'pay-as-you-go' model of the other two
    - vidarh 3y ago
      Doesn't Hetzner storage boxes also have lower redundancy guarantees? It's been a while since I've looked, so not sure - it's not obvious from their ordering page. They're great either way, though, especially given the "unlimited" egress.
    - ph4te 3y ago
      They are all OK to use, and each provider has pros and cons, depending on the use case. Hetzner is the cheapest; however, your data is stored in Germany or Finland. They have free bandwidth, but you are limited to 10 connections at a time. Backblaze B2 has 4 regions across the globe, storage is $5/mo, there is no minimun retention time, but does have a cost for API Calls(transactions), and in addition charges for egress data(downloads), so your $5/TB is a variable factor, and if you use your data, you may not achieve $5/TB, the cost will grow depending on the use case(there are free levels of transactions and egress) Wasabi is $7/TB and has 13 regions across the globe, with free egress and no api charges. It does have 90-day minimum storage charge, which means you are billed for every object for 90 days regardless of if you delete it before 90 days. In addition, the free egress has limits to prevent system abuse. There is a 30-day deleted storage charge available if you purchase in bulk with their RCS(reserved capacity) storage plan. It's good if you want to store a lot of data that does not need deletion. I have accounts with all 3 of these for different use cases.
assbuttbuttass 3y ago
I'm trying to use rclone, but it seems like it's kind of hard to set up compared to sshfs. Do I really need to go through their 20 question setup script?
- remram 3y ago
  You can use connection strings [1] instead of a named remote that you setup ahead of time with the wizard. rclone mount :sftp,host=example.com:path/to/dir /tmp/mountpoint [1]: https://rclone.org/docs/#connection-strings https://rclone.org/docs/#connection-strings
- soraminazuki 3y ago
  You could just edit rclone.conf directly.
menacingly 3y ago
rclone kind of drives me nuts not using standard ssh_config
- aidenn0 3y ago
  Not just rclone, but it seems that so many things are moving away from using the ssh_config; I have things setup to "just work" for so many hosts with OpenSSH, and having to individually configure a half-dozen programs that build on top of SSH is a pain. Sometimes it's because they use paramiko, other times it's because they want to pass their own configuration options to ssh that mess things up. Sometimes you can manage to get things to use the ssh_config again (e.g. with xpra you can do "--ssh=ssh" but other times I've not yet figured a workaround.
  - blueflow 3y ago
    Can you name these programs? I want to add them to my 'painful' list.
    - menacingly 3y ago
      jetbrains products. I think they've improved their situation, but it's still a little quirky
    - aidenn0 3y ago
      I only use programs for which I've found a workaround due to the pain, so I can't list the ones that I don't have a workaround for. TBF two of the programs (emacs tramp and git-annex) I use regularly actually mostly respect ssh_config by default, but they override the ControlMaster by default (to do their own connection muxing) which requires me to do an extra authentication step.
chriswarbo 3y ago
Unfortunately `rclone mount` doesn't support symlinks yet: it always deferences their contents. In contrast, SSHFS has options to either dereference them, or to use them as-is, or to transform absolute links into relative links (which may be more likely to resolve on the client)
deleted 3y ago
[deleted]

synergy20 3y ago

how so? rclone is more like scp to me only.

Timshel 3y ago
Just edited a link to the doc ^^
mrshadowgoose 3y ago
rclone has a "mount" command.
- ape4 3y ago
  Sorry if this is documented (I did take a quick look). Does rclone mounted do whole file compression like rsync? I don't see how the file API would support this. eg fopen, write, close, etc
  - tredre3 3y ago
    In this scenario, rclone mount uses sftp in the background which compresses the entire stream (you can control that by passing args to the ssh command spawned by rclone) but it doesn't do it per-file. In practice I don't know if there is a difference.
lillecarl 3y ago
https://rclone.org/commands/rclone_mount/ https://rclone.org/commands/rclone_mount/
elkhadiy 3y ago
Mix up rclone's mount with the Crypt and maybe even the Union backend to cloud storage for a very interesting proposition. I was using a similar setup to get unlimited storage on VPSes when google was offering unlimited gsuite storage. You just need to play around caching values and options on the Crypt side if you plan on streaming data from it.

jhvkjhk 3y ago

rclone looks promising, but last time I tried it, it was very slow compared with sshfs.

nickcw 3y ago
Interesting. I haven't tried a speed comparison but I know we sped up the sftp backend recently.
- Filligree 3y ago
  Does the FUSE mount work on OSX?
  - nickcw 3y ago
    It does. You can use it with macfuse or with fuse-t. See: https://rclone.org/commands/rclone_mount/#mounting-on-macos https://rclone.org/commands/rclone_mount/#mounting-on-macos
  - duped 3y ago
    VFS on MacOS is a minefield. You either need to use a kext (bad option, for many reasons), a network file system (NFS or SMB) and pretend your VFS is a remote server, or create a FileProvider system extension (which cannot actually function as a VFS). If your workflow relies on a VFS that isn't NFS/SMB then don't use MacOS. fuse-t is kind of clever in that it spins up a TCP server that transpiles NFS requests into FUSE requests, but it comes with a bit of a cost and eats a TCP port. The one benefit is that you can actually mount and use a file system entirely in userspace this way, which you can't do on Linux without sandboxing (fusermount3 is SUID to get around this).
    - callalex 3y ago
      >but it comes with a bit of a cost and eats a TCP port I have never heard of someone running out of TCP ports on a personal computer since, well, the invention of TCP on personal computers.

dpatterbee 3y ago

I love rclone, I'm currently using `rclone mount` to mount a Backblaze B2 bucket to use as backing storage for Jellyfin and it works a treat. There are plenty of dials to tune things like cache size and duration to minimize unnecessary downloads from B2, and with my usage patterns it ends up astoundingly cost effective (although at some point I might move to Hetzner storage boxes for even cheaper storage).

jetbalsa 3y ago
Wasabi might been a good choice for you on that front
- FredFS456 3y ago
  Backblaze B2 is $5/TB/month, Hetzner is $4.08/TB/month for 1TB, Wasabi is $6.99/TB/month Wasabi seems like the most expensive here, with the caveat that Hetzner requires ordering discrete steps of storage rather than the 'pay-as-you-go' model of the other two
  - vidarh 3y ago
    Doesn't Hetzner storage boxes also have lower redundancy guarantees? It's been a while since I've looked, so not sure - it's not obvious from their ordering page. They're great either way, though, especially given the "unlimited" egress.
  - ph4te 3y ago
    They are all OK to use, and each provider has pros and cons, depending on the use case. Hetzner is the cheapest; however, your data is stored in Germany or Finland. They have free bandwidth, but you are limited to 10 connections at a time. Backblaze B2 has 4 regions across the globe, storage is $5/mo, there is no minimun retention time, but does have a cost for API Calls(transactions), and in addition charges for egress data(downloads), so your $5/TB is a variable factor, and if you use your data, you may not achieve $5/TB, the cost will grow depending on the use case(there are free levels of transactions and egress) Wasabi is $7/TB and has 13 regions across the globe, with free egress and no api charges. It does have 90-day minimum storage charge, which means you are billed for every object for 90 days regardless of if you delete it before 90 days. In addition, the free egress has limits to prevent system abuse. There is a 30-day deleted storage charge available if you purchase in bulk with their RCS(reserved capacity) storage plan. It's good if you want to store a lot of data that does not need deletion. I have accounts with all 3 of these for different use cases.

assbuttbuttass 3y ago

I'm trying to use rclone, but it seems like it's kind of hard to set up compared to sshfs. Do I really need to go through their 20 question setup script?

remram 3y ago
You can use connection strings [1] instead of a named remote that you setup ahead of time with the wizard. rclone mount :sftp,host=example.com:path/to/dir /tmp/mountpoint [1]: https://rclone.org/docs/#connection-strings https://rclone.org/docs/#connection-strings
soraminazuki 3y ago
You could just edit rclone.conf directly.

menacingly 3y ago

rclone kind of drives me nuts not using standard ssh_config

aidenn0 3y ago
Not just rclone, but it seems that so many things are moving away from using the ssh_config; I have things setup to "just work" for so many hosts with OpenSSH, and having to individually configure a half-dozen programs that build on top of SSH is a pain. Sometimes it's because they use paramiko, other times it's because they want to pass their own configuration options to ssh that mess things up. Sometimes you can manage to get things to use the ssh_config again (e.g. with xpra you can do "--ssh=ssh" but other times I've not yet figured a workaround.
- blueflow 3y ago
  Can you name these programs? I want to add them to my 'painful' list.
  - menacingly 3y ago
    jetbrains products. I think they've improved their situation, but it's still a little quirky
  - aidenn0 3y ago
    I only use programs for which I've found a workaround due to the pain, so I can't list the ones that I don't have a workaround for. TBF two of the programs (emacs tramp and git-annex) I use regularly actually mostly respect ssh_config by default, but they override the ControlMaster by default (to do their own connection muxing) which requires me to do an extra authentication step.

chriswarbo 3y ago

Unfortunately `rclone mount` doesn't support symlinks yet: it always deferences their contents. In contrast, SSHFS has options to either dereference them, or to use them as-is, or to transform absolute links into relative links (which may be more likely to resolve on the client)

deleted 3y ago

[deleted]