4 ms·

Rclone can also mount cloud storage to local disk, especially nice from kubernetes. Write/read speed isn't the fastest when using as a drive with lots of files

Rclone can also mount cloud storage to local disk, especially nice from kubernetes. Write/read speed isn't the fastest when using as a drive with lots of files in the same folder, but a quick and easy way to utilize cloud storage for projects It can also e2e encrypt locations, so everything you put into the mounted drive is getting written encrypted to Dropbox folder /foo, for example. Nice, because Dropbox and other providers like s3 don't have native e2e support yet All in all, Rclone is great! One of those tools that is good to have on hand, and solves so many usecases

sgbeal 3y ago
It's also a trivial way to set up an ad-hoc ftp server which serves either local or cloud storage. e.g. my Pi4 runs an rclone ftp server which exposes my dropbox to the rest of my intranet, and a separate ftp server which my networked printer can save scans to. The same machine uses rclone to run automated backups of cloud storage for my household and a close friend. rclone is a godsend for my intranet.
- artdigital 3y ago
  Oh wow I didn't know about this! Good tip
- bravura 3y ago
  Do you mind explaining why it's so trivial versus setting up traditional ftp? I'm missing something. Thank you
  - killingtime74 3y ago
    Can traditional FTP talk to Dropbox?
    - OkGoDoIt 3y ago
      Assuming the Dropbox is synchronized somewhere to your file system, the FTP server could serve that directory. Although I guess not everyone synchronizes their Dropbox locally.
      - sgbeal 3y ago
        > Although I guess not everyone synchronizes their Dropbox locally. And i've yet to see a pi-native dropbox client for doing so. PS: i actually do sync dropbox to my main workstation but selectively do so. The lion's share of my dropbox is stuff i don't need locally so don't bother to sync it. The rclone approach gives me easy access to the whole dropbox, when needed, from anywhere in my intranet, and i can use my local file manager instead of the dropbox web interface.
  - sgbeal 3y ago
    > Do you mind explaining why it's so trivial versus setting up traditional ftp? nohup /usr/bin/rclone serve ftp --addr MYIP:2121 $PWD &>/dev/null & No configuration needed (beyond rclone's per-cloud-storage-account config (noting that serving local dirs this way does not require any cloud storage config)) and some variation of that can be added to crontab like: @reboot /usr/bin/sleep 30 && ...the above command... Noting that $PWD can be a cloud drive identifier (part of the rclone config) so it can proxy a remote cloud service this same way. So, for example: rclone serve ftp --addr MYIP:2121 mydropbox: assuming "mydropbox" is the locally-configured name for your rclone dropbox connection, that will serve your whole dropbox.
    - steve_rambo 3y ago
      Just write a systemd unit. These commands are not any easier to support and are far worse from the purely technical point of view. You'll get: - startup only when the network is up - proper logging - automatic restarts on failure - optional protection for your ssh keys and other data if there's a breach (refer to `systemd-analyze security`) Run: $ systemctl --user edit --full --force rclone-ftp.service this opens a text editor; paste these lines: [Unit] After=network-online.target Wants=network-online.target [Install] WantedBy=default.target [Service] ExecStart=/usr/bin/rclone --your-flags /directory and then enable and start the service: $ systemctl --user enable --now rclone-ftp
      - rezonant 3y ago
        Seriously yes. Crontab isn't meant to keep your services up. We have a proper service manager now, out with the hacks.
      - lillecarl 3y ago
        People go out of their way to build their own crappy version of systemd. systemd is far from perfect, and Poettering is radically anti-user. But it's the best we got and it serves us well
        frankjr 3y ago
- rakoo 3y ago
  And then all you need to do is use a trivial curlftpfs to have a synchronized folder available. Dropbox is not needed anymore ?
FwarkALark 3y ago
> Rclone can also mount cloud storage to local disk It's not immediately apparent what this means—does it use FUSE, 9p, a driver, or some other mechanism to convert FS calls into API calls? EDIT: it's FUSE.
safety1st 3y ago
Has anyone used it successfully as a replacement for two-way sync apps (like Insync for Google Drive)?. Insync sucks, and Google Drive sucks, but for something I depend on every day I feel like I really need to have a local copy of the files and immediate sync between local and server, particularly when Internet access is spotty.
- godzillabrennus 3y ago
  Seems like a market opportunity exists for this especially with Apple cutting external drive support for third party sync tools including DropBox.
  - CharlesW 3y ago
    You can still do this, but Dropbox can’t use the File Provider API for that yet, so the experience won’t be quite as integrated as it is with Dropbox for macOS on File Provider. See https://help.dropbox.com/installs/dropbox-for-macos-support https://help.dropbox.com/installs/dropbox-for-macos-support for more.
- sgbeal 3y ago
  > Insync sucks... FWIW, i've been using Insync on Linux since it went online (because Google never released a Linux-native client). Aside from one massive screw-up on their part about 8 or 10 years ago (where they automatically converted all of my 100+ gdocs-format files to MS office and deleted the originals), i've not had any issues with them. (In that one particular case the deleted gdocs were all in the trash bin, so could be recovered. Nothing was lost, it was just a huge pain in the butt.)
  - tomcam 3y ago
    > Aside from one massive screw-up on their part about 8 or 10 years ago (where they automatically converted all of my 100+ gdocs-format files to MS office and deleted the originals) Why. Just why. How does that shit ever happen in a public release?
- 3y ago
xcrunner529 3y ago
I first used it at work to sync a OneDrive folder from a shared drive due to different audiences. Very cool tool. The open source stuff I really love.
akdev1l 3y ago
When you say “e2e” encryption do you mean client-side encryption? Because S3 supports both client and server side encryption. (It doesn’t really need to do anything on the service side to support client-side encryption tbf) For client side encryption they have a whole encrypted S3 client and everything. (https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/what-is-s3-encryption-client.html https://docs.aws.amazon.com/amazon-s3-encryption-client/late...)
- aborsy 3y ago
  This seems to be an SDK or library, not a command line tool.
ddeck 3y ago
It is indeed great for this, but you need to make sure your network is stable. I use it on my desktop and laptop to mount Google drives. The problem on the laptop is that the OS sees the drive as local, and Rclone doesn't timeout on network errors. So if you are not connected to wifi and an application tries to read/write to the drive, it will hang forever. This results in most of the UI locking up under XFCE for example, if you have a Thunar window open.
- stiff 3y ago
  There is in fact a default timeout of 5 minutes and you can change it: https://rclone.org/docs/#timeout-time https://rclone.org/docs/#timeout-time I shorten it to prevent lockups like you are describing.
  - ddeck 3y ago
    Thanks, but unfortunately this doesn't work - for my issues at least. I have this (and conntimeout) set to 15 seconds, but it makes no difference. I tried those based on another user reporting the same issue here: https://forum.rclone.org/t/how-to-get-rclone-mount-to-issue-an-i-o-timeout-after-a-remote-becomes-unavaliable/24220 https://forum.rclone.org/t/how-to-get-rclone-mount-to-issue-... The timeout param is listed as "If a transfer has started but then becomes idle for this long it is considered broken and disconnected". This seems to be only for file transfers in progress. I traced it once, and Rclone gets a "temporary DNS failure" error once the network is down, but just keeps retrying.
    - joshspankit 3y ago
      Sounds like you have enough for a decent bug report
- jmaker 3y ago
  Similar issues with WebDAV mounts on macOS

sgbeal 3y ago

It's also a trivial way to set up an ad-hoc ftp server which serves either local or cloud storage. e.g. my Pi4 runs an rclone ftp server which exposes my dropbox to the rest of my intranet, and a separate ftp server which my networked printer can save scans to. The same machine uses rclone to run automated backups of cloud storage for my household and a close friend. rclone is a godsend for my intranet.

artdigital 3y ago
Oh wow I didn't know about this! Good tip
bravura 3y ago
Do you mind explaining why it's so trivial versus setting up traditional ftp? I'm missing something. Thank you
- killingtime74 3y ago
  Can traditional FTP talk to Dropbox?
  - OkGoDoIt 3y ago
    Assuming the Dropbox is synchronized somewhere to your file system, the FTP server could serve that directory. Although I guess not everyone synchronizes their Dropbox locally.
    - sgbeal 3y ago
      > Although I guess not everyone synchronizes their Dropbox locally. And i've yet to see a pi-native dropbox client for doing so. PS: i actually do sync dropbox to my main workstation but selectively do so. The lion's share of my dropbox is stuff i don't need locally so don't bother to sync it. The rclone approach gives me easy access to the whole dropbox, when needed, from anywhere in my intranet, and i can use my local file manager instead of the dropbox web interface.
- sgbeal 3y ago
  > Do you mind explaining why it's so trivial versus setting up traditional ftp? nohup /usr/bin/rclone serve ftp --addr MYIP:2121 $PWD &>/dev/null & No configuration needed (beyond rclone's per-cloud-storage-account config (noting that serving local dirs this way does not require any cloud storage config)) and some variation of that can be added to crontab like: @reboot /usr/bin/sleep 30 && ...the above command... Noting that $PWD can be a cloud drive identifier (part of the rclone config) so it can proxy a remote cloud service this same way. So, for example: rclone serve ftp --addr MYIP:2121 mydropbox: assuming "mydropbox" is the locally-configured name for your rclone dropbox connection, that will serve your whole dropbox.
  - steve_rambo 3y ago
    Just write a systemd unit. These commands are not any easier to support and are far worse from the purely technical point of view. You'll get: - startup only when the network is up - proper logging - automatic restarts on failure - optional protection for your ssh keys and other data if there's a breach (refer to `systemd-analyze security`) Run: $ systemctl --user edit --full --force rclone-ftp.service this opens a text editor; paste these lines: [Unit] After=network-online.target Wants=network-online.target [Install] WantedBy=default.target [Service] ExecStart=/usr/bin/rclone --your-flags /directory and then enable and start the service: $ systemctl --user enable --now rclone-ftp
    - rezonant 3y ago
      Seriously yes. Crontab isn't meant to keep your services up. We have a proper service manager now, out with the hacks.
    - lillecarl 3y ago
      People go out of their way to build their own crappy version of systemd. systemd is far from perfect, and Poettering is radically anti-user. But it's the best we got and it serves us well
      - frankjr 3y ago
rakoo 3y ago
And then all you need to do is use a trivial curlftpfs to have a synchronized folder available. Dropbox is not needed anymore ?

FwarkALark 3y ago

> Rclone can also mount cloud storage to local disk It's not immediately apparent what this means—does it use FUSE, 9p, a driver, or some other mechanism to convert FS calls into API calls? EDIT: it's FUSE.

safety1st 3y ago

Has anyone used it successfully as a replacement for two-way sync apps (like Insync for Google Drive)?. Insync sucks, and Google Drive sucks, but for something I depend on every day I feel like I really need to have a local copy of the files and immediate sync between local and server, particularly when Internet access is spotty.

godzillabrennus 3y ago
Seems like a market opportunity exists for this especially with Apple cutting external drive support for third party sync tools including DropBox.
- CharlesW 3y ago
  You can still do this, but Dropbox can’t use the File Provider API for that yet, so the experience won’t be quite as integrated as it is with Dropbox for macOS on File Provider. See https://help.dropbox.com/installs/dropbox-for-macos-support https://help.dropbox.com/installs/dropbox-for-macos-support for more.
sgbeal 3y ago
> Insync sucks... FWIW, i've been using Insync on Linux since it went online (because Google never released a Linux-native client). Aside from one massive screw-up on their part about 8 or 10 years ago (where they automatically converted all of my 100+ gdocs-format files to MS office and deleted the originals), i've not had any issues with them. (In that one particular case the deleted gdocs were all in the trash bin, so could be recovered. Nothing was lost, it was just a huge pain in the butt.)
- tomcam 3y ago
  > Aside from one massive screw-up on their part about 8 or 10 years ago (where they automatically converted all of my 100+ gdocs-format files to MS office and deleted the originals) Why. Just why. How does that shit ever happen in a public release?
3y ago

xcrunner529 3y ago

I first used it at work to sync a OneDrive folder from a shared drive due to different audiences. Very cool tool. The open source stuff I really love.

akdev1l 3y ago

When you say “e2e” encryption do you mean client-side encryption? Because S3 supports both client and server side encryption. (It doesn’t really need to do anything on the service side to support client-side encryption tbf) For client side encryption they have a whole encrypted S3 client and everything. (https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/what-is-s3-encryption-client.html https://docs.aws.amazon.com/amazon-s3-encryption-client/late...)

aborsy 3y ago
This seems to be an SDK or library, not a command line tool.

ddeck 3y ago

It is indeed great for this, but you need to make sure your network is stable. I use it on my desktop and laptop to mount Google drives. The problem on the laptop is that the OS sees the drive as local, and Rclone doesn't timeout on network errors. So if you are not connected to wifi and an application tries to read/write to the drive, it will hang forever. This results in most of the UI locking up under XFCE for example, if you have a Thunar window open.

stiff 3y ago
There is in fact a default timeout of 5 minutes and you can change it: https://rclone.org/docs/#timeout-time https://rclone.org/docs/#timeout-time I shorten it to prevent lockups like you are describing.
- ddeck 3y ago
  Thanks, but unfortunately this doesn't work - for my issues at least. I have this (and conntimeout) set to 15 seconds, but it makes no difference. I tried those based on another user reporting the same issue here: https://forum.rclone.org/t/how-to-get-rclone-mount-to-issue-an-i-o-timeout-after-a-remote-becomes-unavaliable/24220 https://forum.rclone.org/t/how-to-get-rclone-mount-to-issue-... The timeout param is listed as "If a transfer has started but then becomes idle for this long it is considered broken and disconnected". This seems to be only for file transfers in progress. I traced it once, and Rclone gets a "temporary DNS failure" error once the network is down, but just keeps retrying.
  - joshspankit 3y ago
    Sounds like you have enough for a decent bug report
jmaker 3y ago
Similar issues with WebDAV mounts on macOS