3 ms·

"What's crazy to me is that neither of these services provide a non-authenticated way to use a CLI tool to download/sync a directory tree, even if it's public.

by rsync 7y ago

"What's crazy to me is that neither of these services provide a non-authenticated way to use a CLI tool to download/sync a directory tree, even if it's public. rclone is probably the best tool in this space, and it requires you to configure "remotes" ahead of time." It is actually possible to run 'rclone' without configuring a remote ahead of time. It's relatively recent, but if you look at this discussion thread: https://forum.rclone.org/t/can-rclone-be-run-solely-with-command-line-options-no-config-no-env-vars/6314 https://forum.rclone.org/t/can-rclone-be-run-solely-with-com... ... and the later development that was done on this ... you will see that you can run a single 'rclone' command (albeit, a very long and complicated one) that will specify everything that was in your remote configuration. I hope this is helpful ... "I think there's still room for improvement and competition in the cloud storage space." We're trying[1] ... and specifically, with rclone[2] ... [1] https://www.rsync.net/products/universal.html https://www.rsync.net/products/universal.html [2] https://rclone.org/ https://rclone.org/

anderspitman 7y ago
That's definitely a step in the right direction for rclone, but I still think the protocols are the problem. Or more precisely, the lack of a protocol designed for sharing data between services. HTTP is good for retrieving, but lacks directories and a standard for uploads. WebDAV is good in theory, but too complicated and implementations aren't compatible enough. I've been playing around with my own take[0], but it's still really early. Also, I think OAuth is too complicated. That's tricky though because most of the features have been added for good reasons. I've been keeping an eye on rsync.net for while. I really like a lot of the things you're doing. I think SSH is pretty much the best way of doing remote file management that we have today. The main problems are it's hard to share stuff and it doesn't work in browsers. [0] https://github.com/remfs/remfs-http-spec https://github.com/remfs/remfs-http-spec
- rsync 7y ago
  "I've been keeping an eye on rsync.net for while. I really like a lot of the things you're doing." Thank you - appreciated! " think SSH is pretty much the best way of doing remote file management that we have today." I will alert you to the issue of deep inefficiencies in SSH/SFTP over long WAN links. This problem has been solved by HPN-SSH[1] and is likely already built into your OpenSSH distribution, but I mention it here because it is not very well known ... we run into customers doing long, distant (hops) transfers over SSH that we sometimes need to step through the process of updating their ssh so they can negotiate HPN-SSH with us. It's also worth noting that you can configure ssh do use a "null" cipher which might be applicable with borg uploads, bit I digress ... "The main problems are it's hard to share stuff and it doesn't work in browsers." This is true - and by design. There will never be anonymous / non-credentialed access to rsync.net accounts and no rsync.net storage array will ever run an httpd. Our nmap results will always be one line long: TCP22/sshd.
  - anderspitman 7y ago
    > I will alert you to the issue of deep inefficiencies in SSH/SFTP over long WAN links Good to know, thanks! > This is true - and by design. There will never be anonymous / non-credentialed access to rsync.net accounts and no rsync.net storage array will ever run an httpd. Our nmap results will always be one line long: TCP22/sshd. Makes sense from a security standpoint. But I think there's a business opportunity for having cloud storage that allows you to host your files over HTTP. Imagine if google drive was your CMS. Just drag your static site over and it's live. Edit HTML files in-place. Set permissions based on path.

by rsync 7y ago

anderspitman 7y ago

That's definitely a step in the right direction for rclone, but I still think the protocols are the problem. Or more precisely, the lack of a protocol designed for sharing data between services. HTTP is good for retrieving, but lacks directories and a standard for uploads. WebDAV is good in theory, but too complicated and implementations aren't compatible enough. I've been playing around with my own take[0], but it's still really early. Also, I think OAuth is too complicated. That's tricky though because most of the features have been added for good reasons. I've been keeping an eye on rsync.net for while. I really like a lot of the things you're doing. I think SSH is pretty much the best way of doing remote file management that we have today. The main problems are it's hard to share stuff and it doesn't work in browsers. [0] https://github.com/remfs/remfs-http-spec https://github.com/remfs/remfs-http-spec

rsync 7y ago
"I've been keeping an eye on rsync.net for while. I really like a lot of the things you're doing." Thank you - appreciated! " think SSH is pretty much the best way of doing remote file management that we have today." I will alert you to the issue of deep inefficiencies in SSH/SFTP over long WAN links. This problem has been solved by HPN-SSH[1] and is likely already built into your OpenSSH distribution, but I mention it here because it is not very well known ... we run into customers doing long, distant (hops) transfers over SSH that we sometimes need to step through the process of updating their ssh so they can negotiate HPN-SSH with us. It's also worth noting that you can configure ssh do use a "null" cipher which might be applicable with borg uploads, bit I digress ... "The main problems are it's hard to share stuff and it doesn't work in browsers." This is true - and by design. There will never be anonymous / non-credentialed access to rsync.net accounts and no rsync.net storage array will ever run an httpd. Our nmap results will always be one line long: TCP22/sshd.
- anderspitman 7y ago
  > I will alert you to the issue of deep inefficiencies in SSH/SFTP over long WAN links Good to know, thanks! > This is true - and by design. There will never be anonymous / non-credentialed access to rsync.net accounts and no rsync.net storage array will ever run an httpd. Our nmap results will always be one line long: TCP22/sshd. Makes sense from a security standpoint. But I think there's a business opportunity for having cloud storage that allows you to host your files over HTTP. Imagine if google drive was your CMS. Just drag your static site over and it's live. Edit HTML files in-place. Set permissions based on path.