8 ms·

Simple SFTP Hosting

158 points by rsync 7y ago

rsync 7y ago
After all of these years I am surprised at the number of new clients that are written against an SFTP back-end. For instance, foldersync[1], restic[2], rclone[3][4], and for ESX(i) (!!) I have just learned of "VerticalBackup"[5]. It makes me happy that a stock, standard OpenSSH serving up what is basically a UNIX home directory is still so powerful and useful. Contrast this with the elegance and state of the art that is ZFS and how immutable snapshots simultaneously solve both retention schedules and ransomware/mallory[6]. [1] https://www.tacit.dk/foldersync/ https://www.tacit.dk/foldersync/ [2] https://restic.net/ https://restic.net/ [3] https://rclone.org/ https://rclone.org/ [4] https://rsync.net/products/rclone.html https://rsync.net/products/rclone.html [5] https://verticalbackup.com/ https://verticalbackup.com/ [6] https://www.rsync.net/resources/howto/snapshots.html https://www.rsync.net/resources/howto/snapshots.html
- yug 7y ago
  For iOS, there's a new amazing app from the developer of Working Copy called Secure ShellFish: https://apps.apple.com/us/app/secure-shellfish/id1336634154?ls=1 https://apps.apple.com/us/app/secure-shellfish/id1336634154?...
- tiffanyh 7y ago
  @rsync I’ve read you’re a major FreeBSD shop. What’s your currently feeling about FreeBSD? How’s the current state of their OS support methodology? How’s stability of the OS these days? Your thoughts on ZFS now separated from FreeBSD. Etc. You’re one of the few individuals who have the breadth and depth of experience with FreeBSD for commercial use, I’m hoping to hear you’re thoughts.
  - 693471 7y ago
    Most of my career has been working for shops running FreeBSD. FreeBSD is doing just fine. ZFS change is not interesting. Previously upstream was Illinois, now it's Linux. Support for releases changed but for the better. Stability is still great, but it will be a while before I see servers with 10 years uptime on modern kernels again. If I'm lucky it won't happen because they're getting patched...
- ktm5j 7y ago
  Ssh is just so ubiquitous. I actually write software that replicates our zfs snapshots off site using ssh as the transfer mechanism. Makes it easy to transfer snapshots between Linux and Solaris.
- LinuxBender 7y ago
  The best client I have used is lftp. [1] It support chroot sftp, can spawn multiple threads per job and even per file. I can max out any internet link with it. It is super easy to write automation with it. [1] - https://serverfault.com/questions/836103/replace-scp-with-sftp/836136#836136 https://serverfault.com/questions/836103/replace-scp-with-sf...
dillonmckay 7y ago
I never realized how good their pricing is!
- voiper1 7y ago
  It's new, just a few years ago it was 20c! https://web.archive.org/web/20150509033924/https://www.rsync.net/pricing.html https://web.archive.org/web/20150509033924/https://www.rsync...
sdan 7y ago
Too expensive given the app. S3/Google give around 2 cents/GB a month (lower for cold storage) but you also get to access it through the web... what's the benefit with this?
- dillonmckay 7y ago
  https://rsync.net/resources/howto/snapshots.html https://rsync.net/resources/howto/snapshots.html You get 7 free snapshots that do not count against your quota. Additional snapshots are charged based on the byte diffs, not total size. It uses a standard protocol and anything that uses ssh can conceivably communicate with it. Oh, and you are not charged for transfer.
  - sdan 7y ago
    Nice. Interesting option.
  - snthpy 7y ago
    How is this sort of thing not abused? Say I generate logs at say 1 GB/hr and store them in files named `logs.$(date +%Y%m%dT%H)` and I have it set up to take snapshots every 4 hours and delete files older than 5 hours, then I could have essentially unlimited storage and only ever pay for 5 GB/month?
    - pnutjam 7y ago
      you only get 7 free snapshots, and if the others are totally different then your live data, you'll pay for them.
      - snthpy 7y ago
        Ok, I had understood it as 7 free snapshots per day.
  - e12e 7y ago
    > Oh, and you are not charged for transfer. This is the big one. Even for backblaze b2, it's the transfer that means 2-3tb data doesn't make sense on a personal budget (eg digital negatives and a few 4k videos). A restore become (too) expensive.
voiper1 7y ago
Wow, the prices are down to 2cents/GB, cheaper than standard AWS + without the fees. [1] Back in 2015, it was 20cents per GB. [2] Besides ZFS with snapshots, one of their benefits vs "dumb" cloud storage is you can run a server side component, e.g. borg which requires both a client and server. They even have slightly reduced pricing (which isn't that big a deal anymore, it was 3 cents vs 8-20 back in 2016 [3]) if you only need borg and don't need the configuration support. [4] EDIT: And, tons of other things I forgot about: git, s3cmd, and others! [5] [1] https://www.rsync.net/pricing.html https://www.rsync.net/pricing.html [2] https://web.archive.org/web/20150509033924/https://www.rsync.net/pricing.html https://web.archive.org/web/20150509033924/https://www.rsync... [3] https://web.archive.org/web/20160926192841/http://rsync.net/products/attic.html https://web.archive.org/web/20160926192841/http://rsync.net/... [4] http://rsync.net/products/borg.html http://rsync.net/products/borg.html [5] https://www.rsync.net/resources/howto/remote_commands.html https://www.rsync.net/resources/howto/remote_commands.html
- rsync 7y ago
  To be fair, we have let s3cmd wither on the vine a bit - I am not sure if we've updated it in the past 18 months. The reason being, anything s3cmd can do, rclone can do better. Right ?
  - ebg13 7y ago
    > The reason being, anything s3cmd can do, rclone can do better. Right ? But is it better than s5cmd?
- sigio 7y ago
  Hetzner is down to 0.4ct/gb (EUR 39.90 for 10TB)
  - Nux 7y ago
    Interesting, I can see 10TB for € 47.88, even less if you don't pay VAT (20%)
    - 7y ago
moviuro 7y ago
That snapshot[0] surely doesn't reflect the reality. Last Modified date is "today" for all snapshots when you advertize an immutable storage. You should also add the current date in the picture (just in case). [0] https://www.rsync.net/images/duck2.png https://www.rsync.net/images/duck2.png on https://archive.is/3kK28 https://archive.is/3kK28
voltagex_ 7y ago
I used to use this for <10GB with the old pricing model. Stopped when the family needed something they could use (ended up on Office Home + OneDrive). I could definitely find a use for 1-5GB of storage, but the minimum is 500GB ($10/USD month or $108/USD annually). Sorry John, looks like you had to chase me a bit due to an expired credit card. Hope I didn't cost too much for support. The most useful thing to me would be being able to use git clone and rclone to pull files closer to me, but I bet it's not an intended use case to use rsync.net as a proxy.
tpmx 7y ago
Does rsync.net have any way of doing two factor authentication, at least for administration purposes? Couldn't find anything on the site. Sure, I get that it's probably primarily built for unattended automated backups, but still.
- rsync 7y ago
  Yes. Our web based admin portal / management interface has 2FA options inside. There are also options to lock your entire account (or just your .ssh/authorized_keys file) to be read-only. The snapshots are always immutable, but you can toggle your entire account to be so, as well. Finally, inside the web based management is an "idle alert" where you can alert yourself (via email, SMS, pushover or webhook - or any combination thereof) to your account being unchanged for a (user configurable) amount of time.
  - tpmx 7y ago
    Nice! These points seem like things you might want to put on the web site. (Did I miss it? I think I read through all of the public tech details/support pages though.) Btw: I found https://secured.fyi/Backup/rsync.net https://secured.fyi/Backup/rsync.net while googling this topic, it seems at least partially incorrect.
dceddia 7y ago
I thought this was refreshing to see :) > We are NOT reselling another cloud - we built and maintain our own infrastructure and have complete control over our platform. > We've been doing this since 2001.
- penagwin 7y ago
  Weekly warrant canaries too! https://www.rsync.net/resources/notices/canary.txt https://www.rsync.net/resources/notices/canary.txt
kup0 7y ago
Would consider using this if there wasn't a 500GB minimum.
sigy 7y ago
The price levels have a gap between 1TB and 10TB, or am I missing something?
- cirrrrrrus 7y ago
  9999 GB roughly equals to 9,999 TB
ivanhoe 7y ago
Is there option for having monthly/yearly snapshots?
- rsync 7y ago
  Yes. You can specify any arbitrary schedule of days/weeks/months/quarters/years. They are efficiently stored so if your data doesn't change much, they will take up almost no space - even though they present your entire filesystem from that date.
bachmeier 7y ago
15 GB of storage would be nice. Go to the "Simple Pricing" page. Click the 0-999 GB for 2 cents/GB link. That's great pricing. Select 15 GB. Get message telling me the minimum is 500 GB. That's not the kind of company I'm going to trust with my data. What other kind of BS are they going to try to pull if they're willing to do something like this before I'm even a customer? Maybe they don't want small customers like me. That's okay. Just be honest about it on your website.
- lixtra 7y ago
  From a UX perspective the first button shouldn’t read “0-9999 GB”, but “< 1 TB”. Otherwise I don’t see a problem with a minimum of 500GB to keep things efficient. On the other hand I believe there is a micro market waiting, paying 1-3 usd/month for 15GB.
  - throwaway8941 7y ago
    You can already have something like this for free with Yandex Disk or Google Drive. rclone will let you mount the so called "cloud storage" locally and will encrypt everything you upload (be aware that encryption has to be configured separately as a remote on top of existing remote).
  - cirrrrrrus 7y ago
    Well, 9999GB is closer to 10TB than to 1TB.
    - knorker 7y ago
      They meant <10TB, and were commenting on the fact that the range started with 0 when actually it starts at 500.
- jsty 7y ago
  Billing 30 cents is unlikely to even cover their credit card handling fees, let alone the cost of actually providing the service.
- deleted 7y ago
fmakunbound 7y ago
Would mounting an SFTP file system using FUSE and then mounting encfs or veracrypt on top of that be reasonably reliable? I'm always concerned about my data at rest on servers others have access to.
- rsync 7y ago
  Yes. sshfs/FUSE is a very common use-case among rsync.net customers. However, borg (and to a lesser degree, restic) is the typical solution for encryption at rest.
  - stevekemp 7y ago
    I use your borg-only service, and love it. I recall offering to donate you the .io domain at some point too!
tiernano 7y ago
Just signed up for 100gb Borg hosting for $18 for a year! That ain't bad. Some lints (no zfs snapshots) but Borg should have snapshots covered. Waiting for activation and will start playing around...
- tiernano 7y ago
  More details here. https://www.rsync.net/products/borg.html https://www.rsync.net/products/borg.html
  - witten 7y ago
    Also check out borgmatic for configuring Borg. Works well with rsync.net: https://torsion.org/borgmatic/ https://torsion.org/borgmatic/
bndw 7y ago
What is Physical Media Shipment?
- rsync 7y ago
  If you need your data right now and your network connection isn't fast enough to do it in a timely manner, we will dump your data to a bare SATA drive and fedex it to you. Alternatively, if you have many TB to upload and your network connection makes that unworkable, we will accept your bare SATA drive for the onloading process.
  - bndw 7y ago
    Awesome, thanks for the response.
knorker 7y ago
Wow, much cheaper than it used to be. I left them as a customer some years back purely based on price, but now I'd consider moving back. Other than being undercut by AWS (now also GCP) they were great.
zafiro17 7y ago
I did some searching around and finally settled on rsync as a backup system about 5 years ago, and I've been happy since. I don't understand the fixation on finding the lowest price solution - pay more, get more. With rsync you get their in-house hardware and engineers on the support desk. I use it to back up my FreeNAS ZFS shares, which is dead-simple process, and a bunch of web servers. Getting historical snapshots is genius. I also love being able to use any SFTP client I like instead of some dedicated client or a web interface. My data is not available through a web page - that's exactly the way I like it. Bravo Rsync, keep up the good work.
CloudBuddy 7y ago
Here is another option if you don't have much to store - https://cloudbuddy.cloud https://cloudbuddy.cloud

158 points by rsync 7y ago

After all of these years I am surprised at the number of new clients that are written against an SFTP back-end. For instance, foldersync[1], restic[2], rclone[3][4], and for ESX(i) (!!) I have just learned of "VerticalBackup"[5]. It makes me happy that a stock, standard OpenSSH serving up what is basically a UNIX home directory is still so powerful and useful. Contrast this with the elegance and state of the art that is ZFS and how immutable snapshots simultaneously solve both retention schedules and ransomware/mallory[6]. [1] https://www.tacit.dk/foldersync/ https://www.tacit.dk/foldersync/ [2] https://restic.net/ https://restic.net/ [3] https://rclone.org/ https://rclone.org/ [4] https://rsync.net/products/rclone.html https://rsync.net/products/rclone.html [5] https://verticalbackup.com/ https://verticalbackup.com/ [6] https://www.rsync.net/resources/howto/snapshots.html https://www.rsync.net/resources/howto/snapshots.html

yug 7y ago
For iOS, there's a new amazing app from the developer of Working Copy called Secure ShellFish: https://apps.apple.com/us/app/secure-shellfish/id1336634154?ls=1 https://apps.apple.com/us/app/secure-shellfish/id1336634154?...
tiffanyh 7y ago
@rsync I’ve read you’re a major FreeBSD shop. What’s your currently feeling about FreeBSD? How’s the current state of their OS support methodology? How’s stability of the OS these days? Your thoughts on ZFS now separated from FreeBSD. Etc. You’re one of the few individuals who have the breadth and depth of experience with FreeBSD for commercial use, I’m hoping to hear you’re thoughts.
- 693471 7y ago
  Most of my career has been working for shops running FreeBSD. FreeBSD is doing just fine. ZFS change is not interesting. Previously upstream was Illinois, now it's Linux. Support for releases changed but for the better. Stability is still great, but it will be a while before I see servers with 10 years uptime on modern kernels again. If I'm lucky it won't happen because they're getting patched...
ktm5j 7y ago
Ssh is just so ubiquitous. I actually write software that replicates our zfs snapshots off site using ssh as the transfer mechanism. Makes it easy to transfer snapshots between Linux and Solaris.
LinuxBender 7y ago
The best client I have used is lftp. [1] It support chroot sftp, can spawn multiple threads per job and even per file. I can max out any internet link with it. It is super easy to write automation with it. [1] - https://serverfault.com/questions/836103/replace-scp-with-sftp/836136#836136 https://serverfault.com/questions/836103/replace-scp-with-sf...

dillonmckay 7y ago

I never realized how good their pricing is!

voiper1 7y ago
It's new, just a few years ago it was 20c! https://web.archive.org/web/20150509033924/https://www.rsync.net/pricing.html https://web.archive.org/web/20150509033924/https://www.rsync...

sdan 7y ago

Too expensive given the app. S3/Google give around 2 cents/GB a month (lower for cold storage) but you also get to access it through the web... what's the benefit with this?

dillonmckay 7y ago
https://rsync.net/resources/howto/snapshots.html https://rsync.net/resources/howto/snapshots.html You get 7 free snapshots that do not count against your quota. Additional snapshots are charged based on the byte diffs, not total size. It uses a standard protocol and anything that uses ssh can conceivably communicate with it. Oh, and you are not charged for transfer.
- sdan 7y ago
  Nice. Interesting option.
- snthpy 7y ago
  How is this sort of thing not abused? Say I generate logs at say 1 GB/hr and store them in files named `logs.$(date +%Y%m%dT%H)` and I have it set up to take snapshots every 4 hours and delete files older than 5 hours, then I could have essentially unlimited storage and only ever pay for 5 GB/month?
  - pnutjam 7y ago
    you only get 7 free snapshots, and if the others are totally different then your live data, you'll pay for them.
    - snthpy 7y ago
      Ok, I had understood it as 7 free snapshots per day.
- e12e 7y ago
  > Oh, and you are not charged for transfer. This is the big one. Even for backblaze b2, it's the transfer that means 2-3tb data doesn't make sense on a personal budget (eg digital negatives and a few 4k videos). A restore become (too) expensive.

voiper1 7y ago

Wow, the prices are down to 2cents/GB, cheaper than standard AWS + without the fees. [1] Back in 2015, it was 20cents per GB. [2] Besides ZFS with snapshots, one of their benefits vs "dumb" cloud storage is you can run a server side component, e.g. borg which requires both a client and server. They even have slightly reduced pricing (which isn't that big a deal anymore, it was 3 cents vs 8-20 back in 2016 [3]) if you only need borg and don't need the configuration support. [4] EDIT: And, tons of other things I forgot about: git, s3cmd, and others! [5] [1] https://www.rsync.net/pricing.html https://www.rsync.net/pricing.html [2] https://web.archive.org/web/20150509033924/https://www.rsync.net/pricing.html https://web.archive.org/web/20150509033924/https://www.rsync... [3] https://web.archive.org/web/20160926192841/http://rsync.net/products/attic.html https://web.archive.org/web/20160926192841/http://rsync.net/... [4] http://rsync.net/products/borg.html http://rsync.net/products/borg.html [5] https://www.rsync.net/resources/howto/remote_commands.html https://www.rsync.net/resources/howto/remote_commands.html

rsync 7y ago
To be fair, we have let s3cmd wither on the vine a bit - I am not sure if we've updated it in the past 18 months. The reason being, anything s3cmd can do, rclone can do better. Right ?
- ebg13 7y ago
  > The reason being, anything s3cmd can do, rclone can do better. Right ? But is it better than s5cmd?
sigio 7y ago
Hetzner is down to 0.4ct/gb (EUR 39.90 for 10TB)
- Nux 7y ago
  Interesting, I can see 10TB for € 47.88, even less if you don't pay VAT (20%)
  - 7y ago

moviuro 7y ago

That snapshot[0] surely doesn't reflect the reality. Last Modified date is "today" for all snapshots when you advertize an immutable storage. You should also add the current date in the picture (just in case). [0] https://www.rsync.net/images/duck2.png https://www.rsync.net/images/duck2.png on https://archive.is/3kK28 https://archive.is/3kK28

voltagex_ 7y ago

I used to use this for <10GB with the old pricing model. Stopped when the family needed something they could use (ended up on Office Home + OneDrive). I could definitely find a use for 1-5GB of storage, but the minimum is 500GB ($10/USD month or $108/USD annually). Sorry John, looks like you had to chase me a bit due to an expired credit card. Hope I didn't cost too much for support. The most useful thing to me would be being able to use git clone and rclone to pull files closer to me, but I bet it's not an intended use case to use rsync.net as a proxy.

tpmx 7y ago

Does rsync.net have any way of doing two factor authentication, at least for administration purposes? Couldn't find anything on the site. Sure, I get that it's probably primarily built for unattended automated backups, but still.

rsync 7y ago
Yes. Our web based admin portal / management interface has 2FA options inside. There are also options to lock your entire account (or just your .ssh/authorized_keys file) to be read-only. The snapshots are always immutable, but you can toggle your entire account to be so, as well. Finally, inside the web based management is an "idle alert" where you can alert yourself (via email, SMS, pushover or webhook - or any combination thereof) to your account being unchanged for a (user configurable) amount of time.
- tpmx 7y ago
  Nice! These points seem like things you might want to put on the web site. (Did I miss it? I think I read through all of the public tech details/support pages though.) Btw: I found https://secured.fyi/Backup/rsync.net https://secured.fyi/Backup/rsync.net while googling this topic, it seems at least partially incorrect.

dceddia 7y ago

I thought this was refreshing to see :) > We are NOT reselling another cloud - we built and maintain our own infrastructure and have complete control over our platform. > We've been doing this since 2001.

penagwin 7y ago
Weekly warrant canaries too! https://www.rsync.net/resources/notices/canary.txt https://www.rsync.net/resources/notices/canary.txt

kup0 7y ago

Would consider using this if there wasn't a 500GB minimum.

sigy 7y ago

The price levels have a gap between 1TB and 10TB, or am I missing something?

cirrrrrrus 7y ago
9999 GB roughly equals to 9,999 TB

ivanhoe 7y ago

Is there option for having monthly/yearly snapshots?

rsync 7y ago
Yes. You can specify any arbitrary schedule of days/weeks/months/quarters/years. They are efficiently stored so if your data doesn't change much, they will take up almost no space - even though they present your entire filesystem from that date.

bachmeier 7y ago

15 GB of storage would be nice. Go to the "Simple Pricing" page. Click the 0-999 GB for 2 cents/GB link. That's great pricing. Select 15 GB. Get message telling me the minimum is 500 GB. That's not the kind of company I'm going to trust with my data. What other kind of BS are they going to try to pull if they're willing to do something like this before I'm even a customer? Maybe they don't want small customers like me. That's okay. Just be honest about it on your website.

lixtra 7y ago
From a UX perspective the first button shouldn’t read “0-9999 GB”, but “< 1 TB”. Otherwise I don’t see a problem with a minimum of 500GB to keep things efficient. On the other hand I believe there is a micro market waiting, paying 1-3 usd/month for 15GB.
- throwaway8941 7y ago
  You can already have something like this for free with Yandex Disk or Google Drive. rclone will let you mount the so called "cloud storage" locally and will encrypt everything you upload (be aware that encryption has to be configured separately as a remote on top of existing remote).
- cirrrrrrus 7y ago
  Well, 9999GB is closer to 10TB than to 1TB.
  - knorker 7y ago
    They meant <10TB, and were commenting on the fact that the range started with 0 when actually it starts at 500.
jsty 7y ago
Billing 30 cents is unlikely to even cover their credit card handling fees, let alone the cost of actually providing the service.
deleted 7y ago

fmakunbound 7y ago

Would mounting an SFTP file system using FUSE and then mounting encfs or veracrypt on top of that be reasonably reliable? I'm always concerned about my data at rest on servers others have access to.

rsync 7y ago
Yes. sshfs/FUSE is a very common use-case among rsync.net customers. However, borg (and to a lesser degree, restic) is the typical solution for encryption at rest.
- stevekemp 7y ago
  I use your borg-only service, and love it. I recall offering to donate you the .io domain at some point too!

tiernano 7y ago

Just signed up for 100gb Borg hosting for $18 for a year! That ain't bad. Some lints (no zfs snapshots) but Borg should have snapshots covered. Waiting for activation and will start playing around...

tiernano 7y ago
More details here. https://www.rsync.net/products/borg.html https://www.rsync.net/products/borg.html
- witten 7y ago
  Also check out borgmatic for configuring Borg. Works well with rsync.net: https://torsion.org/borgmatic/ https://torsion.org/borgmatic/

bndw 7y ago

What is Physical Media Shipment?

rsync 7y ago
If you need your data right now and your network connection isn't fast enough to do it in a timely manner, we will dump your data to a bare SATA drive and fedex it to you. Alternatively, if you have many TB to upload and your network connection makes that unworkable, we will accept your bare SATA drive for the onloading process.
- bndw 7y ago
  Awesome, thanks for the response.

knorker 7y ago

Wow, much cheaper than it used to be. I left them as a customer some years back purely based on price, but now I'd consider moving back. Other than being undercut by AWS (now also GCP) they were great.

zafiro17 7y ago

I did some searching around and finally settled on rsync as a backup system about 5 years ago, and I've been happy since. I don't understand the fixation on finding the lowest price solution - pay more, get more. With rsync you get their in-house hardware and engineers on the support desk. I use it to back up my FreeNAS ZFS shares, which is dead-simple process, and a bunch of web servers. Getting historical snapshots is genius. I also love being able to use any SFTP client I like instead of some dedicated client or a web interface. My data is not available through a web page - that's exactly the way I like it. Bravo Rsync, keep up the good work.

CloudBuddy 7y ago

Here is another option if you don't have much to store - https://cloudbuddy.cloud https://cloudbuddy.cloud