5 ms·

Just a year ago B2 couldn't do server-side file copying.[1] If you wanted to rename or move a file you had to re-upload the whole thing (not great for large mul

Just a year ago B2 couldn't do server-side file copying.[1] If you wanted to rename or move a file you had to re-upload the whole thing (not great for large multi-gigabyte files)! That ruled them out of consideration for storing my personal backups. Glad to see they've since fixed that, and with this update are clearly continuing to improve ergonomics. I'll have to give B2 a fresh look. [1]: https://github.com/Backblaze/B2_Command_Line_Tool/issues/175 https://github.com/Backblaze/B2_Command_Line_Tool/issues/175

atYevP 6y ago
Yev here -> thanks! We're constantly working on making the platform better, and copy-file was definitely a widely requested feature! That plus S3 Compatibility, for folks who wanted to integrated with B2 Cloud Storage but didn't have the resources to write code to our B2 Native API.
- rarrrrrr 6y ago
  I've been using B2 to disrupt a bunch of ugly & entrenched vendors in the price sensitive K12 market. Thanks for building it. :)
  - budmang 6y ago
    Can you share what you do with B2? (Are you hosting educational content? Something else?)
  - atYevP 6y ago
    Ah that's awesome! I'd love to more know! How are you using B2 in general, and does this make it easier for you? If you want to leave a note here, or you can send it to: b2feedback@backblaze.com!
    - ajinkyapatil 6y ago
      are there any plans to host public datasets like aws pds ?
- shliachtx 6y ago
  I just finished writing a new feature for internal use using S3. I hadn't even considered Backblaze at the time, but now that I am seeing this news we may end up using it, considering there is virtually no cost (to switch) and we haven't deployed yet.
nicolaslem 6y ago
For backups, mapping one file to one object rarely works well. Tools that use this strategy come with a long list of scenarios requiring expensive operations. For instance renaming a directory or changing a few bytes in a large file. On the other hand tools that don't view the object storage as a file system have way less gotchas. In my experience B2 + restic works really well.
- waffle_ss 6y ago
  That's a fair point but this is for my family photos and videos; in the event I die I'd rather the S3 bucket I hand over to my wife/kids in my last wishes look like a real filesystem rather than a bazillion blobs that require a special tool with a programmer's expertise to reassemble
  - sime2009 6y ago
    May I ask which solution you ended up going for in this situation? (I'm trying to solve for myself too.)
    - ac29 6y ago
      rclone is a nice way to sync files to B2. It also has a mount option that quite literally mounts the cloud storage as a filesystem, though this requires Linux and a little bit of know how. However, in a pinch B2 has a serviceable web interface, and Backblaze will even ship you a drive of your files if you request it, so I think it would be pretty usable by just about anyone.
    - waffle_ss 6y ago
      I have a local ZFS pool of hard drives and a script that `rsync -avz`s my iPhone's photos+videos to it. Then a separate cron script that periodically syncs from the ZFS pool to my S3 bucket using `aws s3 sync`. The S3 bucket has versioning turned on so it's effectively append-only. I used to be able to trigger my iPhone -> ZFS script when I plugged my iPhone into my Ubuntu desktop using udev (also had to wrap it in flock[1] because it would trigger multiple times for some reason), but at some point that stopped working and I've been too lazy to figure out why. It's far from perfect but for me it works alright. In this scenario I prefer straightforward and slightly kludgy compared to something with hidden complexity that could go wrong in so many ways. Could you imagine if you used a tool like restic or borg and the pack encoding format changed, or if the tool sources are simply gone when your relatives have to figure out how to get at the files in 10, 15, 20 years - I don't want my relatives playing code detective or archaeologist! Which reminds me of a downside to using tools like restic, borg, and the like I forgot to mention. When I evaluated them for my hundreds of GB of family pics+videos, there is a "dedupe" step that all these tools want to perform. When I tested them a couple years ago they were dog slow for my files, because pics + video are already highly compressed and there is very little "deduping" you're going to wring out of them unless you have multiple copies of the same files. IIRC borg took several hours to run at at the end it reported 0.01% or less deduping efficiency. Also as I recall there was no way to opt-out of the dedupe step due to the way borg stores "packs". Very annoying! [1]: https://stackoverflow.com/a/169969/215168 https://stackoverflow.com/a/169969/215168

atYevP 6y ago

Yev here -> thanks! We're constantly working on making the platform better, and copy-file was definitely a widely requested feature! That plus S3 Compatibility, for folks who wanted to integrated with B2 Cloud Storage but didn't have the resources to write code to our B2 Native API.

rarrrrrr 6y ago
I've been using B2 to disrupt a bunch of ugly & entrenched vendors in the price sensitive K12 market. Thanks for building it. :)
- budmang 6y ago
  Can you share what you do with B2? (Are you hosting educational content? Something else?)
- atYevP 6y ago
  Ah that's awesome! I'd love to more know! How are you using B2 in general, and does this make it easier for you? If you want to leave a note here, or you can send it to: b2feedback@backblaze.com!
  - ajinkyapatil 6y ago
    are there any plans to host public datasets like aws pds ?
shliachtx 6y ago
I just finished writing a new feature for internal use using S3. I hadn't even considered Backblaze at the time, but now that I am seeing this news we may end up using it, considering there is virtually no cost (to switch) and we haven't deployed yet.

nicolaslem 6y ago

For backups, mapping one file to one object rarely works well. Tools that use this strategy come with a long list of scenarios requiring expensive operations. For instance renaming a directory or changing a few bytes in a large file. On the other hand tools that don't view the object storage as a file system have way less gotchas. In my experience B2 + restic works really well.

waffle_ss 6y ago
That's a fair point but this is for my family photos and videos; in the event I die I'd rather the S3 bucket I hand over to my wife/kids in my last wishes look like a real filesystem rather than a bazillion blobs that require a special tool with a programmer's expertise to reassemble
- sime2009 6y ago
  May I ask which solution you ended up going for in this situation? (I'm trying to solve for myself too.)
  - ac29 6y ago
    rclone is a nice way to sync files to B2. It also has a mount option that quite literally mounts the cloud storage as a filesystem, though this requires Linux and a little bit of know how. However, in a pinch B2 has a serviceable web interface, and Backblaze will even ship you a drive of your files if you request it, so I think it would be pretty usable by just about anyone.
  - waffle_ss 6y ago
    I have a local ZFS pool of hard drives and a script that `rsync -avz`s my iPhone's photos+videos to it. Then a separate cron script that periodically syncs from the ZFS pool to my S3 bucket using `aws s3 sync`. The S3 bucket has versioning turned on so it's effectively append-only. I used to be able to trigger my iPhone -> ZFS script when I plugged my iPhone into my Ubuntu desktop using udev (also had to wrap it in flock[1] because it would trigger multiple times for some reason), but at some point that stopped working and I've been too lazy to figure out why. It's far from perfect but for me it works alright. In this scenario I prefer straightforward and slightly kludgy compared to something with hidden complexity that could go wrong in so many ways. Could you imagine if you used a tool like restic or borg and the pack encoding format changed, or if the tool sources are simply gone when your relatives have to figure out how to get at the files in 10, 15, 20 years - I don't want my relatives playing code detective or archaeologist! Which reminds me of a downside to using tools like restic, borg, and the like I forgot to mention. When I evaluated them for my hundreds of GB of family pics+videos, there is a "dedupe" step that all these tools want to perform. When I tested them a couple years ago they were dog slow for my files, because pics + video are already highly compressed and there is very little "deduping" you're going to wring out of them unless you have multiple copies of the same files. IIRC borg took several hours to run at at the end it reported 0.01% or less deduping efficiency. Also as I recall there was no way to opt-out of the dedupe step due to the way borg stores "packs". Very annoying! [1]: https://stackoverflow.com/a/169969/215168 https://stackoverflow.com/a/169969/215168