4 ms·
I can speak to part of the rationality around Snowball: The old method of customers sending in hard disks and AWS importing them turned out to be _incredibly_
by Twirrim 6y ago
I can speak to part of the rationality around Snowball:
The old method of customers sending in hard disks and AWS importing them turned out to be _incredibly_ high touch.
* Customers would put the wrong labels on the wrong drives. Given they were encrypted, and the label details were critical to matching the right disk with the right encryption key, that was a big issue.
* Imports would routinely fail due to all sorts of driver bugs both on the AWS side, and on the customer side. The number of fringe NTFS quirks was phenomenal, let alone all the wide range of other formats that had to be handled
* Encryption tooling still really isn't all that user friendly. Too many opportunities for mistakes.
* Drives would get damaged during shipping.
* Stuff would go missing in shipping.
It just didn't scale as a solution, and caused unending customer pain and frustration.
Snowball solves _all_ of that, by taking out of the equation almost anything about the process that customers could get wrong. It handles all of the encryption stuff, is extremely ruggedised, has basic tracking. The label is an e-ink display that automatically contains all the right information. It becomes a "just works" solution.
- aketchum 6y agogreat examples of why to use this over "just" a large external hard drive in a case. Thanks for the insight!
- akersten 6y agoThose are all really good points. I'm interested in hearing more about the NTFS quirks - what kind of issues were cropping up? Now I'm really curious what filesystem Snowball wound up standardizing on, and how/if file metadata/ACLs/alternate data streams are maintained.
- dylz 6y agoDo those even matter if the final file destination is S3?
- akersten 6y agoIt's mostly a technical curiosity to me. Although maybe they would if you're using it in a system backup/archive workflow?
- rainbowzootsuit 6y agoI helped a client do a transfer to S3 via snowball once for a music studio client with a smorgusboard of random hard drives. The software client for doing the transfer is some sort of command line JVM tool from my memory. You locally create S3 buckets on the snowball and copy from the local disks into the buckets. Once the snowball is returned, the buckets appear in your online account.
- Twirrim 6y agoThe main issues with NTFS would be the system either wouldn't be able to see the drive, or would fail to be able to read part of it. They were building up quite a repertoire of tricks to get things to read correctly, but it was almost impossible to safely automate.
- asdfman123 6y agoIt's just sneakernet with bells and whistles. As it should be.
- mrtweetyhack 6y agoDoesn't solve * Drives would get damaged during shipping. * Stuff would go missing in shipping.
- thayne 6y agosure it does. the drives are specifically designed to be rugged enough to handle transportation (and are tamper resistant and tamper evident), and I imagine amazon probably uses their own delivery network to transport it, which means if it is lost in shipping, it is amazon's fault (and presumable can be corrected). It also wouldn't surprise me if at least some of the product contain GPS to make it less likely to get lost.
- joncrane 6y agoPretty sure the E-Ink on the snowball devices are set up for UPS tracking label format.
- Twirrim 6y agoIt does. The Snowball was designed in a ruggedised fashion. It can take some extremely large shocks while in transit without putting customer data at risk. I forget the precise tests carried out, but I believe they even included "dropped from roof of multi-storey building". Stuff going missing in shipping is solved by very low powered tracking chips in them. AWS knows where those devices are and could tell shipping firms if needs be.