3 ms·

Any evidence for any of those claims? Putting anti-tampering failsafes and GPS tracking on the Snowmobile doesn’t seem like such a stretch, when you consider th

by deno 9y ago

Any evidence for any of those claims? Putting anti-tampering failsafes and GPS tracking on the Snowmobile doesn’t seem like such a stretch, when you consider they don’t rent this out to just anyone on the street. And 100 PB is still a lot of data. Facebook in its IPO filing said their entire media library was at the time 100 PB. Backblaze’s total backup size was 150 PB in 2015, just 2 years ago. I’m sure S3 is at exabyte scale now, but even so, 100 PB is not a drop in anyone’s bucket.

Twirrim 9y ago
> Any evidence for any of those claims? Putting anti-tampering failsafes and GPS tracking on the Snowmobile doesn’t seem like such a stretch, when you consider they don’t rent this out to just anyone on the street. > And 100 PB is still a lot of data. Facebook in its IPO filing said their entire media library was at the time 100 PB. Backblaze’s total backup size was 150 PB in 2015, just 2 years ago. I’m sure S3 is at exabyte scale now, but even so, 100 PB is not a drop in anyone’s bucket. I'm ex-AWS, and used to work on a team closely involved with both S3 and Import/Export (the team that produced snowmobile, though that expansion launched after I left) I'm well aware of the kinds of capacity scale S3 operates at, as well as some other storage teams. Remember that backblaze, for all its scale, operates out of a small number of datacentres, where AWS has operations right across the world. Being ex-AWS also why I'm familiar with the way AWS Security thinks, and I can easily picture how they'd hit the roof if S3 was "ported" on to the device. That's before we even begin to tackle the concept of just how hard it would be to port S3 in to a small scale platform as the Snowmobile. Don't forget, you're not just talking about all the things that make up S3, but all the things that make up Amazon infrastructure as a whole. One of the reasons Amazon is able to push out so many services is because they've got a mature and well established ecosystem behind the scenes that is designed to operate at scale. You'd be talking year(s) of effort to port S3 into a snowmobile, at best, vs. a matter of a few months to port and fully test a layer on top of the device. When it all boils down to it, the S3 API is pretty simple. Why on earth would you choose the most technically complicated way to approach the problem?
- deno 9y ago
  That’s much better for a source, thanks. Do notice the context, however. We’re approaching this problem from the perspective of trying to determine the network bandwidth of a portable datacenter driving down the highway. Assuming that container is really just a giant USB harddrive then it’s 10 days for copy at client’s site, 8 hours for the drive, and then another 10 days to assimilate the data back at AWS DC, or more if pushing it to 2 other DCs is slower. Perhaps there’s some way to alleviate some of those concerns by shipping minimal S3-API server in the Snowmobile, then deploy actual production code when it’s assimilated back home. Perhaps the Snowmobile could become its own region temporarily. Still, you’re right that is a lot of moving parts for something that you’d want to avoid… Except in this extremely important hypothetical, obviously. Also, while it is probably true that S3 has way more storage distributed across the globe, I think the Snowmobile still follows the Sat Nav not DNS & BGP ;) I mean it delivers to a single DC, so that’s what needs to be considered. Actually 100 PB is a lot on disk, but it’s even more over the network. They’ll probably want to push it during off-peaks, but should probably consider to just keep driving the truck to the next AZ.

by deno 9y ago

Twirrim 9y ago

> Any evidence for any of those claims? Putting anti-tampering failsafes and GPS tracking on the Snowmobile doesn’t seem like such a stretch, when you consider they don’t rent this out to just anyone on the street. > And 100 PB is still a lot of data. Facebook in its IPO filing said their entire media library was at the time 100 PB. Backblaze’s total backup size was 150 PB in 2015, just 2 years ago. I’m sure S3 is at exabyte scale now, but even so, 100 PB is not a drop in anyone’s bucket. I'm ex-AWS, and used to work on a team closely involved with both S3 and Import/Export (the team that produced snowmobile, though that expansion launched after I left) I'm well aware of the kinds of capacity scale S3 operates at, as well as some other storage teams. Remember that backblaze, for all its scale, operates out of a small number of datacentres, where AWS has operations right across the world. Being ex-AWS also why I'm familiar with the way AWS Security thinks, and I can easily picture how they'd hit the roof if S3 was "ported" on to the device. That's before we even begin to tackle the concept of just how hard it would be to port S3 in to a small scale platform as the Snowmobile. Don't forget, you're not just talking about all the things that make up S3, but all the things that make up Amazon infrastructure as a whole. One of the reasons Amazon is able to push out so many services is because they've got a mature and well established ecosystem behind the scenes that is designed to operate at scale. You'd be talking year(s) of effort to port S3 into a snowmobile, at best, vs. a matter of a few months to port and fully test a layer on top of the device. When it all boils down to it, the S3 API is pretty simple. Why on earth would you choose the most technically complicated way to approach the problem?

deno 9y ago
That’s much better for a source, thanks. Do notice the context, however. We’re approaching this problem from the perspective of trying to determine the network bandwidth of a portable datacenter driving down the highway. Assuming that container is really just a giant USB harddrive then it’s 10 days for copy at client’s site, 8 hours for the drive, and then another 10 days to assimilate the data back at AWS DC, or more if pushing it to 2 other DCs is slower. Perhaps there’s some way to alleviate some of those concerns by shipping minimal S3-API server in the Snowmobile, then deploy actual production code when it’s assimilated back home. Perhaps the Snowmobile could become its own region temporarily. Still, you’re right that is a lot of moving parts for something that you’d want to avoid… Except in this extremely important hypothetical, obviously. Also, while it is probably true that S3 has way more storage distributed across the globe, I think the Snowmobile still follows the Sat Nav not DNS & BGP ;) I mean it delivers to a single DC, so that’s what needs to be considered. Actually 100 PB is a lot on disk, but it’s even more over the network. They’ll probably want to push it during off-peaks, but should probably consider to just keep driving the truck to the next AZ.