17 ms·

I must not interact with big enough data to understand, but what is the point of this? I have watched/read several of the Snow product descriptions from AWS an

I must not interact with big enough data to understand, but what is the point of this? I have watched/read several of the Snow product descriptions from AWS and I am still not quite sure I understand the point. It seems like a way to sync local big data to AWS cloud storage for data that is too large to realistically transfer via the internet. So is this simply a sneakernet external harddrive because physically shipping an 8TB hard drive has better bandwidth than using the internet? I would love it if someone could explain a couple of use cases for the Snow family of products to someone that has never had to handle 100 GB of data much less terabytes.

Gaelan 6y ago
I don't get this one, but for the larger Snow products, yeah, it's for transferring data too large to realistically sending over the internet.
txcwpalpha 6y ago
I'm with you. I think this is really cool but I have a hard time finding a use case for it. The use case for the other Snow products (Snowball, Snowmobile) are clearer, as they are for transferring petabytes of data to/from AWS without having to use slow internet connections. Snowcone could be used for this indeed, but at only 8 TB, the value proposition seems a lot less. Personally, I'd probably just suffer through having to spend a week doing a slow internet upload/download rather than paying for Snowcone. Also, the articles talks multiple times about using Snowcone to upload the data for you, over your own internet connection. Why would I pay for Snowcone to do that when I could just upload directly to S3, without using Snowcone as the middle layer, and get the same speeds? The compute aspect of it seems cool from a "im a tech enthusiast and this is cool" perspective, similar to how a Raspberry Pi is cool, but I don't see the real-world use case for hosting compute workloads on a rented Raspberry Pi. I would just buy a Raspberry Pi instead for cheaper. What this does make me interested in is something like this, but purchaseable outright without renting, a la AWS Outposts but the size of the Snowcone. It would be cool to have a swarm of Raspberry Pi sized devices that I controlled entirely through the AWS console with AWS services, and it would open up some niche use cases like having a tiny server cluster in places where I otherwise wouldn't have infrastructure, like a remote research camp or rural community.
- t0mas88 6y ago
  For upload it doesn't make much sense. For download it's less than half the cost compared to AWS outbound bandwidth to the internet.
  - txcwpalpha 6y ago
    Holy hell. I'm looking at the pricing example on this page [0], and at $460 to transfer out 15 TB, you're telling me this is less than half the price of normal internet outbound traffic? I knew AWS bandwidth costs were absurd, but it wasn't until seeing those numbers that it really hit me just how absurd they are. 0: https://aws.amazon.com/snowcone/pricing/ https://aws.amazon.com/snowcone/pricing/
    - WrtCdEvrydy 6y ago
      Yeah, come for the cheap compute, stay for the expensive network costs... forever.
- opless 6y ago
  > Hardware failures on amazon. “Someone else’s problem” That’s what you’re paying for on any cloud provider. Any hardware problem isn’t your problem and it (should be) abstracted away.
  - txcwpalpha 6y ago
    That's true for hardware that's actually "in the cloud" (aka at an AWS data center) where a hardware failure simply results in your workload automatically shifting to other hardware. But if a Snowcone breaks on you, it still certainly is your problem, because you're the one that now has a physical brick in your office and you need to find a replacement hardware for whatever you were previously using Snowcone for.
    - opless 6y ago
      My bad. I didn’t actually read the article!
ajsharp 6y ago
So yea, pretty much. I had to do this with a project from a couple years ago where we were moving a ton of images from a legacy colo'd server to S3. I can't remember how much data it was but it was enough to where it didn't make sense and would've taken for-fucking-ever to move it to S3 over public internet. When we did it, we bought a drive, went to the colo, offloaded the files, and you have to do some stuff to kick off a job to let them know you're sending a drive and what kind and all that sort of thing. Then you mail it to them and they run an import into an bucket. My guess with these drives is controlling the hardware makes the process significantly more efficient for them, and for the enterprise customers they're trying to move off colo and onto AWS.
t0mas88 6y ago
The speed is questionable in many countries where consumer internet connections already have 500mbit upload speed. But in the US I think bandwidth can be quite limited if you're not in a major city with fiber. The other big part is pricing. AWS charges $ 0.09 per GB for outbound transfer, so getting 8 TB of data out of AWS via the internet costs over $ 700. Doing it through Snowcone costs around $ 300 + shipping. Reasonable use-case would be transferring video material or high resolution sensor data.
- 3pt14159 6y ago
  It never ceases to amaze me that people pay that much for transfer. It's one tenth the price on DigitalOcean and it isn't like this is comparing two different types of things.
  - binaryblitz 6y ago
    DO is incredibly limited in terms of products compared to AWS.
    - 3pt14159 6y ago
      Yes but this is just data storage. I get it if you need to be on network, yadda, yadda, but so many people just senselessly use S3 and rack up bills that are easily avoidable.
      - p0rkbelly 6y ago
        Digital Ocean Spaces doesn't have the same durability designs of S3. Or performance in terms of TPS. It's not apples to apples. DO is great for small personal projects, but, I wouldn't run a fortune 500 company on it.
    - jeremyjh 6y ago
      What does that have to do with the price of bandwidth?
  - 6y ago
whoisjuan 6y ago
When talking about petabytes of data is definitely faster to physically offload the data, ship it, and physically load it on an edge location. Regarding the size of the data. Well, yes. There are companies with petabytes of data. Banking is one example that comes to my mind when thinking about petabytes scale data.
useless_foghorn 6y ago
I would assume that this would be for gathering information from remote data collection nodes or from computers that don't access networks. Instead of hooking up with a laptop and some kind of portable NAS, you could connect a snow product to slurp up the data and then mail it to Amazon to upload. I think their selling point is the reliability of the snow family, and not having to worry about the data after the initial upload which can be super critical for some business models.
austinkhale 6y ago
We used them in a B2B setting when signing an enterprise data contract for nationwide real estate MLS data, including photos. The photo's needed to be uploaded to our S3 bucket and the only method available to download was via FTP. It made more sense to ask the third party provider to download the images to these disks, send them to us, and then we shipped them to AWS.
thoraway1010 6y ago
Lots of potential uses: Imagine Amazon was bidding on a big Department of Defense contract ($ in billion). DoD let's say wants to send drones / UAV's whatever to do big sensor passes out in far off areas / imagery, wideband radio etc). There is NO fast internet for the TB's generated. You pop this out of the drone, put it on the next resupply mail flight out, and ingest into Amazon cloud on far side for analytics. Let's stay a store wants to collect 30 days of customer video related to all traffic in their 100 stores to do some sort of AI analysis on how folks use the stores. Again, some poorly paid store staff person can unplug this after a week and ship it off. Let's say I generate data but instead of wanting to do a snowmobile every year, I want to do more granular moves. Again, I could do a weekly or whatever ship, and again from maybe a few locations. I think especially in places with lower high speed internet penetration (africa, china etc) there is room for something "snackable" like this. Let's say I distribute movies to movie theatres. Again, do a weekly ship in/out. It's $60/job. Not bad. No data IN fees.
reaperducer 6y ago
I work in healthcare, sometimes we have to go to places with no internet, and sometimes with no electricity. We gather a lot of HIPAA-protected data. This might be useful for those events. Sometimes, even when we're in the middle of a big city, there's still no secure or reliable internet connection at the location we've been given, so this might be good for those cases, too. I'm not on the infrastructure side of things, so I can't say for sure. But I've already forwarded the page to our mobile IT team to make sure they've heard about it. Edit: They might also be good for television and movie production. In the early part of this century, rushes were sent from remote filming locations to the studios loaded onto iPods. With everything being 4K and higher now, this might be useful to move those rushes to a cloud location where they can be accessed by multiple people who need to see them in multiple cities.
- typedef_struct 6y ago
  Lord of the Rings was almost stolen this way: https://www.youtube.com/watch?v=Ndge8WlM9q8 https://www.youtube.com/watch?v=Ndge8WlM9q8
ThisIsTheWay 6y ago
For a sense of scale, my company used an AWS Snowmobile to transfer 100PB of archived satellite imagery to AWS. More details here: https://blog.maxar.com/earth-intelligence/2017/digitalglobe-moves-to-the-cloud-with-aws-snowmobile https://blog.maxar.com/earth-intelligence/2017/digitalglobe-...
- obviyus 6y ago
  That was an excellent read. Thanks for sharing.
- dcolkitt 6y ago
  "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
  - downerending 6y ago
    Or these days, micro flash cards.
  - ThisIsTheWay 6y ago
    For real! Reminds me of this article... "A carrier pigeon with a cargo of microSD cards can transfer large amounts of data faster, and more cheaply, than just about any other method" Link: https://spectrum.ieee.org/tech-talk/computing/networks/pigeonbased-feathernet-still-wingsdown-fastest-way-of-transferring-lots-of-data https://spectrum.ieee.org/tech-talk/computing/networks/pigeo...
    - firloop 6y ago
      This almost got me and then I looked at the date of publication.
      - CobrastanJorji 6y ago
        If you're new to joke RFCs, I suggest reading up on RFC 3514, a suggestion that malicious IP packets should be required to set the high order bit of the IP fragment offset field, which is to be designated the "evil" bit. This significantly simplifies implementing effective network monitoring.
sradman 6y ago
The video [1] quickly lists a number of use cases. > Explore the new AWS Snowcone with Bill Vass and Jeff Barr (2:46) [1] https://www.youtube.com/watch?v=8RNRssCiR_E&feature=emb_rel_end https://www.youtube.com/watch?v=8RNRssCiR_E&feature=emb_rel_...
Twirrim 6y ago
I can speak to part of the rationality around Snowball: The old method of customers sending in hard disks and AWS importing them turned out to be _incredibly_ high touch. * Customers would put the wrong labels on the wrong drives. Given they were encrypted, and the label details were critical to matching the right disk with the right encryption key, that was a big issue. * Imports would routinely fail due to all sorts of driver bugs both on the AWS side, and on the customer side. The number of fringe NTFS quirks was phenomenal, let alone all the wide range of other formats that had to be handled * Encryption tooling still really isn't all that user friendly. Too many opportunities for mistakes. * Drives would get damaged during shipping. * Stuff would go missing in shipping. It just didn't scale as a solution, and caused unending customer pain and frustration. Snowball solves _all_ of that, by taking out of the equation almost anything about the process that customers could get wrong. It handles all of the encryption stuff, is extremely ruggedised, has basic tracking. The label is an e-ink display that automatically contains all the right information. It becomes a "just works" solution.
- aketchum 6y ago
  great examples of why to use this over "just" a large external hard drive in a case. Thanks for the insight!
- akersten 6y ago
  Those are all really good points. I'm interested in hearing more about the NTFS quirks - what kind of issues were cropping up? Now I'm really curious what filesystem Snowball wound up standardizing on, and how/if file metadata/ACLs/alternate data streams are maintained.
  - dylz 6y ago
    Do those even matter if the final file destination is S3?
    - akersten 6y ago
      It's mostly a technical curiosity to me. Although maybe they would if you're using it in a system backup/archive workflow?
acdha 6y ago
It's sneakernet optimized to deliver direct to S3, with full end-to-end integrity checks. Depending on the amount of data you have and your internet connectivity, this can be significant optimization — for example, say, you work with professional photography or video but live in the United States where only a small fraction of people have an uplink speed greater than 10-30Mbps even if their download speeds are 500+Mbps. If you can generate 8TB of data in less than a month, this is faster and doesn't interfere with all of your other network usage.
jedberg 6y ago
One use case I know of is in a mining operation. They collect a ton of data about their equipment and their mines as they work, but they are often out in very desolate areas with no internet. The mining equipment runs greengrass right on the machine to do real time inference to help make mining more efficient as well as flag potential equipment failure to the operator. So they would strap this device onto the equipment, it would collect data and do inference, and then they can swap it out for a new one and send the old one back so the data can be added to their data lake to update all of the AI models.
searine 6y ago
I've used AWS snowballs fairly frequently and it is definitely a specialty product. For our use case it was genome sequencing. Each individual genome was a terabyte or so depending on what types of data we sent. We can crank out a few hundred genomes per week at full production so when our collaborators need to receive their data they usually neither have the capacity in bandwidth or storage for cohort of thousands. So it gets shipped to AWS. We've made improvements to get more direct connections to cloud services so we don't use them anymore but for awhile we were filling a snowball and shipping it every week or two. We also used the Google Transfer Appliance once or twice (480tb) to transfer projects all at once instead of piecemeal.
kanar1e 6y ago
I recently listened to an interview with the editorial team of Deadliest Catch (Art of the Cut #51). I could see this product being used by similar media productions where thousands of hours of film footage is generated in remote locations with low uplink speeds. Instead of shipping the drives all the way to the post-production house, the drives could be ingested at the nearest AWS facility.
downerending 6y ago
When you realize that your data is going to take 24 months to copy elsewhere (e.g., the cloud or another DC) over the fastest link you have, you have a real need for something like this.
rkagerer 6y ago
I imagine if you're already integrated into the AWS ecosystem, there's a convenience in treating your on-premise node as "just another node". You can run EC2 instances on it, and manage it using the same tools. But with better latency and more robust local connectivity due to the proximity. The ability to build AMIs and run them on Snowcones gives you the power to build applications that do all sorts of interesting filtering, pre-processing, and analysis at the edge. To me the product feels like snakeoil and I'm skeptical it will take off in a big way, but I think more than enough customers will use it to justify their investment. Remember, AWS is selling a platform. The more claws they can sink into their customers, the harder it is for them to switch to a competing one.
djohnston 6y ago
I had up back up several years of training data off-site and we used snowball to ship a few tbs of images. Not sure it was completely necessary but it was fun
vmception 6y ago
its physical edge storage, wasn't the use case clear? you can have the cloud right next to you! an entire node of a content distribution network, by itself in your rugged remote environment, but with you! you can cloud compute separated from the cloud, but together, with your snowcone!
airstrike 6y ago
I'm thinking defense / electronic warfare. All the data we'll be collecting in the field needs to travel back to a data center somehow. Not sure what the rules of the game for on vs. off-premise storage in that scenario, but I still think this is one of the main demand drivers
romanovcode 6y ago
> So is this simply a sneakernet external harddrive because physically shipping an 8TB hard drive has better bandwidth than using the internet? Yeah. When you have to migrate 100TB data from your own data store to AWS it is much easier and practical to just send the drives. They even help you with that, for a price of course.

Gaelan 6y ago

I don't get this one, but for the larger Snow products, yeah, it's for transferring data too large to realistically sending over the internet.

txcwpalpha 6y ago

I'm with you. I think this is really cool but I have a hard time finding a use case for it. The use case for the other Snow products (Snowball, Snowmobile) are clearer, as they are for transferring petabytes of data to/from AWS without having to use slow internet connections. Snowcone could be used for this indeed, but at only 8 TB, the value proposition seems a lot less. Personally, I'd probably just suffer through having to spend a week doing a slow internet upload/download rather than paying for Snowcone. Also, the articles talks multiple times about using Snowcone to upload the data for you, over your own internet connection. Why would I pay for Snowcone to do that when I could just upload directly to S3, without using Snowcone as the middle layer, and get the same speeds? The compute aspect of it seems cool from a "im a tech enthusiast and this is cool" perspective, similar to how a Raspberry Pi is cool, but I don't see the real-world use case for hosting compute workloads on a rented Raspberry Pi. I would just buy a Raspberry Pi instead for cheaper. What this does make me interested in is something like this, but purchaseable outright without renting, a la AWS Outposts but the size of the Snowcone. It would be cool to have a swarm of Raspberry Pi sized devices that I controlled entirely through the AWS console with AWS services, and it would open up some niche use cases like having a tiny server cluster in places where I otherwise wouldn't have infrastructure, like a remote research camp or rural community.

t0mas88 6y ago
For upload it doesn't make much sense. For download it's less than half the cost compared to AWS outbound bandwidth to the internet.
- txcwpalpha 6y ago
  Holy hell. I'm looking at the pricing example on this page [0], and at $460 to transfer out 15 TB, you're telling me this is less than half the price of normal internet outbound traffic? I knew AWS bandwidth costs were absurd, but it wasn't until seeing those numbers that it really hit me just how absurd they are. 0: https://aws.amazon.com/snowcone/pricing/ https://aws.amazon.com/snowcone/pricing/
  - WrtCdEvrydy 6y ago
    Yeah, come for the cheap compute, stay for the expensive network costs... forever.
opless 6y ago
> Hardware failures on amazon. “Someone else’s problem” That’s what you’re paying for on any cloud provider. Any hardware problem isn’t your problem and it (should be) abstracted away.
- txcwpalpha 6y ago
  That's true for hardware that's actually "in the cloud" (aka at an AWS data center) where a hardware failure simply results in your workload automatically shifting to other hardware. But if a Snowcone breaks on you, it still certainly is your problem, because you're the one that now has a physical brick in your office and you need to find a replacement hardware for whatever you were previously using Snowcone for.
  - opless 6y ago
    My bad. I didn’t actually read the article!

ajsharp 6y ago

So yea, pretty much. I had to do this with a project from a couple years ago where we were moving a ton of images from a legacy colo'd server to S3. I can't remember how much data it was but it was enough to where it didn't make sense and would've taken for-fucking-ever to move it to S3 over public internet. When we did it, we bought a drive, went to the colo, offloaded the files, and you have to do some stuff to kick off a job to let them know you're sending a drive and what kind and all that sort of thing. Then you mail it to them and they run an import into an bucket. My guess with these drives is controlling the hardware makes the process significantly more efficient for them, and for the enterprise customers they're trying to move off colo and onto AWS.

t0mas88 6y ago

The speed is questionable in many countries where consumer internet connections already have 500mbit upload speed. But in the US I think bandwidth can be quite limited if you're not in a major city with fiber. The other big part is pricing. AWS charges $ 0.09 per GB for outbound transfer, so getting 8 TB of data out of AWS via the internet costs over $ 700. Doing it through Snowcone costs around $ 300 + shipping. Reasonable use-case would be transferring video material or high resolution sensor data.

3pt14159 6y ago
It never ceases to amaze me that people pay that much for transfer. It's one tenth the price on DigitalOcean and it isn't like this is comparing two different types of things.
- binaryblitz 6y ago
  DO is incredibly limited in terms of products compared to AWS.
  - 3pt14159 6y ago
    Yes but this is just data storage. I get it if you need to be on network, yadda, yadda, but so many people just senselessly use S3 and rack up bills that are easily avoidable.
    - p0rkbelly 6y ago
      Digital Ocean Spaces doesn't have the same durability designs of S3. Or performance in terms of TPS. It's not apples to apples. DO is great for small personal projects, but, I wouldn't run a fortune 500 company on it.
  - jeremyjh 6y ago
    What does that have to do with the price of bandwidth?
- 6y ago

whoisjuan 6y ago

When talking about petabytes of data is definitely faster to physically offload the data, ship it, and physically load it on an edge location. Regarding the size of the data. Well, yes. There are companies with petabytes of data. Banking is one example that comes to my mind when thinking about petabytes scale data.

useless_foghorn 6y ago

I would assume that this would be for gathering information from remote data collection nodes or from computers that don't access networks. Instead of hooking up with a laptop and some kind of portable NAS, you could connect a snow product to slurp up the data and then mail it to Amazon to upload. I think their selling point is the reliability of the snow family, and not having to worry about the data after the initial upload which can be super critical for some business models.

austinkhale 6y ago

We used them in a B2B setting when signing an enterprise data contract for nationwide real estate MLS data, including photos. The photo's needed to be uploaded to our S3 bucket and the only method available to download was via FTP. It made more sense to ask the third party provider to download the images to these disks, send them to us, and then we shipped them to AWS.

thoraway1010 6y ago

Lots of potential uses: Imagine Amazon was bidding on a big Department of Defense contract ($ in billion). DoD let's say wants to send drones / UAV's whatever to do big sensor passes out in far off areas / imagery, wideband radio etc). There is NO fast internet for the TB's generated. You pop this out of the drone, put it on the next resupply mail flight out, and ingest into Amazon cloud on far side for analytics. Let's stay a store wants to collect 30 days of customer video related to all traffic in their 100 stores to do some sort of AI analysis on how folks use the stores. Again, some poorly paid store staff person can unplug this after a week and ship it off. Let's say I generate data but instead of wanting to do a snowmobile every year, I want to do more granular moves. Again, I could do a weekly or whatever ship, and again from maybe a few locations. I think especially in places with lower high speed internet penetration (africa, china etc) there is room for something "snackable" like this. Let's say I distribute movies to movie theatres. Again, do a weekly ship in/out. It's $60/job. Not bad. No data IN fees.

reaperducer 6y ago

I work in healthcare, sometimes we have to go to places with no internet, and sometimes with no electricity. We gather a lot of HIPAA-protected data. This might be useful for those events. Sometimes, even when we're in the middle of a big city, there's still no secure or reliable internet connection at the location we've been given, so this might be good for those cases, too. I'm not on the infrastructure side of things, so I can't say for sure. But I've already forwarded the page to our mobile IT team to make sure they've heard about it. Edit: They might also be good for television and movie production. In the early part of this century, rushes were sent from remote filming locations to the studios loaded onto iPods. With everything being 4K and higher now, this might be useful to move those rushes to a cloud location where they can be accessed by multiple people who need to see them in multiple cities.

typedef_struct 6y ago
Lord of the Rings was almost stolen this way: https://www.youtube.com/watch?v=Ndge8WlM9q8 https://www.youtube.com/watch?v=Ndge8WlM9q8

ThisIsTheWay 6y ago

For a sense of scale, my company used an AWS Snowmobile to transfer 100PB of archived satellite imagery to AWS. More details here: https://blog.maxar.com/earth-intelligence/2017/digitalglobe-moves-to-the-cloud-with-aws-snowmobile https://blog.maxar.com/earth-intelligence/2017/digitalglobe-...

obviyus 6y ago
That was an excellent read. Thanks for sharing.
dcolkitt 6y ago
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
- downerending 6y ago
  Or these days, micro flash cards.
- ThisIsTheWay 6y ago
  For real! Reminds me of this article... "A carrier pigeon with a cargo of microSD cards can transfer large amounts of data faster, and more cheaply, than just about any other method" Link: https://spectrum.ieee.org/tech-talk/computing/networks/pigeonbased-feathernet-still-wingsdown-fastest-way-of-transferring-lots-of-data https://spectrum.ieee.org/tech-talk/computing/networks/pigeo...
  - firloop 6y ago
    This almost got me and then I looked at the date of publication.
    - CobrastanJorji 6y ago
      If you're new to joke RFCs, I suggest reading up on RFC 3514, a suggestion that malicious IP packets should be required to set the high order bit of the IP fragment offset field, which is to be designated the "evil" bit. This significantly simplifies implementing effective network monitoring.

sradman 6y ago

The video [1] quickly lists a number of use cases. > Explore the new AWS Snowcone with Bill Vass and Jeff Barr (2:46) [1] https://www.youtube.com/watch?v=8RNRssCiR_E&feature=emb_rel_end https://www.youtube.com/watch?v=8RNRssCiR_E&feature=emb_rel_...

Twirrim 6y ago

I can speak to part of the rationality around Snowball: The old method of customers sending in hard disks and AWS importing them turned out to be _incredibly_ high touch. * Customers would put the wrong labels on the wrong drives. Given they were encrypted, and the label details were critical to matching the right disk with the right encryption key, that was a big issue. * Imports would routinely fail due to all sorts of driver bugs both on the AWS side, and on the customer side. The number of fringe NTFS quirks was phenomenal, let alone all the wide range of other formats that had to be handled * Encryption tooling still really isn't all that user friendly. Too many opportunities for mistakes. * Drives would get damaged during shipping. * Stuff would go missing in shipping. It just didn't scale as a solution, and caused unending customer pain and frustration. Snowball solves _all_ of that, by taking out of the equation almost anything about the process that customers could get wrong. It handles all of the encryption stuff, is extremely ruggedised, has basic tracking. The label is an e-ink display that automatically contains all the right information. It becomes a "just works" solution.

aketchum 6y ago
great examples of why to use this over "just" a large external hard drive in a case. Thanks for the insight!
akersten 6y ago
Those are all really good points. I'm interested in hearing more about the NTFS quirks - what kind of issues were cropping up? Now I'm really curious what filesystem Snowball wound up standardizing on, and how/if file metadata/ACLs/alternate data streams are maintained.
- dylz 6y ago
  Do those even matter if the final file destination is S3?
  - akersten 6y ago
    It's mostly a technical curiosity to me. Although maybe they would if you're using it in a system backup/archive workflow?

acdha 6y ago

It's sneakernet optimized to deliver direct to S3, with full end-to-end integrity checks. Depending on the amount of data you have and your internet connectivity, this can be significant optimization — for example, say, you work with professional photography or video but live in the United States where only a small fraction of people have an uplink speed greater than 10-30Mbps even if their download speeds are 500+Mbps. If you can generate 8TB of data in less than a month, this is faster and doesn't interfere with all of your other network usage.

jedberg 6y ago

One use case I know of is in a mining operation. They collect a ton of data about their equipment and their mines as they work, but they are often out in very desolate areas with no internet. The mining equipment runs greengrass right on the machine to do real time inference to help make mining more efficient as well as flag potential equipment failure to the operator. So they would strap this device onto the equipment, it would collect data and do inference, and then they can swap it out for a new one and send the old one back so the data can be added to their data lake to update all of the AI models.

searine 6y ago

I've used AWS snowballs fairly frequently and it is definitely a specialty product. For our use case it was genome sequencing. Each individual genome was a terabyte or so depending on what types of data we sent. We can crank out a few hundred genomes per week at full production so when our collaborators need to receive their data they usually neither have the capacity in bandwidth or storage for cohort of thousands. So it gets shipped to AWS. We've made improvements to get more direct connections to cloud services so we don't use them anymore but for awhile we were filling a snowball and shipping it every week or two. We also used the Google Transfer Appliance once or twice (480tb) to transfer projects all at once instead of piecemeal.

kanar1e 6y ago

I recently listened to an interview with the editorial team of Deadliest Catch (Art of the Cut #51). I could see this product being used by similar media productions where thousands of hours of film footage is generated in remote locations with low uplink speeds. Instead of shipping the drives all the way to the post-production house, the drives could be ingested at the nearest AWS facility.

downerending 6y ago

When you realize that your data is going to take 24 months to copy elsewhere (e.g., the cloud or another DC) over the fastest link you have, you have a real need for something like this.

rkagerer 6y ago

I imagine if you're already integrated into the AWS ecosystem, there's a convenience in treating your on-premise node as "just another node". You can run EC2 instances on it, and manage it using the same tools. But with better latency and more robust local connectivity due to the proximity. The ability to build AMIs and run them on Snowcones gives you the power to build applications that do all sorts of interesting filtering, pre-processing, and analysis at the edge. To me the product feels like snakeoil and I'm skeptical it will take off in a big way, but I think more than enough customers will use it to justify their investment. Remember, AWS is selling a platform. The more claws they can sink into their customers, the harder it is for them to switch to a competing one.

djohnston 6y ago

I had up back up several years of training data off-site and we used snowball to ship a few tbs of images. Not sure it was completely necessary but it was fun

vmception 6y ago

its physical edge storage, wasn't the use case clear? you can have the cloud right next to you! an entire node of a content distribution network, by itself in your rugged remote environment, but with you! you can cloud compute separated from the cloud, but together, with your snowcone!

airstrike 6y ago

I'm thinking defense / electronic warfare. All the data we'll be collecting in the field needs to travel back to a data center somehow. Not sure what the rules of the game for on vs. off-premise storage in that scenario, but I still think this is one of the main demand drivers

romanovcode 6y ago

> So is this simply a sneakernet external harddrive because physically shipping an 8TB hard drive has better bandwidth than using the internet? Yeah. When you have to migrate 100TB data from your own data store to AWS it is much easier and practical to just send the drives. They even help you with that, for a price of course.