5 ms·

They do have some impressive "I didn't know anyone would need that!" services. I also love this one: https://aws.amazon.com/snowmobile/ https://aws.amazon.com

They do have some impressive "I didn't know anyone would need that!" services. I also love this one: https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/ > AWS Snowmobile is an Exabyte-scale data transfer service used to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. (...)

phinnaeus 7y ago
They also have Snowballs, which are luggage-sized versions of the Snowmobile that can be shipped. They have a built-in e-ink display for the shipping label.
closeparen 7y ago
I wanted a small-scale version of this for my personal photos, back when I realized that uploading a few hundred GB on my 5mbps down/0.5mbps up residential cable connection was going to take weeks. (Ended up doing it from university, where I had symmetric 100mbit in my dorm room).
- paranoidrobot 7y ago
  That's what the snowballs are for.
  - holmberd 7y ago
    Without a doubt.
- tim-- 7y ago
  They also have a service where you can ship them a USB hard drive. https://aws.amazon.com/blogs/aws/send-us-that-data/ https://aws.amazon.com/blogs/aws/send-us-that-data/
- 2J0 7y ago
  This is my view of Renaissance if Internet cafés!
lonelappde 7y ago
Now I'm staring at my kitchen table visualizing a 100^3 pile of 1TB hard drives.
rockostrich 7y ago
The announcement for this at re:Invent was them literally driving an 18-wheeler onto the stage: https://www.youtube.com/watch?v=8vQmTZTq7nw https://www.youtube.com/watch?v=8vQmTZTq7nw
giaour 7y ago
I think the same customer (Digital Globe, an earth imagery LEO satellite operator) was on stage at both announcements. Snowmobile for getting PB of data from their ground network into AWS and Ground Station to get it there directly from space
partiallypro 7y ago
Microsoft has a similar service called Databox, wouldn't be surprised if GCS has something like this as well https://azure.microsoft.com/en-us/services/databox/ https://azure.microsoft.com/en-us/services/databox/
- deleted 7y ago
  [deleted]
- smueller1234 7y ago
  GCP Transfer Appliance: https://cloud.google.com/transfer-appliance https://cloud.google.com/transfer-appliance (I work at Google, don't work on that product.)
- wildfire 7y ago
  I actually used this personally (Jan 2019) You'll get supplied (at least) two SSDs. With instructions for how to use in both Windows and Linux. Unfortunately even though it was "GA", it appears that they never really actually tested using it in Linux. I submitted (and they were merged) a number of documentation bugfixes related to Linux. Two firmware updates later, one bricked disk, one which failed to verify the copied data -- and 9 months of time all down the drain. This reason is why I will never use Azure. It wasn't the bugs and device failures that was the problem. Just the sheer work in having to teach Azures' support staff about their own product (after fighting to get them to record the support case -- you normally can't raise one if you don't pay). Why did I pick Azure for this? Because (at the time) both GCP and AWS catered for 100Gb+ volumes. And we were (and remain) around 50Gb. I haven't re-evaluated to see if either GCP or AWS have better options now.
jsolson 7y ago
I work on Google Cloud and have an obvious bias here, but having thought about this for a couple year now, this does not seem like a good or cost-effective way to move that much data. In particular if you have ~100PB of data to move and you are in a location that can be reached by a 45-foot long shipping container on a truck with access to the ~350KW of power it takes to run Snowmobile, you're clearly not somewhere _completely_ inaccessible. Given that 100PB of data will cost you $400k/mo in storage costs on AWS Glacier (before the discounts that you'll obviously negotiate for), even relatively remote locations become "accessible" for 100 Gbps+ fiber or microwave connectivity, and _very_ remote locations for tens of Gbps. The Snowmobile itself is only 1 Tb/s, so if you think you're going to save time this work, consider how long it takes to move it to you and then move it to an AWS facility versus the time it takes to fill it. I don't get where this is the right call for _any_ customers, even as a one-off. I'd love it if someone from AWS could tell me where my math is off on this.
- snowwrestler 7y ago
  It reads to me like you're saying: if you're going to spend $400k/month on storage cost, why not pay for a network buildout too? I don't think that allocating X per month for op-ex implies that you also have Y amount of money to spend on cap-ex... especially if you don't own the location. 350kw of power is just another trailer to pull up.
  - jsolson 7y ago
    I am assuming that anyone who managed to accumulate 100 PB of data (or close enough to make this viable) is still accumulating data at a solid clip, or likely has a lot _more_ than just 100 PB of data
- KaiserPro 7y ago
  its about timing. Say I have datacentre in london, Sure I could plumb in 4 100gig lines. But they are typically on a year lease and have a long lead time. Plus, once its on the datatruck its AWS's problem.
- mike_d 7y ago
josephmosby 7y ago
The real reason Snowmobile exists: to knock out potential opposition to large-scale migration from on-prem data centers into AWS. Most of the intro decisions around moving to the cloud happen before Amazon is ever involved in the sales cycle. There are some managerial accountants and IT people getting involved to evaluate options in a process called "Evaluation of Alternatives." At some point, Amazon probably started hearing that the IT people were saying "we can't possibly move all of that data to the cloud, it would swamp our network or take five years just to finish the network transfer." Enter Snowmobile, which cuts that conversation thread off at the knees. Amazon can go to a Fortune 100 company that is evaluating a move and honestly claim that they are prepared to move exabytes of data, physically, into the cloud. And it's not some premium feature that they're going to get charged bajillions for, it's a stock feature of AWS. (Even if it costs a lot of money) Snowmobile isn't targeted at the average customer, it's targeted specifically at the accountants, IT architects, and management consultants evaluating the decision to move all of Fortune 100 Company X from on-prem data centers into 100% AWS. One extra hug from Amazon to let the most risk-averse person know that they have it covered, and would they please go ahead and write their 9-figure check to AWS.

phinnaeus 7y ago

They also have Snowballs, which are luggage-sized versions of the Snowmobile that can be shipped. They have a built-in e-ink display for the shipping label.

closeparen 7y ago

I wanted a small-scale version of this for my personal photos, back when I realized that uploading a few hundred GB on my 5mbps down/0.5mbps up residential cable connection was going to take weeks. (Ended up doing it from university, where I had symmetric 100mbit in my dorm room).

paranoidrobot 7y ago
That's what the snowballs are for.
- holmberd 7y ago
  Without a doubt.
tim-- 7y ago
They also have a service where you can ship them a USB hard drive. https://aws.amazon.com/blogs/aws/send-us-that-data/ https://aws.amazon.com/blogs/aws/send-us-that-data/
2J0 7y ago
This is my view of Renaissance if Internet cafés!

lonelappde 7y ago

Now I'm staring at my kitchen table visualizing a 100^3 pile of 1TB hard drives.

rockostrich 7y ago

The announcement for this at re:Invent was them literally driving an 18-wheeler onto the stage: https://www.youtube.com/watch?v=8vQmTZTq7nw https://www.youtube.com/watch?v=8vQmTZTq7nw

giaour 7y ago

I think the same customer (Digital Globe, an earth imagery LEO satellite operator) was on stage at both announcements. Snowmobile for getting PB of data from their ground network into AWS and Ground Station to get it there directly from space

partiallypro 7y ago

Microsoft has a similar service called Databox, wouldn't be surprised if GCS has something like this as well https://azure.microsoft.com/en-us/services/databox/ https://azure.microsoft.com/en-us/services/databox/

deleted 7y ago
[deleted]
smueller1234 7y ago
GCP Transfer Appliance: https://cloud.google.com/transfer-appliance https://cloud.google.com/transfer-appliance (I work at Google, don't work on that product.)
wildfire 7y ago
I actually used this personally (Jan 2019) You'll get supplied (at least) two SSDs. With instructions for how to use in both Windows and Linux. Unfortunately even though it was "GA", it appears that they never really actually tested using it in Linux. I submitted (and they were merged) a number of documentation bugfixes related to Linux. Two firmware updates later, one bricked disk, one which failed to verify the copied data -- and 9 months of time all down the drain. This reason is why I will never use Azure. It wasn't the bugs and device failures that was the problem. Just the sheer work in having to teach Azures' support staff about their own product (after fighting to get them to record the support case -- you normally can't raise one if you don't pay). Why did I pick Azure for this? Because (at the time) both GCP and AWS catered for 100Gb+ volumes. And we were (and remain) around 50Gb. I haven't re-evaluated to see if either GCP or AWS have better options now.

jsolson 7y ago

I work on Google Cloud and have an obvious bias here, but having thought about this for a couple year now, this does not seem like a good or cost-effective way to move that much data. In particular if you have ~100PB of data to move and you are in a location that can be reached by a 45-foot long shipping container on a truck with access to the ~350KW of power it takes to run Snowmobile, you're clearly not somewhere _completely_ inaccessible. Given that 100PB of data will cost you $400k/mo in storage costs on AWS Glacier (before the discounts that you'll obviously negotiate for), even relatively remote locations become "accessible" for 100 Gbps+ fiber or microwave connectivity, and _very_ remote locations for tens of Gbps. The Snowmobile itself is only 1 Tb/s, so if you think you're going to save time this work, consider how long it takes to move it to you and then move it to an AWS facility versus the time it takes to fill it. I don't get where this is the right call for _any_ customers, even as a one-off. I'd love it if someone from AWS could tell me where my math is off on this.

snowwrestler 7y ago
It reads to me like you're saying: if you're going to spend $400k/month on storage cost, why not pay for a network buildout too? I don't think that allocating X per month for op-ex implies that you also have Y amount of money to spend on cap-ex... especially if you don't own the location. 350kw of power is just another trailer to pull up.
- jsolson 7y ago
  I am assuming that anyone who managed to accumulate 100 PB of data (or close enough to make this viable) is still accumulating data at a solid clip, or likely has a lot _more_ than just 100 PB of data
KaiserPro 7y ago
its about timing. Say I have datacentre in london, Sure I could plumb in 4 100gig lines. But they are typically on a year lease and have a long lead time. Plus, once its on the datatruck its AWS's problem.
mike_d 7y ago

josephmosby 7y ago

The real reason Snowmobile exists: to knock out potential opposition to large-scale migration from on-prem data centers into AWS. Most of the intro decisions around moving to the cloud happen before Amazon is ever involved in the sales cycle. There are some managerial accountants and IT people getting involved to evaluate options in a process called "Evaluation of Alternatives." At some point, Amazon probably started hearing that the IT people were saying "we can't possibly move all of that data to the cloud, it would swamp our network or take five years just to finish the network transfer." Enter Snowmobile, which cuts that conversation thread off at the knees. Amazon can go to a Fortune 100 company that is evaluating a move and honestly claim that they are prepared to move exabytes of data, physically, into the cloud. And it's not some premium feature that they're going to get charged bajillions for, it's a stock feature of AWS. (Even if it costs a lot of money) Snowmobile isn't targeted at the average customer, it's targeted specifically at the accountants, IT architects, and management consultants evaluating the decision to move all of Fortune 100 Company X from on-prem data centers into 100% AWS. One extra hug from Amazon to let the most risk-averse person know that they have it covered, and would they please go ahead and write their 9-figure check to AWS.