4 ms·

First sentence on the linked page: "With AWS Snowball, you pay only for your use of the device and for data transfer out of AWS."

leetrout 5y ago
Wow that’s up to $500,000 just to export 10PB (depending on region).
- canucker2016 5y ago
  According to https://aws.amazon.com/snowball/pricing/ https://aws.amazon.com/snowball/pricing/, egress fees depends on the region, which can range from $0.03/GB (North America & parts of Europe) to $0.05/GB (parts of Asia and Africa). So US$300K to US$500K for egress fees + cost of Snowball devices. The major downside of Snowball in this export case is the size limit of 80TB per device - from https://aws.amazon.com/snowball/features/ https://aws.amazon.com/snowball/features/ : "Snowball Edge Storage Optimized provides 80 TB of HDD capacity for block volumes and Amazon S3-compatible object storage, and 1 TB of SATA SSD for block volumes." That'd be around 125 Snowball devices to get 10PB out. If OP actually has 10PB on S3 currently, the OP may want to fallback to leaving the existing data on S3 and accessing new data in the new location.
  - nicoburns 5y ago
    There's also the snowmobile https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/
    - canucker2016 5y ago
      The AWS Snowmobile pages only talk about migrating INTO AWS, not OUT OF. from https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/ : AWS Snowmobile is an Exabyte-scale data transfer service used to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. Snowmobile makes it easy to move massive volumes of data to the cloud, including video libraries, image repositories, or even a complete data center migration. from https://aws.amazon.com/snowmobile/faqs/ https://aws.amazon.com/snowmobile/faqs/ : Q: What is AWS Snowmobile? AWS Snowmobile is the first exabyte-scale data migration service that allows you to move very large datasets from on-premises to AWS.
      - hn_throwaway_99 5y ago
        I mean, the title on the snowmobile page says: > Migrate or transport exabyte-scale data sets into and out of AWS
        orcinus8 5y ago
        Unfortunately the header is misleading. The FAQ says: Q: Can I export data from AWS with Snowmobile? Snowmobile does not support data export. It is designed to let you quickly, easily, and more securely migrate exabytes of data to AWS. When you need to export data from AWS, you can use AWS Snowball Edge to quickly export up to 100TB per appliance and run multiple export jobs in parallel as necessary
  - deleted 5y ago
    [deleted]
  - phekunde 5y ago
    > If OP actually has 10PB on S3 currently, the OP may want to fallback to leaving the existing data on S3 and accessing new data in the new location. I remember asking an Amazon executive in London when AWS was very new and they were evangelising it to developers; I asked him what is the cost of getting data out of AWS if I want to move it to other service provider, or how easy it will be? And he avoided giving a straight simple answer. I realised then than the business model from the start was to lock-in developer/startups/companies in to the AWS ecosystem.
  - prirun 5y ago
    > If OP actually has 10PB on S3 currently, the OP may want to fallback to leaving the existing data on S3 and accessing new data in the new location. Another option would be to leave data on S3, store new data locally, and proxy all S3 download requests, ie, all requests go to the local system first. If an object is on S3, download it, store it locally, then pass it on to your customer. That way your data will gradually migrate away from S3. Of course you can speed this up to any degree you want by copying objects from S3 without a customer request. An advantage of doing this is that you can phase in your solution gradually, for example: Phase 1: direct all requests to local proxies, always get the data from S3, send it to customers. You can do this before any local storage servers are setup. Phase 2: configure a local storage server, send all requests to S3, store the S3 data before sending to customers. If the local storage server is full, skip the store. Phase 3: send requests to S3, if local servers have the data, verify it matches, send to customer Phase 4: if local servers have the data, send it w/o S3 request. If not, make S3 request, store it locally, send data Phase 5: store new data both locally and on S3 At this point you are still storing data on S3, so it can be considered your master copy and your local copy is basically a cache. If you lose your entire local store, everything will still work, assuming your proxies work. For the next phase, your local copy becomes the master, so you need to make sure backups, replication, etc are all working before proceeding. Phase 5: start storing new content locally only. Phase 6: as a background maintenance task, start sending list requests to S3. For objects that are stored locally, issue S3 delete requests to the biggest objects first, at whatever rate you want. If an object isn't stored locally, make a note that you need to sync it sometime. Phase 7: using the sync list, copy S3 objects locally, biggest objects first, and remove them from S3. The advantage IMO is that it's a gradual cutover, so you don't have to have a complete, perfect local solution before you start gaining experience with new technology.
natch 5y ago
That wording is not inconsistent with the interpretation that Snowball is for in only.

leetrout 5y ago

Wow that’s up to $500,000 just to export 10PB (depending on region).

canucker2016 5y ago
According to https://aws.amazon.com/snowball/pricing/ https://aws.amazon.com/snowball/pricing/, egress fees depends on the region, which can range from $0.03/GB (North America & parts of Europe) to $0.05/GB (parts of Asia and Africa). So US$300K to US$500K for egress fees + cost of Snowball devices. The major downside of Snowball in this export case is the size limit of 80TB per device - from https://aws.amazon.com/snowball/features/ https://aws.amazon.com/snowball/features/ : "Snowball Edge Storage Optimized provides 80 TB of HDD capacity for block volumes and Amazon S3-compatible object storage, and 1 TB of SATA SSD for block volumes." That'd be around 125 Snowball devices to get 10PB out. If OP actually has 10PB on S3 currently, the OP may want to fallback to leaving the existing data on S3 and accessing new data in the new location.
- nicoburns 5y ago
  There's also the snowmobile https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/
  - canucker2016 5y ago
    The AWS Snowmobile pages only talk about migrating INTO AWS, not OUT OF. from https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/ : AWS Snowmobile is an Exabyte-scale data transfer service used to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. Snowmobile makes it easy to move massive volumes of data to the cloud, including video libraries, image repositories, or even a complete data center migration. from https://aws.amazon.com/snowmobile/faqs/ https://aws.amazon.com/snowmobile/faqs/ : Q: What is AWS Snowmobile? AWS Snowmobile is the first exabyte-scale data migration service that allows you to move very large datasets from on-premises to AWS.
    - hn_throwaway_99 5y ago
      I mean, the title on the snowmobile page says: > Migrate or transport exabyte-scale data sets into and out of AWS
      - orcinus8 5y ago
        Unfortunately the header is misleading. The FAQ says: Q: Can I export data from AWS with Snowmobile? Snowmobile does not support data export. It is designed to let you quickly, easily, and more securely migrate exabytes of data to AWS. When you need to export data from AWS, you can use AWS Snowball Edge to quickly export up to 100TB per appliance and run multiple export jobs in parallel as necessary
- deleted 5y ago
  [deleted]
- phekunde 5y ago
  > If OP actually has 10PB on S3 currently, the OP may want to fallback to leaving the existing data on S3 and accessing new data in the new location. I remember asking an Amazon executive in London when AWS was very new and they were evangelising it to developers; I asked him what is the cost of getting data out of AWS if I want to move it to other service provider, or how easy it will be? And he avoided giving a straight simple answer. I realised then than the business model from the start was to lock-in developer/startups/companies in to the AWS ecosystem.
- prirun 5y ago
  > If OP actually has 10PB on S3 currently, the OP may want to fallback to leaving the existing data on S3 and accessing new data in the new location. Another option would be to leave data on S3, store new data locally, and proxy all S3 download requests, ie, all requests go to the local system first. If an object is on S3, download it, store it locally, then pass it on to your customer. That way your data will gradually migrate away from S3. Of course you can speed this up to any degree you want by copying objects from S3 without a customer request. An advantage of doing this is that you can phase in your solution gradually, for example: Phase 1: direct all requests to local proxies, always get the data from S3, send it to customers. You can do this before any local storage servers are setup. Phase 2: configure a local storage server, send all requests to S3, store the S3 data before sending to customers. If the local storage server is full, skip the store. Phase 3: send requests to S3, if local servers have the data, verify it matches, send to customer Phase 4: if local servers have the data, send it w/o S3 request. If not, make S3 request, store it locally, send data Phase 5: store new data both locally and on S3 At this point you are still storing data on S3, so it can be considered your master copy and your local copy is basically a cache. If you lose your entire local store, everything will still work, assuming your proxies work. For the next phase, your local copy becomes the master, so you need to make sure backups, replication, etc are all working before proceeding. Phase 5: start storing new content locally only. Phase 6: as a background maintenance task, start sending list requests to S3. For objects that are stored locally, issue S3 delete requests to the biggest objects first, at whatever rate you want. If an object isn't stored locally, make a note that you need to sync it sometime. Phase 7: using the sync list, copy S3 objects locally, biggest objects first, and remove them from S3. The advantage IMO is that it's a gradual cutover, so you don't have to have a complete, perfect local solution before you start gaining experience with new technology.

natch 5y ago

That wording is not inconsistent with the interpretation that Snowball is for in only.