3 ms·

My funny story is built on the idea that AWS is Hotel California for your data. A customer had an interest in merging the data from an older account into a new

My funny story is built on the idea that AWS is Hotel California for your data. A customer had an interest in merging the data from an older account into a new one, just to simplify matters. Enterprise data. Going back years. Not even leaving the region. The AWS rep in the meeting kinda pauses, says: "We'll get back to you on the cost to do that." The sticker shock was enough that the customer simply inherited the old account, rather than making things tidy.

hhsectech 3y ago
Eh? I've never had a problem moving data out of AWS. Have people lost the ability to write export and backup scripts?
- LadyCailin 3y ago
  It’s the cost of data egress, which isn’t free.
  - mciancia 3y ago
    But there is no paid egress when we are moving data between account within one region, rigth?
    - storyinmemo 3y ago
      There is. You pay a price for any cross-VPC traffic.
      - CubsFan1060 3y ago
        This isn't true, at least not anymore. You can peer two vpc's and as long as you are transferring within the same (real) AZ, it's free: https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/ https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vp... Even peered VPC's only pay "normal" prices: https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer "Data transferred "in" to and "out" from Amazon EC2, Amazon RDS, Amazon Redshift, Amazon DynamoDB Accelerator (DAX), and Amazon ElastiCache instances, Elastic Network Interfaces or VPC Peering connections across Availability Zones in the same AWS Region is charged at $0.01/GB in each direction."
- Draiken 3y ago
  The ingress/egress cost is ridiculously high. Some companies don't care, but it is there and I've seen it catch people off guard multiple times.
  - varjag 3y ago
    Oh come on from the description both accounts could be sitting on the same datacenter LAN.
- interroboink 3y ago
  My (peripheral) experience is that it is much cheaper to get data in than to get data out. When you have the amount of data being discussed — "Enterprise data. Going back years." — that can get very costly. It's the amount of data where it makes more sense to put hard drives on a truck and drive across the country rather than send it over a network, where this becomes an issue (actually, probably a bit before then).
  - fcarraldo 3y ago
    AWS actually has a service for this - Snowmobile, a storage datacenter inside of a shipping container, which is driven to you on a semi truck. https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/
    - xmcqdpt2 3y ago
      They do not! > Q: Can I export data from AWS with Snowmobile? > > Snowmobile does not support data export. It is designed to let you quickly, easily, and more securely migrate exabytes of data to AWS. When you need to export data from AWS, you can use AWS Snowball Edge to quickly export up to 100TB per appliance and run multiple export jobs in parallel as necessary. Visit the Snowball Edge FAQs to learn more. https://aws.amazon.com/snowmobile/faqs/?nc2=h_mo-lang https://aws.amazon.com/snowmobile/faqs/?nc2=h_mo-lang Why would they make it convenient to leave?
      - fcarraldo 3y ago
        Oh, TIL! Thanks for correcting me.
    - brickteacup 3y ago
      That's only for data into AWS though, not data out
- Shorel 3y ago
  Just in network costs, there's a huge asymmetry. Uploading data to AWS is free. Downloading data from them, you have to pay. When you have enough data, that cost is quite significant.
- mijoharas 3y ago
  There's a cost for data egress (but not ingress)
banku_brougham 3y ago
Is R2 a sensible option for hosting data? I understand egress is chesp.
- stickfigure 3y ago
  R2 is great. Our GCS bill (almost all egress) jumped from a few hundred dollars a month to a couple thousand dollars a month last year due to a usage spike. We rush-migrated to R2 and now that part of the bill is $0. I've heard some people here on HN say that it's slow, but I haven't noticed a difference. We're mainly dealing with multi-megabyte image files, so YMMV if you have a different workload.
  - banku_brougham 3y ago
    awesome. I remember reading about this a while ago, but never tried. Since it has the same API i can imagine its not daunting as a multi-cloud infrastructure. I guess permissions might be more complex, as in EC2 instance profiles wouldnt grant access, etc.
    - stickfigure 3y ago
      Just to make sure nobody is confused by this - R2 has the same API as S3, not GCS. We had to build a simple abstraction around GCS/S3 to perform the migration. But if you're migrating from S3, it's pretty much drop-in. We even use the AWS-provided S3 Java library (especially convenient for making signed URLs).

hhsectech 3y ago

Eh? I've never had a problem moving data out of AWS. Have people lost the ability to write export and backup scripts?

LadyCailin 3y ago
It’s the cost of data egress, which isn’t free.
- mciancia 3y ago
  But there is no paid egress when we are moving data between account within one region, rigth?
  - storyinmemo 3y ago
    There is. You pay a price for any cross-VPC traffic.
    - CubsFan1060 3y ago
      This isn't true, at least not anymore. You can peer two vpc's and as long as you are transferring within the same (real) AZ, it's free: https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/ https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vp... Even peered VPC's only pay "normal" prices: https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer "Data transferred "in" to and "out" from Amazon EC2, Amazon RDS, Amazon Redshift, Amazon DynamoDB Accelerator (DAX), and Amazon ElastiCache instances, Elastic Network Interfaces or VPC Peering connections across Availability Zones in the same AWS Region is charged at $0.01/GB in each direction."
Draiken 3y ago
The ingress/egress cost is ridiculously high. Some companies don't care, but it is there and I've seen it catch people off guard multiple times.
- varjag 3y ago
  Oh come on from the description both accounts could be sitting on the same datacenter LAN.
interroboink 3y ago
My (peripheral) experience is that it is much cheaper to get data in than to get data out. When you have the amount of data being discussed — "Enterprise data. Going back years." — that can get very costly. It's the amount of data where it makes more sense to put hard drives on a truck and drive across the country rather than send it over a network, where this becomes an issue (actually, probably a bit before then).
- fcarraldo 3y ago
  AWS actually has a service for this - Snowmobile, a storage datacenter inside of a shipping container, which is driven to you on a semi truck. https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/
  - xmcqdpt2 3y ago
    They do not! > Q: Can I export data from AWS with Snowmobile? > > Snowmobile does not support data export. It is designed to let you quickly, easily, and more securely migrate exabytes of data to AWS. When you need to export data from AWS, you can use AWS Snowball Edge to quickly export up to 100TB per appliance and run multiple export jobs in parallel as necessary. Visit the Snowball Edge FAQs to learn more. https://aws.amazon.com/snowmobile/faqs/?nc2=h_mo-lang https://aws.amazon.com/snowmobile/faqs/?nc2=h_mo-lang Why would they make it convenient to leave?
    - fcarraldo 3y ago
      Oh, TIL! Thanks for correcting me.
  - brickteacup 3y ago
    That's only for data into AWS though, not data out
Shorel 3y ago
Just in network costs, there's a huge asymmetry. Uploading data to AWS is free. Downloading data from them, you have to pay. When you have enough data, that cost is quite significant.
mijoharas 3y ago
There's a cost for data egress (but not ingress)

banku_brougham 3y ago

Is R2 a sensible option for hosting data? I understand egress is chesp.

stickfigure 3y ago
R2 is great. Our GCS bill (almost all egress) jumped from a few hundred dollars a month to a couple thousand dollars a month last year due to a usage spike. We rush-migrated to R2 and now that part of the bill is $0. I've heard some people here on HN say that it's slow, but I haven't noticed a difference. We're mainly dealing with multi-megabyte image files, so YMMV if you have a different workload.
- banku_brougham 3y ago
  awesome. I remember reading about this a while ago, but never tried. Since it has the same API i can imagine its not daunting as a multi-cloud infrastructure. I guess permissions might be more complex, as in EC2 instance profiles wouldnt grant access, etc.
  - stickfigure 3y ago
    Just to make sure nobody is confused by this - R2 has the same API as S3, not GCS. We had to build a simple abstraction around GCS/S3 to perform the migration. But if you're migrating from S3, it's pretty much drop-in. We even use the AWS-provided S3 Java library (especially convenient for making signed URLs).