3 ms·
I'll add that it isn't just data egress. It's also inter-AZ transfer which costs $0.01/GB. But it's really $0.02/GB since you get charged 1 cent for the traffic
by mdasen 2y ago
I'll add that it isn't just data egress. It's also inter-AZ transfer which costs $0.01/GB. But it's really $0.02/GB since you get charged 1 cent for the traffic leaving and 1 cent for the traffic arriving on the other side.
If you're running in AWS, you want to have servers in multiple availability zones in the same region so that if one AZ goes down you're still up. Let's say you're running your APIs and Kafka in two AZs (us-east-1a, us-east-1b). A user hits your API in 1a and you publish a message to Kafka. You need to write to 3 boxes. On average, you're paying $0.03/GB for your Kafka writes - $0.02/GB when the API server is in the same AZ as two of the quorum and $0.04/GB when the API server is in the same AZ as only one of the quorum.
That's 30x what Hetzner is charging for internet egress from its US data centers. People here were outraged that Hetzner would charge $0.001/GB for internet egress while using AWS is costing 30x that for transfer in the same region. Plus, a lot of companies use 3 AZs (so that they still have high availability if one AZ goes down). Then you're paying the $0.04/GB on all writes.
And this doesn't just apply to Kafka. Let's say you're running PostgreSQL with standbys: you're paying for all that data transfer from the API nodes in other AZs to the leader node and all that data transfer to replicate the writes to the standby nodes. When reading, you could do AZ-aware reads from the standby nodes if you don't need the latest data to avoid charges.
At some point, it feels like we're spending a lot of time and effort trying to minimize cloud costs - costs that we don't have to worry about from traditional datacenter providers. The cloud was supposed to make things easier, but the costs have made a lot of stuff harder.
And I'm not saying that AWS should charge nothing for inter-AZ traffic - they're just charging way too much. I don't know how much it really costs to service that traffic, but Hetzner can send the traffic over the internet for $0.001/GB from data centers that they don't own. It feels like Amazon should be able to send that traffic down the road for a good bit less than that. I believe in charging customers for what they're using so that you don't have bad actors exploiting pricing loopholes - but Amazon charges an absolute fortune for stuff that's relatively cheap.