5 ms·

"In other words, ingress (data sent to AWS) doesn’t cost them any more or less than egress (data sent from AWS). And yet, they charge customers more to take dat

"In other words, ingress (data sent to AWS) doesn’t cost them any more or less than egress (data sent from AWS). And yet, they charge customers more to take data out than put it in. It’s a head scratcher. We’ve tried to be charitable in trying to understand why AWS would charge this way. Disappointingly, there just doesn't seem to be an innocent explanation. " One consideration could be that you can't directly control data ingress. Charging for ingress opens an avenue for attackers to attempt to bankrupt you by sending unrequested packets to your network. At the very least, this could cause a customer service headache. Perhaps AWS decided they'd rather not deal with that?

sitkack 5y ago
> One consideration could be that you can't directly control data ingress. Are you thinking UDP? Or for a DDOS? Connection setup overhead could be accounted for. I highly doubt this is the reason.
- linuxftw 5y ago
  If there's an IP exposed on the internet, you can just send it tcp payloads. The end destination will silently drop them, but it doesn't mean people can't send you gigs of useless data.
  - otterley 5y ago
    I think you mean UDP. TCP requires a 3-way handshake first.
    - ithkuil 5y ago
      Intermediate routers don't care about that; they only forward the IP packets; four target host/firewall will drop them (because they don't belong to a valid connection) but they will be still accounted for as ingress traffic.
      - otterley 5y ago
        Correct, but you can't get much bandwidth through until the 3-way handshake is completed. Sending a bunch of unanswered SYN packets isn't really a great way to instigate a DDoS, compared to sending avalanches of 64KiB UDP packets.
        ithkuil 5y ago
        As long as there is no connection tracking you can send whatever crap you want, including replayed packets from the middle of a connection, perhaps even huge packets with a syn flag ... As long as the accounting happens before a firewall performs basic TCP sanity checks you're going to pay for it
        kazen44 5y ago
        in some setups, you could send non IP packets (especially in MPLS fabric if accounting happens by LSP).
- blamethenetwork 5y ago
  At a routing and peering level. Once you have an announcement for your netblock out there, traffic will start to head towards it. A lot of this is due to the BGP Path Selection Algorithm. You can try and influence how traffic arrives, by doing things like, AS prepends, but you are still going to get traffic. The main reason for this is that the other side that is egressing to you has their own egress policy that also follows path selection. Things like localpref and weight will force my traffic to leave via a path before it considers how a network has AS padded. As an example: Lets say I want to egress (company A) to a downstream company (company B). If I learn routes to Company B via multiple ways: peering fabric (low cost), paid peering (medium cost), transit1 (high cost, variable quality), transit2 (low cost variable quality), I can choose which way my traffic goes, via localpref, weight etc. Only when I view the paths equally (equal localpref, weight etc.) will I evaluate the shortest AS Path (which the receiving company has influence on). The only way to completely not get inbound traffic via a specific link, is to remove your BGP advertisement for your netblock from that link. (some providers also let you do this selectively via BGP communities). There are also some other tips/tricks - such as adding a more specific prefix to a certain link, to attract traffic, but care needs to be made to have a fallback route in case things go wonky.
tw04 5y ago
Coming from someone who believes that the AWS egress fees are highway robbery, I'd throw this out there. In order to maintain settlement free peering, you need to keep egress and ingress somewhat even with your peers. Amazon proper probably has 3x egress vs ingress between prime and twitch. AWS is an easy avenue to balance it out. All that being said: I would guess there's almost 0 chance that's what's happening here. Amazon is just using it as a form of lock-in IMO.
- hnpasf 5y ago
  I agree that it is just lock-in. The price of egress is the same even when using devices like the Snowball where the data is loaded onto a server and physically shipped to you.
qeternity 5y ago
> One consideration could be that you can't directly control data ingress. Charging for ingress opens an avenue for attackers to attempt to bankrupt you by sending unrequested packets to your network. Most request/response models are very asymmetric. I can request a relatively large asset from S3 with a small request and maliciously generate fees as an attacker. AWS egress is vendor lock-in 100%.
eli 5y ago
It's also just a way to segment customers. The people doing tons of egress, on average, probably have bigger budgets for cloud computing. There's no particular reason AWS costs have to mirror how they charge.
res0nat0r 5y ago
The only service AWS hasn't consistently lowered pricing on (unlike almost everything else) is their egress cost. Everyone complains about vendor "lock in" for anything AWS related, and I've never thought so since they've always reduced prices across the board. But egress is the exception. They're keeping pricing high for this exact reason to prevent folks from moving their data elsewhere. I've got about 25TB of stuff I'd love to keep all in Glacier Deep Archive, but the restore costs are just too insane to justify due to the egress pricing still at .09c after all these years. Too bad.
- skuhn 5y ago
  I view network egress as the primary lock-in mechanism for all of AWS. If you want to migrate to another provider, network egress costs mean that you'll spend multiples of your normal monthly operating costs to do so. That stifles competition. But well before that point, most services in AWS have already been built around avoiding network egress wherever possible. You're always going to prefer AWS APIs and services, even if they aren't the best for your use case, because services outside AWS have a network egress tariff placed on them. So you aren't always buying best-of-breed, you're buying the best of what AWS chooses to offer (or the selection of vendors that choose to build in AWS to remain competitive). And if that isn't good enough, your only other option is to migrate out (and pay those egress costs!).
- qeternity 5y ago
  > Everyone complains about vendor "lock in" for anything AWS related, and I've never thought so since they've always reduced prices across the board. How would keeping prices high for these services keep people locked in? The whole point of vendor lock in is there is some other indirect cost that you bear by switching, which is why you don't switch. For AWS, this indirect cost is bandwidth. It is the lock in mechanism.
  - res0nat0r 5y ago
    > How would keeping prices high for these services keep people locked in? By doing the opposite and raising prices across the board randomly because they know folks can't spend the time and money to migrate without a large support burden or time/money sunk cost. The only service not going down in price is egress and the only thing I've seen which has been like this, to directly discourage folks from migrating their data to say GCP when their offering(s) look more attractive.
isbvhodnvemrwvn 5y ago
If you get DDoSed you also pay through the roof in fees (for whatever is in your VPC - e.g. ELB which are billed per throughput) unless you buy their anti-DDoS services (AWS Shield Advanced) for 3k$/mo - without that AWS won't waive the fees.
RKearney 5y ago
> One consideration could be that you can't directly control data ingress. Charging for ingress opens an avenue for attackers to attempt to bankrupt you by sending unrequested packets to your network. At the very least, this could cause a customer service headache. Perhaps AWS decided they'd rather not deal with that? An equally likely reason is by making ingress free it helps balance out ingress and egress traffic flows which are usually desirable for settlement-free peering agreements.
kevincox 5y ago
The simple explanation is that ingress is far, far lower than egress so the cost isn't relevant since links tend to be symmetrical. So the cost to them is determined based on egress, their ingress provisioning happens at no additional cost.

sitkack 5y ago

> One consideration could be that you can't directly control data ingress. Are you thinking UDP? Or for a DDOS? Connection setup overhead could be accounted for. I highly doubt this is the reason.

linuxftw 5y ago
If there's an IP exposed on the internet, you can just send it tcp payloads. The end destination will silently drop them, but it doesn't mean people can't send you gigs of useless data.
- otterley 5y ago
  I think you mean UDP. TCP requires a 3-way handshake first.
  - ithkuil 5y ago
    Intermediate routers don't care about that; they only forward the IP packets; four target host/firewall will drop them (because they don't belong to a valid connection) but they will be still accounted for as ingress traffic.
    - otterley 5y ago
      Correct, but you can't get much bandwidth through until the 3-way handshake is completed. Sending a bunch of unanswered SYN packets isn't really a great way to instigate a DDoS, compared to sending avalanches of 64KiB UDP packets.
      - ithkuil 5y ago
        As long as there is no connection tracking you can send whatever crap you want, including replayed packets from the middle of a connection, perhaps even huge packets with a syn flag ... As long as the accounting happens before a firewall performs basic TCP sanity checks you're going to pay for it
        kazen44 5y ago
        in some setups, you could send non IP packets (especially in MPLS fabric if accounting happens by LSP).
blamethenetwork 5y ago
At a routing and peering level. Once you have an announcement for your netblock out there, traffic will start to head towards it. A lot of this is due to the BGP Path Selection Algorithm. You can try and influence how traffic arrives, by doing things like, AS prepends, but you are still going to get traffic. The main reason for this is that the other side that is egressing to you has their own egress policy that also follows path selection. Things like localpref and weight will force my traffic to leave via a path before it considers how a network has AS padded. As an example: Lets say I want to egress (company A) to a downstream company (company B). If I learn routes to Company B via multiple ways: peering fabric (low cost), paid peering (medium cost), transit1 (high cost, variable quality), transit2 (low cost variable quality), I can choose which way my traffic goes, via localpref, weight etc. Only when I view the paths equally (equal localpref, weight etc.) will I evaluate the shortest AS Path (which the receiving company has influence on). The only way to completely not get inbound traffic via a specific link, is to remove your BGP advertisement for your netblock from that link. (some providers also let you do this selectively via BGP communities). There are also some other tips/tricks - such as adding a more specific prefix to a certain link, to attract traffic, but care needs to be made to have a fallback route in case things go wonky.

tw04 5y ago

Coming from someone who believes that the AWS egress fees are highway robbery, I'd throw this out there. In order to maintain settlement free peering, you need to keep egress and ingress somewhat even with your peers. Amazon proper probably has 3x egress vs ingress between prime and twitch. AWS is an easy avenue to balance it out. All that being said: I would guess there's almost 0 chance that's what's happening here. Amazon is just using it as a form of lock-in IMO.

hnpasf 5y ago
I agree that it is just lock-in. The price of egress is the same even when using devices like the Snowball where the data is loaded onto a server and physically shipped to you.

qeternity 5y ago

> One consideration could be that you can't directly control data ingress. Charging for ingress opens an avenue for attackers to attempt to bankrupt you by sending unrequested packets to your network. Most request/response models are very asymmetric. I can request a relatively large asset from S3 with a small request and maliciously generate fees as an attacker. AWS egress is vendor lock-in 100%.

eli 5y ago

It's also just a way to segment customers. The people doing tons of egress, on average, probably have bigger budgets for cloud computing. There's no particular reason AWS costs have to mirror how they charge.

res0nat0r 5y ago

The only service AWS hasn't consistently lowered pricing on (unlike almost everything else) is their egress cost. Everyone complains about vendor "lock in" for anything AWS related, and I've never thought so since they've always reduced prices across the board. But egress is the exception. They're keeping pricing high for this exact reason to prevent folks from moving their data elsewhere. I've got about 25TB of stuff I'd love to keep all in Glacier Deep Archive, but the restore costs are just too insane to justify due to the egress pricing still at .09c after all these years. Too bad.

skuhn 5y ago
I view network egress as the primary lock-in mechanism for all of AWS. If you want to migrate to another provider, network egress costs mean that you'll spend multiples of your normal monthly operating costs to do so. That stifles competition. But well before that point, most services in AWS have already been built around avoiding network egress wherever possible. You're always going to prefer AWS APIs and services, even if they aren't the best for your use case, because services outside AWS have a network egress tariff placed on them. So you aren't always buying best-of-breed, you're buying the best of what AWS chooses to offer (or the selection of vendors that choose to build in AWS to remain competitive). And if that isn't good enough, your only other option is to migrate out (and pay those egress costs!).
qeternity 5y ago
> Everyone complains about vendor "lock in" for anything AWS related, and I've never thought so since they've always reduced prices across the board. How would keeping prices high for these services keep people locked in? The whole point of vendor lock in is there is some other indirect cost that you bear by switching, which is why you don't switch. For AWS, this indirect cost is bandwidth. It is the lock in mechanism.
- res0nat0r 5y ago
  > How would keeping prices high for these services keep people locked in? By doing the opposite and raising prices across the board randomly because they know folks can't spend the time and money to migrate without a large support burden or time/money sunk cost. The only service not going down in price is egress and the only thing I've seen which has been like this, to directly discourage folks from migrating their data to say GCP when their offering(s) look more attractive.

isbvhodnvemrwvn 5y ago

If you get DDoSed you also pay through the roof in fees (for whatever is in your VPC - e.g. ELB which are billed per throughput) unless you buy their anti-DDoS services (AWS Shield Advanced) for 3k$/mo - without that AWS won't waive the fees.

RKearney 5y ago

> One consideration could be that you can't directly control data ingress. Charging for ingress opens an avenue for attackers to attempt to bankrupt you by sending unrequested packets to your network. At the very least, this could cause a customer service headache. Perhaps AWS decided they'd rather not deal with that? An equally likely reason is by making ingress free it helps balance out ingress and egress traffic flows which are usually desirable for settlement-free peering agreements.

kevincox 5y ago

The simple explanation is that ingress is far, far lower than egress so the cost isn't relevant since links tend to be symmetrical. So the cost to them is determined based on egress, their ingress provisioning happens at no additional cost.