8 ms·

I'll share my trick :) Lightsail instances can be used to "proxy" data from other AWS resources (eg EC2 instances or S3 buckets). Each Lightsail instance has a

I'll share my trick :) Lightsail instances can be used to "proxy" data from other AWS resources (eg EC2 instances or S3 buckets). Each Lightsail instance has a certain amount of data transfer included in it's price ($3.5 instance has 1TB, $5 instance has 2TB, $10 instance has 3TB, $20 instance has 4TB, $40 instance has 5TB). The best value (dollar per transferred data) is the $10 instance, which gives you 3TB of traffic. Using the data provided by the post: 3TB worth of traffic from an EC2 would cost $276.48 (us-east-1). 3TB worth of traffic from a S3 bucket would cost $69. Note: one downside of using Lightsail instances is that both ingress and egress traffic counts as "traffic".

jonatron 3y ago
https://aws.amazon.com/service-terms/ https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspend your data services or suspend your account.
- develatio 3y ago
  A had a suspicion that this was against AWS's terms, but I never bothered to look if that was actually the case. Thank you for the heads up!
  - slashdev 3y ago
    It’s mostly in there to scare people into not doing it. AFAIK they’ve never taken action on that. Of course if you abuse it, you’re asking for trouble.
    - mdasen 3y ago
      As someone who has dealt with users who use a system in an unintended way, you don't go looking for those people and you don't build something to enforce a policy like this. When you're running services for lots of customers, you often don't know a lot of what's going on in the system and how people are using it. Then something seems weird or something is causing a problem and you want to deal with it - and you want the language out there so that you can deal with it. In Amazon's case, their bandwidth pricing isn't really defendable. It's just crap. However, sometimes you're trying to offer something reasonable, but need to make sure that a customer doesn't end up abusing something. For example, Chia is a cryptocurrency that will basically wear through SSDs (it's a proof-of-space system). There aren't explicit limits on how frequently you can write to a disk from most hosting providers, but Chia goes beyond what normal usage would do to a disk. Chia farmers would rather burn someone else's SSD that they're renting than their own. But no one at most hosting providers was probably looking at how frequently people were writing before noticing "hey, why are the disks failing faster than we'd expect?" They probably haven't taken action on it because they probably haven't noticed it being a problem. But if you're a whale of a customer and suddenly your data transfer charges drop off a cliff, someone might end up looking into that and seeing what's going on.
- Hamuko 3y ago
  At least AWS is fully aware how premium their normal data transfer is and that one might want to optimise those costs.
  - vidarh 3y ago
    It's extreme enough that I never willingly serve data directly from AWS without a caching proxy elsewhere in front unless the egress is tint. It takes very low hitrates before it pays for itself several times over including management overheads. Sometimes you can justify a complete replica outside AWS (one of the things I will gladly pay AWS for is durability)
- mmh0000 3y ago
  Yeah, but "service terms" are just recommendations that should often be ignored.
- sangnoir 3y ago
  > You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services This requires proving the users intent, which is not obvious except in the most blatant of cases (i.e. using Lightsail as a bent-pipe by writing the exact bytes you're reading). If it is a "CSV to Parquet translation layer", how would AWS possibly prove it's anything other than what it claims to be? You'd be paying a few more cents for compute, but that's the price of plausible deniability
  - greyface- 3y ago
    > This requires proving the users intent Companies are permitted to deny service to anyone at any time for any (non-protected) reason. They typically don't have to justify service terminations to a court of law. Who would they be required to prove user intent to, and why?
    - cstrahan 3y ago
      I don't think you parsed their message correctly. It's not about litigation. Re-posting a bit of the service terms for easy reference: > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services [...] As you point out, they may terminate your service without any justification in a court of law. So how do they go about terminating the offenders? Well, one trivial way (from a technology and/or policy perspective): terminate everyone's service! If you blindly terminate everyone's service, that will certainly prevent anyone abusing LightSail. But that's, uh, not good for business. So they probably want to terminate the service of only those people actually abusing it. But how do you do that? You'd have to look at each account's usage and do something to determine if that traffic is or isn't a means of avoiding data fees from other services. In other words, you'd have to determine the intent of that traffic. Or, put yet another way: "this requires proving the users intent". If doing so was as trivial as detecting any traffic between LightSail and the other services, they'd just prevent such connections in the first place. So how can AWS tell if some traffic between services is legitimate or not? The unspoken premise of the person you're replying to is that this probably isn't feasible for AWS to catch any and all people abusing LightSail in this way, with the conclusion being that you can (in practice) probably get away with it unnoticed.
deleted 3y ago
[deleted]
rfoo 3y ago
Here's another one: You can download 1TB of data for free from AWS each month, as Cloudfront has a free tier [1] with 1TB monthly egress included. Point it to S3 or whatever HTTP server you want and voila. [1] It used to be 50GB per month for the first 12 months. It was changed to 1TB free forever shortly after Cloudflare posted https://blog.cloudflare.com/aws-egregious-egress https://blog.cloudflare.com/aws-egregious-egress
- overstay8930 3y ago
  That "shortly" was 2 years, that Cloudflare post had nothing to do with it, Amazon barely considers them a competitor to begin with.
  - rfoo 3y ago
    Sigh, I should have quoted both side, here is it: https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer-expansion-100-gb-from-regions-and-1-tb-from-amazon-cloudfront-per-month/ https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer... Cloudflare rant was posted in July 2021, new Cloudfront free tier was there Nov 2021. I consider changing pricing like this for AWS in 4 months pretty fast. Where is your 2 years number comes from?
andruby 3y ago
Nice! Nitpick: $5 for 2TB is better than $10 for 3TB.
- develatio 3y ago
  Ooohhh!! It is, indeed!
intelVISA 3y ago
Nice trick but you are playing with fire due to the AWS' terms.

jonatron 3y ago

https://aws.amazon.com/service-terms/ https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspend your data services or suspend your account.

develatio 3y ago
A had a suspicion that this was against AWS's terms, but I never bothered to look if that was actually the case. Thank you for the heads up!
- slashdev 3y ago
  It’s mostly in there to scare people into not doing it. AFAIK they’ve never taken action on that. Of course if you abuse it, you’re asking for trouble.
  - mdasen 3y ago
    As someone who has dealt with users who use a system in an unintended way, you don't go looking for those people and you don't build something to enforce a policy like this. When you're running services for lots of customers, you often don't know a lot of what's going on in the system and how people are using it. Then something seems weird or something is causing a problem and you want to deal with it - and you want the language out there so that you can deal with it. In Amazon's case, their bandwidth pricing isn't really defendable. It's just crap. However, sometimes you're trying to offer something reasonable, but need to make sure that a customer doesn't end up abusing something. For example, Chia is a cryptocurrency that will basically wear through SSDs (it's a proof-of-space system). There aren't explicit limits on how frequently you can write to a disk from most hosting providers, but Chia goes beyond what normal usage would do to a disk. Chia farmers would rather burn someone else's SSD that they're renting than their own. But no one at most hosting providers was probably looking at how frequently people were writing before noticing "hey, why are the disks failing faster than we'd expect?" They probably haven't taken action on it because they probably haven't noticed it being a problem. But if you're a whale of a customer and suddenly your data transfer charges drop off a cliff, someone might end up looking into that and seeing what's going on.
Hamuko 3y ago
At least AWS is fully aware how premium their normal data transfer is and that one might want to optimise those costs.
- vidarh 3y ago
  It's extreme enough that I never willingly serve data directly from AWS without a caching proxy elsewhere in front unless the egress is tint. It takes very low hitrates before it pays for itself several times over including management overheads. Sometimes you can justify a complete replica outside AWS (one of the things I will gladly pay AWS for is durability)
mmh0000 3y ago
Yeah, but "service terms" are just recommendations that should often be ignored.
sangnoir 3y ago
> You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services This requires proving the users intent, which is not obvious except in the most blatant of cases (i.e. using Lightsail as a bent-pipe by writing the exact bytes you're reading). If it is a "CSV to Parquet translation layer", how would AWS possibly prove it's anything other than what it claims to be? You'd be paying a few more cents for compute, but that's the price of plausible deniability
- greyface- 3y ago
  > This requires proving the users intent Companies are permitted to deny service to anyone at any time for any (non-protected) reason. They typically don't have to justify service terminations to a court of law. Who would they be required to prove user intent to, and why?
  - cstrahan 3y ago
    I don't think you parsed their message correctly. It's not about litigation. Re-posting a bit of the service terms for easy reference: > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services [...] As you point out, they may terminate your service without any justification in a court of law. So how do they go about terminating the offenders? Well, one trivial way (from a technology and/or policy perspective): terminate everyone's service! If you blindly terminate everyone's service, that will certainly prevent anyone abusing LightSail. But that's, uh, not good for business. So they probably want to terminate the service of only those people actually abusing it. But how do you do that? You'd have to look at each account's usage and do something to determine if that traffic is or isn't a means of avoiding data fees from other services. In other words, you'd have to determine the intent of that traffic. Or, put yet another way: "this requires proving the users intent". If doing so was as trivial as detecting any traffic between LightSail and the other services, they'd just prevent such connections in the first place. So how can AWS tell if some traffic between services is legitimate or not? The unspoken premise of the person you're replying to is that this probably isn't feasible for AWS to catch any and all people abusing LightSail in this way, with the conclusion being that you can (in practice) probably get away with it unnoticed.

deleted 3y ago

[deleted]

rfoo 3y ago

Here's another one: You can download 1TB of data for free from AWS each month, as Cloudfront has a free tier [1] with 1TB monthly egress included. Point it to S3 or whatever HTTP server you want and voila. [1] It used to be 50GB per month for the first 12 months. It was changed to 1TB free forever shortly after Cloudflare posted https://blog.cloudflare.com/aws-egregious-egress https://blog.cloudflare.com/aws-egregious-egress

overstay8930 3y ago
That "shortly" was 2 years, that Cloudflare post had nothing to do with it, Amazon barely considers them a competitor to begin with.
- rfoo 3y ago
  Sigh, I should have quoted both side, here is it: https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer-expansion-100-gb-from-regions-and-1-tb-from-amazon-cloudfront-per-month/ https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer... Cloudflare rant was posted in July 2021, new Cloudfront free tier was there Nov 2021. I consider changing pricing like this for AWS in 4 months pretty fast. Where is your 2 years number comes from?

andruby 3y ago

Nice! Nitpick: $5 for 2TB is better than $10 for 3TB.

develatio 3y ago
Ooohhh!! It is, indeed!

intelVISA 3y ago

Nice trick but you are playing with fire due to the AWS' terms.