10 ms·

That's a whole lot of text for "I had a 14 GiB VM image publicly linked and people discovered it", and none of it has much to do with AWS or Cloudflare. Presum

That's a whole lot of text for "I had a 14 GiB VM image publicly linked and people discovered it", and none of it has much to do with AWS or Cloudflare. Presumably the author would do much better with a VM or something from OVH, they'll just shut you off or limit you before it becomes a problem (not that they would care about 30 TiB).

Thaxll 6y ago
What's more surprising is that he works at Redhat on Kubernetes it seems, rookie mistake.
- majewsky 6y ago
  It actually makes perfect sense. As a sysadmin deploying applications on Kubernetes, you're used to have all the details of compute/network/storage hidden away from you. (Until shit hits the fan, that is.)
  - Thaxll 6y ago
    It's a linter in an IDE it's actually not type checking and it's not even close to what Go can do.
syshum 6y ago
I think this is less about the bill, and more about the customer service of the 2 companies, as well the "hidden" provisions in the ToS that are not clear in CloudFlare Marketing or normal setup docs.
- blahbhthrow3748 6y ago
  The limits aren't exactly hidden in the ToS, the author just wasn't aware of them. If you don't pay money for a service what kind of support can you expect on a holiday?
  - hn_check 6y ago
    Not only are they not hidden, they should really be common sense. That he seems surprised that Cloudflare doesn't cache 14GB files is surprising. There is some common belief that every service has infinite capacity in every dimension.
    - syrgian 6y ago
      To be fair, CloudFront's limit is 20GB.
      - hn_check 6y ago
        And the price for what he did would have been extreme. Serving a 14GB is something like $1.19 USD on CloudFront. It sounds like he was operating in tiny land and expected bills to be commensurate, so it just seems irrational to expect a low or free tier of another service to do this for nothing.
- robertlagrant 6y ago
  I don't know. I think their CloudFlare's pricing is pretty clear.
iforgotpassword 6y ago
I thought the same. This guy isn't even doing business via his site as far as I can tell. How is it that you set up such a monstrosity of services instead of renting a server for a couple bucks that comes with a decent amount of traffic per month and just caps your link or throttles if you go over it. But I guess simply running Debian on some server and keeping that up to date just isn't cool anymore. Everyone running a blog about their dog needs Cloudflare, S3, heroku, micro services and docker. Obviously with no limit on what kind of bill they will generate should something go wrong. You can just vent on twitter and generate enough attention that the vendor will make an exception for you and pay you back to limit the negative publicity. Who wouldn't prefer that to the tedious work that is maintaining a Linux install on a server?
- awinder 6y ago
  I mean the “monstrosity of services” is a bucket holding files and a CDN. It’s 2 services. I do this for my site (that gets no traffic and hasnt been updated in years) and it’s quite nice to not have to admin a server. Now that said it looks like his job is in cloud evangelism too so I’m sure a large part is that he wants to maintain a personal account to learn/hone skills. I’d recommend anyone do that, getting a look at aws without whatever your company is doing on top of it is pretty awesome for learning. Just don’t host massive S3 images and shoot yourself in the foot ;-)
  - andialo 6y ago
    What do you guys do when admining a server to make it so time-consuming/bothersome? If you just host static content you don't even need to care about security updates. Just deploy a new instance, "apt install nginx", done, forget about it. I would say less work than setting it all up on AWS. When is the last time there was an remote exploit for nginx serving static files or in Linux kernel? You dont even need to worry about security updates in this scenario. > I’m sure a large part is that he wants to maintain a personal account to learn/hone skills. Yeah, he seems to be responsible for OpenShift (marketing?) at Red Hat and he only learnt how expensive AWS is when he got a bill? Bit embarassing.
8organicbits 6y ago
> I made a poor decision to distribute a trial Windows 2019 SQL Server virtual machine images (fully patched with all necessary drivers and VM extensions) in the form of a qcow2 file. The article doesn't use the term "piracy", but I'm curious what Microsoft's license says about public redistribution.
- cuchoi 6y ago
  Is it considered piracy to distribute a free trial?
  - 8organicbits 6y ago
    I'm not a lawyer, but generally a software license can set limits on redistribution. Redistribution outside the terms of the license is likely copyright infringement. Some sources define piracy as redistribution outside the terms of the license. That implies that you can pirate free software, if you violate the license. Without getting bogged down on definitions, I believe MS could issue takedown requests to anyone hosting their free trials, when the license forbids it.
  - deleted 6y ago
    [deleted]
- cerberusss 6y ago
  It would perhaps legally be more acceptable, and more efficient in bandwidth, to create a binary patch that makes an official virtual machine image into what the author offered ("fully patched with all necessary drivers and VM extensions").
TAForObvReasons 6y ago
> The maximum file size Cloudflare's CDN caches is 512MB for Free, Pro, and Business customers and 5GB for Enterprise customers. What's not stated here, and the OP learned, is that requests for larger files are always passed through directly. https://support.cloudflare.com/hc/en-us/articles/200172516-Understanding-Cloudflare-s-CDN#:~:text=The%20maximum%20file%20size%20Cloudflare's,request%20caching%20of%20larger%20files https://support.cloudflare.com/hc/en-us/articles/200172516-U....
- ivalm 6y ago
  If it doesn’t hit a cache then it should be passed on... that seems to be the basic premise of how caches work.
  - GordonS 6y ago
    I agree with this - Cloudflare's pricing and function is clear here, and it seems perfectly reasonable and expected to have a cutoff size that they don't cache at the edge. I get the OP was pissed about a big bill and felt like lashing out, but it's unfair to complain about Cloudflare when they've done nothing wrong.
    - Can_Not 6y ago
      CF's size limit is the first thing I checked when considering them.
  - ec109685 6y ago
    Is the file size known as the request starts?
    - afwaller 6y ago
      Anything the cache doesn’t have, causes a cache miss, which passes it through. So when you request the large file, the CDN (cache) doesn’t have it, so it immediately passes through to the original source. The details of how this is implemented don’t really matter in the sense that they are not going to host the large file(s) in the CDN, so they will always fall through.
fxtentacle 6y ago
I would say the main story here is that AWS overcharges on traffic to such a ridiculous degree. My SaaS regularly consumes 50+ TB in traffic, but I pay €150 in monthly fees for that. For someone coming from my perspective, that would be a huge and unpleasant surprise to be billed $2600 for a service that I know costs $180 elsewhere. Edit: especially so because people keep repeating the mantra that one should use the cloud to save money by not paying for what you don't use. Obviously, my 3 Xeon servers with 64 GB RAM each are way overpowered for sending out some GB of static files, but I wanted to have a bit of redundancy. But with my setup, there should be plenty of obvious inefficiencies for "the cloud" to eliminate => It feels like cloud should be cheaper than my dedicated servers. But it's not, and that is the unpleasant surprise.
- Malcx 6y ago
  Out of interest who are your servers with? That seems a good price and might be something I'm interested in...
  - thg 6y ago
    Not the OP, but it could be Hetzner. They're in that price range, with that kind of hardware. I myself have been a happy customer there for almost 20 years now.
    - fxtentacle 6y ago
      OP here, yes I use the hetzner marketplace for used dedicated servers. I have found that using hardware that's 6+ months old actually decreased failure rate. Plus it's cheaper and they waive the setup fee.
      - Malcx 6y ago
        The marketplace has some really good deals on it! Thanks for sharing.
    - dvfjsdhgfv 6y ago
spottybanana 6y ago
> Presumably the author would do much better with a VM or something from OVH, they'll just shut you off or limit you before it becomes a problem (not that they would care about 30 TiB). That's actually what I love about these dedicated providers. Often I prefer to be surprised by cutting the service instead of a gigantic invoice. For some business-critical applications I can understand the need to have it scale (and price) accordingly, for many other use cases it is better just to have the serve switched out when the traffic limit is hit off.
mikl 6y ago
Yep, the author publicly shared a 14GiB file from his own S3 bucket. AWS would be completely justified in charging him for it. He can just thank his lucky stars that they decided to let it slide. As for Cloudflare, hosting multi-gigabyte files is not what their service is for, I can’t see how you can blame them for having a limit on how large files they cache.
- andialo 6y ago
  Reading about this, this looks like a potential new take on DDoS attacks. AWS charge attack. You survived the DDoS, that's great, now go see your bill.
tgsovlerkhgsel 6y ago
> That's a whole lot of text for "I had a 14 GiB VM image publicly linked and people discovered it" I really hate long-form articles that feel the need to explain the weather, someone's clothing, or their family member's eating habits. COME TO THE DAMN POINT. This is the Internet. 99% of content is garbage, and I'm not interested reading through pointless content-free filler that could be generated by a neural network just to find out if your article somewhere contains useful/interesting information.
m3kw9 6y ago
I thought it wasn’t that, and it was a combination of cloud flare didn’t cache his 13gig file and also this 3555 error bug that he didn’t seem to really explain?
- _msw_ 6y ago
  Disclosure: I work at AWS where I build cloud infrastructure My hypothesis is that the client connected to Cloudflare and performed a HTTP range request for a portion of the 13.7 GB file. For an unknown reason, Cloudflare did not preserve this range request to S3 as the origin. It transferred the entire file, returned the range requested by the client, and dropped all bytes transferred into /dev/null because it is not caching. The end result is that Cloudflare pulled down 30 TB of data while delivering 67 GB to clients in the one month period shown in the screenshots from the blog post. See also https://blog.vbgn.be/2019/06/20/nextcloud-cloudflare.html https://blog.vbgn.be/2019/06/20/nextcloud-cloudflare.html
dilandau 6y ago
No fucking shit, came here to say: If you literally put a massive file on the public internet, what the actual fuck are you complaining about? The quality here is such shit. Let's snipe at amzn, everyone hates them because they're winning, because of hn readers dependence on AWS to run their shit web apps... It's like: ope! It's been a day since we had a "amzn scammed me" post. Nevermind that the people complaining are either a) too ignorant and footgunned themselves, b) minimizing or lying about their own complicity, or c) outright scammers themselves.

Thaxll 6y ago

What's more surprising is that he works at Redhat on Kubernetes it seems, rookie mistake.

majewsky 6y ago
It actually makes perfect sense. As a sysadmin deploying applications on Kubernetes, you're used to have all the details of compute/network/storage hidden away from you. (Until shit hits the fan, that is.)
- Thaxll 6y ago
  It's a linter in an IDE it's actually not type checking and it's not even close to what Go can do.

syshum 6y ago

I think this is less about the bill, and more about the customer service of the 2 companies, as well the "hidden" provisions in the ToS that are not clear in CloudFlare Marketing or normal setup docs.

blahbhthrow3748 6y ago
The limits aren't exactly hidden in the ToS, the author just wasn't aware of them. If you don't pay money for a service what kind of support can you expect on a holiday?
- hn_check 6y ago
  Not only are they not hidden, they should really be common sense. That he seems surprised that Cloudflare doesn't cache 14GB files is surprising. There is some common belief that every service has infinite capacity in every dimension.
  - syrgian 6y ago
    To be fair, CloudFront's limit is 20GB.
    - hn_check 6y ago
      And the price for what he did would have been extreme. Serving a 14GB is something like $1.19 USD on CloudFront. It sounds like he was operating in tiny land and expected bills to be commensurate, so it just seems irrational to expect a low or free tier of another service to do this for nothing.
robertlagrant 6y ago
I don't know. I think their CloudFlare's pricing is pretty clear.

iforgotpassword 6y ago

I thought the same. This guy isn't even doing business via his site as far as I can tell. How is it that you set up such a monstrosity of services instead of renting a server for a couple bucks that comes with a decent amount of traffic per month and just caps your link or throttles if you go over it. But I guess simply running Debian on some server and keeping that up to date just isn't cool anymore. Everyone running a blog about their dog needs Cloudflare, S3, heroku, micro services and docker. Obviously with no limit on what kind of bill they will generate should something go wrong. You can just vent on twitter and generate enough attention that the vendor will make an exception for you and pay you back to limit the negative publicity. Who wouldn't prefer that to the tedious work that is maintaining a Linux install on a server?

awinder 6y ago
I mean the “monstrosity of services” is a bucket holding files and a CDN. It’s 2 services. I do this for my site (that gets no traffic and hasnt been updated in years) and it’s quite nice to not have to admin a server. Now that said it looks like his job is in cloud evangelism too so I’m sure a large part is that he wants to maintain a personal account to learn/hone skills. I’d recommend anyone do that, getting a look at aws without whatever your company is doing on top of it is pretty awesome for learning. Just don’t host massive S3 images and shoot yourself in the foot ;-)
- andialo 6y ago
  What do you guys do when admining a server to make it so time-consuming/bothersome? If you just host static content you don't even need to care about security updates. Just deploy a new instance, "apt install nginx", done, forget about it. I would say less work than setting it all up on AWS. When is the last time there was an remote exploit for nginx serving static files or in Linux kernel? You dont even need to worry about security updates in this scenario. > I’m sure a large part is that he wants to maintain a personal account to learn/hone skills. Yeah, he seems to be responsible for OpenShift (marketing?) at Red Hat and he only learnt how expensive AWS is when he got a bill? Bit embarassing.

8organicbits 6y ago

> I made a poor decision to distribute a trial Windows 2019 SQL Server virtual machine images (fully patched with all necessary drivers and VM extensions) in the form of a qcow2 file. The article doesn't use the term "piracy", but I'm curious what Microsoft's license says about public redistribution.

cuchoi 6y ago
Is it considered piracy to distribute a free trial?
- 8organicbits 6y ago
  I'm not a lawyer, but generally a software license can set limits on redistribution. Redistribution outside the terms of the license is likely copyright infringement. Some sources define piracy as redistribution outside the terms of the license. That implies that you can pirate free software, if you violate the license. Without getting bogged down on definitions, I believe MS could issue takedown requests to anyone hosting their free trials, when the license forbids it.
- deleted 6y ago
  [deleted]
cerberusss 6y ago
It would perhaps legally be more acceptable, and more efficient in bandwidth, to create a binary patch that makes an official virtual machine image into what the author offered ("fully patched with all necessary drivers and VM extensions").

TAForObvReasons 6y ago

> The maximum file size Cloudflare's CDN caches is 512MB for Free, Pro, and Business customers and 5GB for Enterprise customers. What's not stated here, and the OP learned, is that requests for larger files are always passed through directly. https://support.cloudflare.com/hc/en-us/articles/200172516-Understanding-Cloudflare-s-CDN#:~:text=The%20maximum%20file%20size%20Cloudflare's,request%20caching%20of%20larger%20files https://support.cloudflare.com/hc/en-us/articles/200172516-U....

ivalm 6y ago
If it doesn’t hit a cache then it should be passed on... that seems to be the basic premise of how caches work.
- GordonS 6y ago
  I agree with this - Cloudflare's pricing and function is clear here, and it seems perfectly reasonable and expected to have a cutoff size that they don't cache at the edge. I get the OP was pissed about a big bill and felt like lashing out, but it's unfair to complain about Cloudflare when they've done nothing wrong.
  - Can_Not 6y ago
    CF's size limit is the first thing I checked when considering them.
- ec109685 6y ago
  Is the file size known as the request starts?
  - afwaller 6y ago
    Anything the cache doesn’t have, causes a cache miss, which passes it through. So when you request the large file, the CDN (cache) doesn’t have it, so it immediately passes through to the original source. The details of how this is implemented don’t really matter in the sense that they are not going to host the large file(s) in the CDN, so they will always fall through.

fxtentacle 6y ago

I would say the main story here is that AWS overcharges on traffic to such a ridiculous degree. My SaaS regularly consumes 50+ TB in traffic, but I pay €150 in monthly fees for that. For someone coming from my perspective, that would be a huge and unpleasant surprise to be billed $2600 for a service that I know costs $180 elsewhere. Edit: especially so because people keep repeating the mantra that one should use the cloud to save money by not paying for what you don't use. Obviously, my 3 Xeon servers with 64 GB RAM each are way overpowered for sending out some GB of static files, but I wanted to have a bit of redundancy. But with my setup, there should be plenty of obvious inefficiencies for "the cloud" to eliminate => It feels like cloud should be cheaper than my dedicated servers. But it's not, and that is the unpleasant surprise.

Malcx 6y ago
Out of interest who are your servers with? That seems a good price and might be something I'm interested in...
- thg 6y ago
  Not the OP, but it could be Hetzner. They're in that price range, with that kind of hardware. I myself have been a happy customer there for almost 20 years now.
  - fxtentacle 6y ago
    OP here, yes I use the hetzner marketplace for used dedicated servers. I have found that using hardware that's 6+ months old actually decreased failure rate. Plus it's cheaper and they waive the setup fee.
    - Malcx 6y ago
      The marketplace has some really good deals on it! Thanks for sharing.
  - dvfjsdhgfv 6y ago

spottybanana 6y ago

> Presumably the author would do much better with a VM or something from OVH, they'll just shut you off or limit you before it becomes a problem (not that they would care about 30 TiB). That's actually what I love about these dedicated providers. Often I prefer to be surprised by cutting the service instead of a gigantic invoice. For some business-critical applications I can understand the need to have it scale (and price) accordingly, for many other use cases it is better just to have the serve switched out when the traffic limit is hit off.

mikl 6y ago

Yep, the author publicly shared a 14GiB file from his own S3 bucket. AWS would be completely justified in charging him for it. He can just thank his lucky stars that they decided to let it slide. As for Cloudflare, hosting multi-gigabyte files is not what their service is for, I can’t see how you can blame them for having a limit on how large files they cache.

andialo 6y ago
Reading about this, this looks like a potential new take on DDoS attacks. AWS charge attack. You survived the DDoS, that's great, now go see your bill.

tgsovlerkhgsel 6y ago

> That's a whole lot of text for "I had a 14 GiB VM image publicly linked and people discovered it" I really hate long-form articles that feel the need to explain the weather, someone's clothing, or their family member's eating habits. COME TO THE DAMN POINT. This is the Internet. 99% of content is garbage, and I'm not interested reading through pointless content-free filler that could be generated by a neural network just to find out if your article somewhere contains useful/interesting information.

m3kw9 6y ago

I thought it wasn’t that, and it was a combination of cloud flare didn’t cache his 13gig file and also this 3555 error bug that he didn’t seem to really explain?

_msw_ 6y ago
Disclosure: I work at AWS where I build cloud infrastructure My hypothesis is that the client connected to Cloudflare and performed a HTTP range request for a portion of the 13.7 GB file. For an unknown reason, Cloudflare did not preserve this range request to S3 as the origin. It transferred the entire file, returned the range requested by the client, and dropped all bytes transferred into /dev/null because it is not caching. The end result is that Cloudflare pulled down 30 TB of data while delivering 67 GB to clients in the one month period shown in the screenshots from the blog post. See also https://blog.vbgn.be/2019/06/20/nextcloud-cloudflare.html https://blog.vbgn.be/2019/06/20/nextcloud-cloudflare.html

dilandau 6y ago

No fucking shit, came here to say: If you literally put a massive file on the public internet, what the actual fuck are you complaining about? The quality here is such shit. Let's snipe at amzn, everyone hates them because they're winning, because of hn readers dependence on AWS to run their shit web apps... It's like: ope! It's been a day since we had a "amzn scammed me" post. Nevermind that the people complaining are either a) too ignorant and footgunned themselves, b) minimizing or lying about their own complicity, or c) outright scammers themselves.