8 ms·

Image hoster pushes 1.8 PB per month through Cloudflare CDN

71 points by sheraz 10y ago

Namrog84 10y ago
Cloudflare claims unlimited bandwidth then says you are using too much bandwidth? Isn't this one of those blatant false advertising claims? Also it's super unfortunate if they go down breaking so many images hosted so many places.
- mdasen 10y ago
  Cloudflare's terms of service specifically deal with this: "Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited." If they put their website behind Cloudflare, but not their image serving domain, presumably that would be fine. Cloudflare is made for a site that serves up web pages with some content like your logo, a few images, etc. Those images can be cached and it's a small amount of data since something like the YC "Y" image is the same served for everyone on HN. A site like Postimage serves image content that's hard to cache since it's mostly different for each visitor. Cloudflare is pretty explicit that it's not meant for a site that will be mostly serving up loads of pictures.
  - dismantlethesun 10y ago
    Postimage is an image hoster. 20 million of their 28 million unique monthy visitors are viewing images on pages they do not control. This means the majority of their bandwidth is going to people whom: a) Cannot be monetized via ads b) Are not on Postimage.com thus the site is breaking Section 10 of Cloudflare's services.
  - derefr 10y ago
    > Cloudflare is made for a site that serves up web pages with some content like your logo, a few images, etc. For an interesting counter-example: 4chan (including its image-CDN subdomain!) is served through CloudFlare. It seems that CloudFlare really is willing to host and distribute multiple petabytes of image content per month, as long as it's for the sake of something else (e.g. pages of threaded conversation) that's not an "online storage space." Though, I think in 4chan's case it might also help that pages and their image resources both get expired off of the site quite quickly. You can create a page that hotlinks image URLs from 4chan's CDN subdomain, but the correspondent resources at those URLs won't be there five hours later, so there's little point to doing so. Unlike most image-hosts one could name.
ChrisNorstrom 10y ago
Yeah unlimited does have it's limita....OMG you're funneling 1.8 petabytes?!?!! That's like 1,800,000 gigabytes or 1,800 terabytes every month. EVERY MONTH. Unlimited does not mean you can serve the bandwidth of a small country through 1 ISP. "Please contact us if you have a CDN that is capable and willing of serving 1.8 Petabytes of outgoing traffic per month free of charge," -PostImage Is the company run by spoiled entitled millennials? It takes a lot of equipment and staff to create a network that can serve that and they just want it all for under $1k? You know data centers have costs too. Other startups spend years of their lives putting together a team that can assemble infrastructure to handle that and they want it for free? For their barely-cash-flow-positive project that doesn't earn them a decent living... I can't believe they just asked that. Sorry but PostImage sounds like the biggest moocher in the web right now. If your ISP provider says "unlimited data" does that mean you can funnel the traffic of Russia through your internet connection? You want for "free" what others are paying thousands and thousands of dollars for? What is wrong with you? Seriously? It's like you're trying to run a top-tier web company on pocket change.
- spiderfarmer 10y ago
  I agree, but to be fair, I think "unlimited anything" is almost always misleading, Why not advertise with a generous free tier with clear limits?
  - SallySwanSmith 10y ago
    Because if it was static human readable text content, they'd let you push that much though just fine. It's unlikely that you ever would get to that level, but they don't mind if you do.
  - marktangotango 10y ago
    Services offer this because if you don't have any users, it's easier than building in and enforcing limits. If your service gets traction then it makes sense to spend the time and money on development, not before. I personally wasted a lot of time on building limits for a service that has yet to gain traction.
  - 10y ago
deleted 10y ago
[deleted]
mark242 10y ago
Equivalent pricing: Amazon Cloudfront: $59770 per month Google Cloud CDN: $50000 per month Fastly: $144200 per month etc etc. If they can't get together $12000 per month for Cloudflare, they sure as hell don't have a viable business model.
- spiderfarmer 10y ago
  I can't believe they never researched CDN prices beforehand.
  - mkroman 10y ago
    It probably started as a hobby project to fill a need and grew way more than they expected it to.
    - idp 10y ago
      The project is now certainly larger than it was expected to become, although this doesn't absolve the team of not taking preemptive action sooner. Disclaimer: I'm with the team.
- sheraz 10y ago
  Yeah I priced Azure as well, and it clocked in at just over $60k USD
- zzzcpan 10y ago
  Or like $1000-$1500 per month for a dedicated server with 10G unlimited bandwidth. This is as low as it gets.
  - Artemis2 10y ago
    Would one server be enough for their load though?
  - mark242 10y ago
    Could you run the service off a few OVH dedicated boxes with 3Gbps? Sure, you could, but it's going to be painful, and at that point imgur is the better choice anyways.
codexon 10y ago
The solution is really simple, don't host images through a CDN. Though it might not be as fast, it will be way cheaper. I'm really suprised they didn't look at all the stories of people before trying to start image or video hosts on Cloudflare and getting booted, as if all the other image hosts were too stupid to use Cloudflare for $200/month instead of paying CDNs tens of thousands a month.
- idp 10y ago
  Of course we were puzzled by CloudFlare's pricing structure. However, all of their advertising material kept reiterating that they were making money off low-tier customers by collecting lots of data required to properly serve their business and enterprise customers. So our sense of alarm and disbelief was somewhat suspended until CF contacted us about our bandwidth consumption, but even then we didn't realize how much trouble we were facing, as they first told us that a $1k subscription would do the trick. Watching competing projects also added some degree of assurance. Without giving out any specific names, we are aware of a number of competing projects that even now keep heavily relying on CloudFlare (although I am obviously unaware of how much they pay), so we didn't consider it an abusive behavior. In contrast, we were also aware of at least one competitor that kept pushing a considerable portion of their uploads to imgur and basically parasitizing on them for a long time until imgur's abuse team finally took notice and brought down the hammer.
  - jgrahamc 10y ago
    However, all of their advertising material kept reiterating that they were making money off low-tier customers by collecting lots of data required to properly serve their business and enterprise customers. This makes it sound like we are doing some monetization of low-tier customer data. We are not doing that with any data (low-tier or anything else). That would be hella creepy. We do track abuse (DDoS attacks, etc.) whoever they hit and use that information to protect other customers. So, in that sense low-tier customers help our overall business, but it's a common lie from our competitors that we are somehow monetizing traffic. Cloudflare's business is pretty simple: work out how to operate our services as cheaply as possible, charge web site/application owners more than that amount.
LusoTycoon 10y ago
Building your own CDN is not an option? Something small only to reign in the costs
- codexon 10y ago
  Building your own CDN these days requires you to own a large ip block and ASN, and colo at several locations so the DC will actually bother routing to you, in order to deploy anycast. Certainly too expensive for a small deployment which is why people buy from CDNs instead of setting up their own. If it was as easy as just doing dns geolocation (which is awful due to geolocation failing and ISP caching), few people would bother buying from CDNs.
  - zzzcpan 10y ago
    > If it was as easy as just doing dns geolocation It is that easy. The importance of anycast is just PR, nothing more. Some very large CDNs use DNS just fine.
  - toomuchtodo 10y ago
    You shouldn't be anycasting TCP, only UDP. DNS is sufficient.
    - codexon 10y ago
      If anycast TCP didn't work, then cloudflare wouldn't exist. DNS is not sufficient. It takes forever to handle downtime because ISPs don't respect TTL.
      - zzzcpan 10y ago
        You are incorrect, I'll try to explain. Anycast is essentially an SPOF. Well, not only anycast and not anycast per se, but a single AS it is under. It breaks from time to time because of various mistakes, bugs, etc. and brings down every server as a consequence. This occurs roughly every couple of years and takes hours to resolve. So, with anycast, if you have 10 servers in different places, you get hours of downtime for 100% of users from time to time. With DNS, on the other hand, if one server goes down, it affects only 1% or so of users of a particular server, that have incorrect TTL in the resolvers they use, others see change in DNS right away and use working server. But, those 1% of users don't all go to that server at the same time, only small percentage of them does and also sees the old record. Leaving us with let's say 10% of that 1% on 1 out of 10 servers, or 0.01% of all users unable to see the new DNS record for an hour or so. If a typical server on some random AS goes down five times a year, you get 0.01% * 5 * 10 or 0.5% of users affected for an hour per year. Now if you use round robin and let users see multiple records nothing is even going to stop working in the browser for them, just going to make them wait longer until they see a set of working records. To summarize, anycast is 100% of users not able to reach any server for hours every couple of years, while DNS is 0.5% of users experiencing slowness for an hour per year. In other words: anycast alone cannot be reliable enough for a CDN.
_ursu 10y ago
1.8PB/month is an average of 5.48Gb/s. Assuming you can make do with peak capacity of ~4x average, you can buy 20Gb/s of transit for ~$2k (these guys for example https://www.fdcservers.net/ip-transit.php https://www.fdcservers.net/ip-transit.php). You can probably get a decent deal from somewhere more well known if you ask for a quote, with that kind of traffic, especially if it get a little publicity.
- dismantlethesun 10y ago
  You'd get something, but it won't be a CloudFlare level CDN. At best, you can get AdvanceHosters. They're Russian based but have 7 CDN pops across the US and Western Europe, they'll let you push 1.2PB for 7500 $/month You won't get asia/oceana/australia traffic for those prices, and you certainly won't get S. American of S. African traffic for those prices.
winteriscoming 10y ago
Never heard of them before but apparently they have been around since 2004 and they support 450k websites and their only revenue model is advertising. I doubt they had a solid technical and business plan behind this service.
bluedino 10y ago
Never heard of them. Does imgur say how much traffic they have per month?

71 points by sheraz 10y ago

Cloudflare claims unlimited bandwidth then says you are using too much bandwidth? Isn't this one of those blatant false advertising claims? Also it's super unfortunate if they go down breaking so many images hosted so many places.

mdasen 10y ago
Cloudflare's terms of service specifically deal with this: "Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited." If they put their website behind Cloudflare, but not their image serving domain, presumably that would be fine. Cloudflare is made for a site that serves up web pages with some content like your logo, a few images, etc. Those images can be cached and it's a small amount of data since something like the YC "Y" image is the same served for everyone on HN. A site like Postimage serves image content that's hard to cache since it's mostly different for each visitor. Cloudflare is pretty explicit that it's not meant for a site that will be mostly serving up loads of pictures.
- dismantlethesun 10y ago
  Postimage is an image hoster. 20 million of their 28 million unique monthy visitors are viewing images on pages they do not control. This means the majority of their bandwidth is going to people whom: a) Cannot be monetized via ads b) Are not on Postimage.com thus the site is breaking Section 10 of Cloudflare's services.
- derefr 10y ago
  > Cloudflare is made for a site that serves up web pages with some content like your logo, a few images, etc. For an interesting counter-example: 4chan (including its image-CDN subdomain!) is served through CloudFlare. It seems that CloudFlare really is willing to host and distribute multiple petabytes of image content per month, as long as it's for the sake of something else (e.g. pages of threaded conversation) that's not an "online storage space." Though, I think in 4chan's case it might also help that pages and their image resources both get expired off of the site quite quickly. You can create a page that hotlinks image URLs from 4chan's CDN subdomain, but the correspondent resources at those URLs won't be there five hours later, so there's little point to doing so. Unlike most image-hosts one could name.

ChrisNorstrom 10y ago

Yeah unlimited does have it's limita....OMG you're funneling 1.8 petabytes?!?!! That's like 1,800,000 gigabytes or 1,800 terabytes every month. EVERY MONTH. Unlimited does not mean you can serve the bandwidth of a small country through 1 ISP. "Please contact us if you have a CDN that is capable and willing of serving 1.8 Petabytes of outgoing traffic per month free of charge," -PostImage Is the company run by spoiled entitled millennials? It takes a lot of equipment and staff to create a network that can serve that and they just want it all for under $1k? You know data centers have costs too. Other startups spend years of their lives putting together a team that can assemble infrastructure to handle that and they want it for free? For their barely-cash-flow-positive project that doesn't earn them a decent living... I can't believe they just asked that. Sorry but PostImage sounds like the biggest moocher in the web right now. If your ISP provider says "unlimited data" does that mean you can funnel the traffic of Russia through your internet connection? You want for "free" what others are paying thousands and thousands of dollars for? What is wrong with you? Seriously? It's like you're trying to run a top-tier web company on pocket change.

spiderfarmer 10y ago
I agree, but to be fair, I think "unlimited anything" is almost always misleading, Why not advertise with a generous free tier with clear limits?
- SallySwanSmith 10y ago
  Because if it was static human readable text content, they'd let you push that much though just fine. It's unlikely that you ever would get to that level, but they don't mind if you do.
- marktangotango 10y ago
  Services offer this because if you don't have any users, it's easier than building in and enforcing limits. If your service gets traction then it makes sense to spend the time and money on development, not before. I personally wasted a lot of time on building limits for a service that has yet to gain traction.
- 10y ago

deleted 10y ago

[deleted]

mark242 10y ago

Equivalent pricing: Amazon Cloudfront: $59770 per month Google Cloud CDN: $50000 per month Fastly: $144200 per month etc etc. If they can't get together $12000 per month for Cloudflare, they sure as hell don't have a viable business model.

spiderfarmer 10y ago
I can't believe they never researched CDN prices beforehand.
- mkroman 10y ago
  It probably started as a hobby project to fill a need and grew way more than they expected it to.
  - idp 10y ago
    The project is now certainly larger than it was expected to become, although this doesn't absolve the team of not taking preemptive action sooner. Disclaimer: I'm with the team.
sheraz 10y ago
Yeah I priced Azure as well, and it clocked in at just over $60k USD
zzzcpan 10y ago
Or like $1000-$1500 per month for a dedicated server with 10G unlimited bandwidth. This is as low as it gets.
- Artemis2 10y ago
  Would one server be enough for their load though?
- mark242 10y ago
  Could you run the service off a few OVH dedicated boxes with 3Gbps? Sure, you could, but it's going to be painful, and at that point imgur is the better choice anyways.

codexon 10y ago

The solution is really simple, don't host images through a CDN. Though it might not be as fast, it will be way cheaper. I'm really suprised they didn't look at all the stories of people before trying to start image or video hosts on Cloudflare and getting booted, as if all the other image hosts were too stupid to use Cloudflare for $200/month instead of paying CDNs tens of thousands a month.

idp 10y ago
Of course we were puzzled by CloudFlare's pricing structure. However, all of their advertising material kept reiterating that they were making money off low-tier customers by collecting lots of data required to properly serve their business and enterprise customers. So our sense of alarm and disbelief was somewhat suspended until CF contacted us about our bandwidth consumption, but even then we didn't realize how much trouble we were facing, as they first told us that a $1k subscription would do the trick. Watching competing projects also added some degree of assurance. Without giving out any specific names, we are aware of a number of competing projects that even now keep heavily relying on CloudFlare (although I am obviously unaware of how much they pay), so we didn't consider it an abusive behavior. In contrast, we were also aware of at least one competitor that kept pushing a considerable portion of their uploads to imgur and basically parasitizing on them for a long time until imgur's abuse team finally took notice and brought down the hammer.
- jgrahamc 10y ago
  However, all of their advertising material kept reiterating that they were making money off low-tier customers by collecting lots of data required to properly serve their business and enterprise customers. This makes it sound like we are doing some monetization of low-tier customer data. We are not doing that with any data (low-tier or anything else). That would be hella creepy. We do track abuse (DDoS attacks, etc.) whoever they hit and use that information to protect other customers. So, in that sense low-tier customers help our overall business, but it's a common lie from our competitors that we are somehow monetizing traffic. Cloudflare's business is pretty simple: work out how to operate our services as cheaply as possible, charge web site/application owners more than that amount.

LusoTycoon 10y ago

Building your own CDN is not an option? Something small only to reign in the costs

codexon 10y ago
Building your own CDN these days requires you to own a large ip block and ASN, and colo at several locations so the DC will actually bother routing to you, in order to deploy anycast. Certainly too expensive for a small deployment which is why people buy from CDNs instead of setting up their own. If it was as easy as just doing dns geolocation (which is awful due to geolocation failing and ISP caching), few people would bother buying from CDNs.
- zzzcpan 10y ago
  > If it was as easy as just doing dns geolocation It is that easy. The importance of anycast is just PR, nothing more. Some very large CDNs use DNS just fine.
- toomuchtodo 10y ago
  You shouldn't be anycasting TCP, only UDP. DNS is sufficient.
  - codexon 10y ago
    If anycast TCP didn't work, then cloudflare wouldn't exist. DNS is not sufficient. It takes forever to handle downtime because ISPs don't respect TTL.
    - zzzcpan 10y ago
      You are incorrect, I'll try to explain. Anycast is essentially an SPOF. Well, not only anycast and not anycast per se, but a single AS it is under. It breaks from time to time because of various mistakes, bugs, etc. and brings down every server as a consequence. This occurs roughly every couple of years and takes hours to resolve. So, with anycast, if you have 10 servers in different places, you get hours of downtime for 100% of users from time to time. With DNS, on the other hand, if one server goes down, it affects only 1% or so of users of a particular server, that have incorrect TTL in the resolvers they use, others see change in DNS right away and use working server. But, those 1% of users don't all go to that server at the same time, only small percentage of them does and also sees the old record. Leaving us with let's say 10% of that 1% on 1 out of 10 servers, or 0.01% of all users unable to see the new DNS record for an hour or so. If a typical server on some random AS goes down five times a year, you get 0.01% * 5 * 10 or 0.5% of users affected for an hour per year. Now if you use round robin and let users see multiple records nothing is even going to stop working in the browser for them, just going to make them wait longer until they see a set of working records. To summarize, anycast is 100% of users not able to reach any server for hours every couple of years, while DNS is 0.5% of users experiencing slowness for an hour per year. In other words: anycast alone cannot be reliable enough for a CDN.

_ursu 10y ago

1.8PB/month is an average of 5.48Gb/s. Assuming you can make do with peak capacity of ~4x average, you can buy 20Gb/s of transit for ~$2k (these guys for example https://www.fdcservers.net/ip-transit.php https://www.fdcservers.net/ip-transit.php). You can probably get a decent deal from somewhere more well known if you ask for a quote, with that kind of traffic, especially if it get a little publicity.

dismantlethesun 10y ago
You'd get something, but it won't be a CloudFlare level CDN. At best, you can get AdvanceHosters. They're Russian based but have 7 CDN pops across the US and Western Europe, they'll let you push 1.2PB for 7500 $/month You won't get asia/oceana/australia traffic for those prices, and you certainly won't get S. American of S. African traffic for those prices.

winteriscoming 10y ago

Never heard of them before but apparently they have been around since 2004 and they support 450k websites and their only revenue model is advertising. I doubt they had a solid technical and business plan behind this service.

bluedino 10y ago

Never heard of them. Does imgur say how much traffic they have per month?