3 ms·

There is no way this figure is accurate. The annual spend cited of $1,000,000 is purely hypothetical, as admitted here: "However, all this data and processes n

There is no way this figure is accurate. The annual spend cited of $1,000,000 is purely hypothetical, as admitted here: "However, all this data and processes need to happen on a server and, of course, we used AWS for it. A few years of growth later, we’re handling over 70,000 pages per minute, storing around 560 million pages, and paying well over $1,000,000 per year. Or at least we would be paying that much if we stayed with AWS. Instead, we were able to cut costs by 80% in a little over three months with some out-of-the-box thinking and a clear plan."

pessimizer 4y ago
> There is no way this figure is accurate. You've got the FUD covered, but you also need to add at least some substance to your claim. How do you know this figure would not be accurate? Why is your (hypothetical, not offered) estimate better than the author's?
- merb 4y ago
  well the problem with the article is basically that they left out a lot of important detail, like which bare metal servers, how many, where do they host now, did they use cloudfront what about cloudflare did they use an edge cache? what about reducing costs by killing stopping unneeded resources? a lot of their workload looks dynamic. it's also fishy what they wrote here: > After testing whether Prerender pages could be cached in both S3 and minio, we slowly diverted traffic away from AWS S3 and towards minio. if they served directly from s3 that would be, stupid? > In the last four weeks, we moved most of the cache workload from AWS S3 to our own Cassandra cluster. is also strange. it misses a lot of detail but it does not look like they just migrated away from s3... (looks like their new hoster is hetzner, from service.prerender.io )
  - pessimizer 4y ago
    > well the problem with the article is basically that they left out a lot of important detail, The problem with your comment is that you insisted that they gave you enough detail to definitively determine that they were either lying or mistaken. > There is no way this figure is accurate.
- hazmazlaz 4y ago
  Just as a basic sanity check, they claim to store 560 million pages of pure HTML. Internet Archive estimates the median HTML weight of a page to be ~20kb, which I fudged to 30kb in order to make a liberal estimate. Multiplied by the number of pages that would amount to approximately 16.8TB of data stored in s3. According to the AWS pricing calculator that would cost between $300-400 per month, and that's in s3 standard tier pricing with no intelligent tiering set up for less often retrieved data. I didn't bother calculating the compute cost for their rendering because I don't really understand how they do it, and also because the storage estimate was so far off of their claim that it didn't seem necessary to bother in order to refute the claim. I appreciate your challenge for additional evidence, and this is what I have to offer in response. I won't pretend like I did an exhaustive scientific study, this is more back of the napkin type stuff FWIW, but here it is.

pessimizer 4y ago

> There is no way this figure is accurate. You've got the FUD covered, but you also need to add at least some substance to your claim. How do you know this figure would not be accurate? Why is your (hypothetical, not offered) estimate better than the author's?

merb 4y ago
well the problem with the article is basically that they left out a lot of important detail, like which bare metal servers, how many, where do they host now, did they use cloudfront what about cloudflare did they use an edge cache? what about reducing costs by killing stopping unneeded resources? a lot of their workload looks dynamic. it's also fishy what they wrote here: > After testing whether Prerender pages could be cached in both S3 and minio, we slowly diverted traffic away from AWS S3 and towards minio. if they served directly from s3 that would be, stupid? > In the last four weeks, we moved most of the cache workload from AWS S3 to our own Cassandra cluster. is also strange. it misses a lot of detail but it does not look like they just migrated away from s3... (looks like their new hoster is hetzner, from service.prerender.io )
- pessimizer 4y ago
  > well the problem with the article is basically that they left out a lot of important detail, The problem with your comment is that you insisted that they gave you enough detail to definitively determine that they were either lying or mistaken. > There is no way this figure is accurate.
hazmazlaz 4y ago
Just as a basic sanity check, they claim to store 560 million pages of pure HTML. Internet Archive estimates the median HTML weight of a page to be ~20kb, which I fudged to 30kb in order to make a liberal estimate. Multiplied by the number of pages that would amount to approximately 16.8TB of data stored in s3. According to the AWS pricing calculator that would cost between $300-400 per month, and that's in s3 standard tier pricing with no intelligent tiering set up for less often retrieved data. I didn't bother calculating the compute cost for their rendering because I don't really understand how they do it, and also because the storage estimate was so far off of their claim that it didn't seem necessary to bother in order to refute the claim. I appreciate your challenge for additional evidence, and this is what I have to offer in response. I won't pretend like I did an exhaustive scientific study, this is more back of the napkin type stuff FWIW, but here it is.