4 ms·

I don't really buy this article. They claim you get "caching, more efficient bandwidth utilization, centralized access with logging and security." S3 gives you

I don't really buy this article. They claim you get "caching, more efficient bandwidth utilization, centralized access with logging and security." S3 gives you caching already. I fail to see how your bandwidth utilization improves, if anything you've introduced another hop (depending where you're hosting Varnish). Centralized access? I guess, if you're already using Varnish for other things, but if you're at the point that you need Varnish, you probably are aggregating multiple log sources already. Security? You're just adding another possible exploitation layer. And on top of that you've just added another box to maintain. Not to rag on Varnish, I love it, I just don't really see this use case. Personally, I would never route plain S3 data through Varnish.

reza_n 11y ago
So I wrote that blog post, was intended to be a quick little guide for S3 integration. As for your point regarding caching and bandwidth utilization, this actually comes up a bit, so its a fairly valid point. A lot of times software (and people) are accessing the same S3 asset, multiple times, all on the same network. So by going thru a local cache, you bypass having to reach all the way back to S3.
- fgsch 11y ago
  Also S3 has different costs depending on number of requests and transfer.
- andrewguenther 11y ago
  That use case makes a lot of sense. Thanks for the response!
- coleifer 11y ago
  Lame of you not to link my post from yesterday about putting nginx in front of s3. This post seems clearly to be in response to mine. http://charlesleifer.com/blog/nginx-a-caching-thumbnailing-reverse-proxying-image-server-/ http://charlesleifer.com/blog/nginx-a-caching-thumbnailing-r...
true_religion 11y ago
One use case: reducing ongoing bandwidth costs. S3 bandwidth is pretty expensive. Even Akamai will give you less expensive bandwidth and their target market is enterprise media companies (e.g. ESPN) with deep pockets. For someone with bandwidth costs issues who wants to use S3 as storage, they can put Varnish in front of it and serve the data with less expense. What do you lose out on? HTTP/2, SSL, etc. unless you put another HTTP server in front of varnish.
- andrewguenther 11y ago
  If you're comparing to Akamai, then shouldn't your data be in a CDN rather than S3?
  - true_religion 11y ago
    Not necessarily. I'm comparing bandwidth costs. Amazon S3 out-going bandwidth is an order of magnitude more expensive than some of its competitors (OVH for example), and, at least, 2x the price of Dreamhost's OpenStack solution (which includes a CDN for outgoing bandwidth). Amazon S3 is essentially more expensive than a CDN for outgoing bandwidth even though it has only one PoP by default. That's surprising since the typical case was that the CDN would be more expensive, and increasing the # of PoP you used in CDN would make it more expensive (e.g. there's a huge jump in pricing if you use Asia/Australia nodes).
babo 11y ago
and it gives you logging as well

reza_n 11y ago

So I wrote that blog post, was intended to be a quick little guide for S3 integration. As for your point regarding caching and bandwidth utilization, this actually comes up a bit, so its a fairly valid point. A lot of times software (and people) are accessing the same S3 asset, multiple times, all on the same network. So by going thru a local cache, you bypass having to reach all the way back to S3.

fgsch 11y ago
Also S3 has different costs depending on number of requests and transfer.
andrewguenther 11y ago
That use case makes a lot of sense. Thanks for the response!
coleifer 11y ago
Lame of you not to link my post from yesterday about putting nginx in front of s3. This post seems clearly to be in response to mine. http://charlesleifer.com/blog/nginx-a-caching-thumbnailing-reverse-proxying-image-server-/ http://charlesleifer.com/blog/nginx-a-caching-thumbnailing-r...

true_religion 11y ago

One use case: reducing ongoing bandwidth costs. S3 bandwidth is pretty expensive. Even Akamai will give you less expensive bandwidth and their target market is enterprise media companies (e.g. ESPN) with deep pockets. For someone with bandwidth costs issues who wants to use S3 as storage, they can put Varnish in front of it and serve the data with less expense. What do you lose out on? HTTP/2, SSL, etc. unless you put another HTTP server in front of varnish.

andrewguenther 11y ago
If you're comparing to Akamai, then shouldn't your data be in a CDN rather than S3?
- true_religion 11y ago
  Not necessarily. I'm comparing bandwidth costs. Amazon S3 out-going bandwidth is an order of magnitude more expensive than some of its competitors (OVH for example), and, at least, 2x the price of Dreamhost's OpenStack solution (which includes a CDN for outgoing bandwidth). Amazon S3 is essentially more expensive than a CDN for outgoing bandwidth even though it has only one PoP by default. That's surprising since the typical case was that the CDN would be more expensive, and increasing the # of PoP you used in CDN would make it more expensive (e.g. there's a huge jump in pricing if you use Asia/Australia nodes).

babo 11y ago

and it gives you logging as well