4 ms·

Definitely a strong use case for a caching proxy like Varnish or Nginx. The post describes using Redis as an application cache which means your application need

Definitely a strong use case for a caching proxy like Varnish or Nginx. The post describes using Redis as an application cache which means your application needs custom logic. A cache lookup before going to S3, and then it needs to write back out to cache when done. Simple, but possibly a lot more work than necessary. A caching proxy does away with this, you make a single call to the proxy, which behaves identically to S3, and it does all the work for you. No application changes needed other than changing the S3 endpoint address. Also, much greater performance because you are potentially cutting out multiple costly round trips.

welder 5y ago
Cache invalidation is much easier with SSDB than Nginx or Varnish. Http proxies are good for public facing uses but not a good use case here.
- haggy 5y ago
  How do you figure? Http proxies support various common cache invalidation strategies such as hash-TTL and ETags
  - SahAssar 5y ago
    ETags are not cache invalidation. ETags are a way for a client to validate the cache in the client, while cache invalidation for a http proxy would involve telling the proxy that it should discard a cached file. --- Basically: cache validation: Check if a file matches a cached version cache invalidation: Preemptively tell a cache to discard a cached file --- For example: Client A requests file X. Proxy caches file X. File X changes in the upstream. Client B requests file X. Proxy does not know the file changed upstream, so it sends a stale version. --- In this case either the proxy needs to revalidate each request with upstream (which is expensive) or we need some way to tell the proxy that it should discard it's cache for file X since it has changed. Nginx plus supports this but the free version does not: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge https://nginx.org/en/docs/http/ngx_http_proxy_module.html#pr...
    - e12e 5y ago
      For Varnish PURGE seems the most sensible: https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html https://book.varnish-software.com/4.0/chapters/Cache_Invalid... I think I'll have to look into this to see if we can get away with document/blob cache in one of our solutions actually. I've resisted add-in Varnish in the mix, but it might be time to reconsider.

welder 5y ago

Cache invalidation is much easier with SSDB than Nginx or Varnish. Http proxies are good for public facing uses but not a good use case here.

haggy 5y ago
How do you figure? Http proxies support various common cache invalidation strategies such as hash-TTL and ETags
- SahAssar 5y ago
  ETags are not cache invalidation. ETags are a way for a client to validate the cache in the client, while cache invalidation for a http proxy would involve telling the proxy that it should discard a cached file. --- Basically: cache validation: Check if a file matches a cached version cache invalidation: Preemptively tell a cache to discard a cached file --- For example: Client A requests file X. Proxy caches file X. File X changes in the upstream. Client B requests file X. Proxy does not know the file changed upstream, so it sends a stale version. --- In this case either the proxy needs to revalidate each request with upstream (which is expensive) or we need some way to tell the proxy that it should discard it's cache for file X since it has changed. Nginx plus supports this but the free version does not: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge https://nginx.org/en/docs/http/ngx_http_proxy_module.html#pr...
  - e12e 5y ago
    For Varnish PURGE seems the most sensible: https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html https://book.varnish-software.com/4.0/chapters/Cache_Invalid... I think I'll have to look into this to see if we can get away with document/blob cache in one of our solutions actually. I've resisted add-in Varnish in the mix, but it might be time to reconsider.