4 ms·

Why not an http proxy with disk caching, e.g. location / { proxy_http_version 1.1; proxy_set_header Connection ''; proxy_set_header .....

by latch 5y ago

Why not an http proxy with disk caching, e.g. location / { proxy_http_version 1.1; proxy_set_header Connection ''; proxy_set_header ..... proxy_cache s3_zone; proxy_pass https://s3 } If you wanted to scale this to multiple proxies, without having each one be a duplicate, do a consistent hash of the URL and proxy it to the owner. We used to do this to cache the first chunk of videos. Encoded with `-movflags faststart` this typically ensured that the moov chunk was cached at an edge, which dramatically decreased the wait time of video playback (supporting arbitrary seeking of large/long video file)

welder 5y ago
Because we invalidate some SSDB keys in various cases, for ex: like old code stats data incoming, renaming a project, etc. Http proxies with etag seem like a good public-facing cache option, but our caching is internal to our DigitalOcean servers. Here's more info on our infra, which might help with our decision: https://wakatime.com/blog/46-latency-of-digitalocean-spaces-vs-aws-s3 https://wakatime.com/blog/46-latency-of-digitalocean-spaces-...
- latch 5y ago
  nginx has proxy_cache_purge (1), but I agree that nginx doesn't expose a particularly good API around its cache. Plus, the awkwardness of doing disk IO from OpenResty (unless this has been improved), is a drag. Maybe this is something envoy or the newer breads do better. (1) http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge http://nginx.org/en/docs/http/ngx_http_proxy_module.html#pro...
- otterley 5y ago
  Consider using opaque symbols for your object/cache identifiers instead of using meaningful literals. That way, you can simply update the identifier mapping (Project A cache prefix moves from ajsdf09yu8sbvoihjasdg -> klajnsdg9fasf8avby), and the old values will naturally be evicted from the cache as they become less frequently accessed. In my experience, having to perform cache invalidation is usually a sign of design immaturity. Senior engineers have been through this trial before :-)
ignoramous 5y ago
In essence, we are going to reinvent CDNs aren't we? For our production needs, we use S3 as a database too, and Cloudflare's CDN as the caching layer. Cache invalidation is handled through versioning.
- derefr 5y ago
  CDNs aren’t intended as a “canonical store”; content can be invalidated from a CDN’s caches at any time, for any reason (e.g. because the CDN replaced one of their disk nodes), and the CDN expects to be able to re-fetch it from the origin. You need to maintain the canonical store yourself — usually in the form of an object store. (Also, because CDNs try to be nearly-stateless, they don’t tend to be built with an architecture capable of fetching one “primary” copy of your canonical-store data and then mirroring it from there; but rather they usually have each CDN node fetch its own copy directly from your origin. That can be expensive for you, if this data is being computed each time it’s fetched!) Your own HTTP reverse-proxy caching scheme, meanwhile, can be made durable, such that the cache is guaranteed to only re-fetch at explicit controlled intervals. In that sense, it can be the “canonical store”, replacing an object store — at least for the type of data that “expires.” This provides a very nice pipeline: you can write “reporting” code in your backend, exposed on a regular HTTP route, that does some very expensive computations and then just streams them out as an HTTP response; and then you can put your HTTP reverse-proxy cache in front of that route. As long as the cache is durable, and the caching headers are set correctly, you’ll only actually have the reporting endpoint on the backend re-requested when the previous report expires; so you’ll never do a “redundant” re-computation. And yet you don’t need to write a single scrap of rate-limiting code in the backend itself to protect that endpoint from being used to DDoS your system. It’s inherently protected by the caching. You get essentially the same semantics as if the backend itself was a worker running a scheduler that triggered the expensive computation and then pushed the result into an object store, which is then fronted by a CDN; but your backend doesn’t need to know anything about scheduling, or object stores, or any of that. It can be completely stateless, just doing some database/upstream-API queries in response to an HTTP request, building a response, and streaming it. It can be a Lambda, or a single non-framework PHP file, or whatever-you-like.
  - deleted 5y ago

by latch 5y ago

welder 5y ago

Because we invalidate some SSDB keys in various cases, for ex: like old code stats data incoming, renaming a project, etc. Http proxies with etag seem like a good public-facing cache option, but our caching is internal to our DigitalOcean servers. Here's more info on our infra, which might help with our decision: https://wakatime.com/blog/46-latency-of-digitalocean-spaces-vs-aws-s3 https://wakatime.com/blog/46-latency-of-digitalocean-spaces-...

latch 5y ago
nginx has proxy_cache_purge (1), but I agree that nginx doesn't expose a particularly good API around its cache. Plus, the awkwardness of doing disk IO from OpenResty (unless this has been improved), is a drag. Maybe this is something envoy or the newer breads do better. (1) http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_purge http://nginx.org/en/docs/http/ngx_http_proxy_module.html#pro...
otterley 5y ago
Consider using opaque symbols for your object/cache identifiers instead of using meaningful literals. That way, you can simply update the identifier mapping (Project A cache prefix moves from ajsdf09yu8sbvoihjasdg -> klajnsdg9fasf8avby), and the old values will naturally be evicted from the cache as they become less frequently accessed. In my experience, having to perform cache invalidation is usually a sign of design immaturity. Senior engineers have been through this trial before :-)

ignoramous 5y ago

In essence, we are going to reinvent CDNs aren't we? For our production needs, we use S3 as a database too, and Cloudflare's CDN as the caching layer. Cache invalidation is handled through versioning.

derefr 5y ago
CDNs aren’t intended as a “canonical store”; content can be invalidated from a CDN’s caches at any time, for any reason (e.g. because the CDN replaced one of their disk nodes), and the CDN expects to be able to re-fetch it from the origin. You need to maintain the canonical store yourself — usually in the form of an object store. (Also, because CDNs try to be nearly-stateless, they don’t tend to be built with an architecture capable of fetching one “primary” copy of your canonical-store data and then mirroring it from there; but rather they usually have each CDN node fetch its own copy directly from your origin. That can be expensive for you, if this data is being computed each time it’s fetched!) Your own HTTP reverse-proxy caching scheme, meanwhile, can be made durable, such that the cache is guaranteed to only re-fetch at explicit controlled intervals. In that sense, it can be the “canonical store”, replacing an object store — at least for the type of data that “expires.” This provides a very nice pipeline: you can write “reporting” code in your backend, exposed on a regular HTTP route, that does some very expensive computations and then just streams them out as an HTTP response; and then you can put your HTTP reverse-proxy cache in front of that route. As long as the cache is durable, and the caching headers are set correctly, you’ll only actually have the reporting endpoint on the backend re-requested when the previous report expires; so you’ll never do a “redundant” re-computation. And yet you don’t need to write a single scrap of rate-limiting code in the backend itself to protect that endpoint from being used to DDoS your system. It’s inherently protected by the caching. You get essentially the same semantics as if the backend itself was a worker running a scheduler that triggered the expensive computation and then pushed the result into an object store, which is then fronted by a CDN; but your backend doesn’t need to know anything about scheduling, or object stores, or any of that. It can be completely stateless, just doing some database/upstream-API queries in response to an HTTP request, building a response, and streaming it. It can be a Lambda, or a single non-framework PHP file, or whatever-you-like.
- deleted 5y ago