5 ms·

In essence, we are going to reinvent CDNs aren't we? For our production needs, we use S3 as a database too, and Cloudflare's CDN as the caching layer. Cache in

In essence, we are going to reinvent CDNs aren't we? For our production needs, we use S3 as a database too, and Cloudflare's CDN as the caching layer. Cache invalidation is handled through versioning.

derefr 6y ago
CDNs aren’t intended as a “canonical store”; content can be invalidated from a CDN’s caches at any time, for any reason (e.g. because the CDN replaced one of their disk nodes), and the CDN expects to be able to re-fetch it from the origin. You need to maintain the canonical store yourself — usually in the form of an object store. (Also, because CDNs try to be nearly-stateless, they don’t tend to be built with an architecture capable of fetching one “primary” copy of your canonical-store data and then mirroring it from there; but rather they usually have each CDN node fetch its own copy directly from your origin. That can be expensive for you, if this data is being computed each time it’s fetched!) Your own HTTP reverse-proxy caching scheme, meanwhile, can be made durable, such that the cache is guaranteed to only re-fetch at explicit controlled intervals. In that sense, it can be the “canonical store”, replacing an object store — at least for the type of data that “expires.” This provides a very nice pipeline: you can write “reporting” code in your backend, exposed on a regular HTTP route, that does some very expensive computations and then just streams them out as an HTTP response; and then you can put your HTTP reverse-proxy cache in front of that route. As long as the cache is durable, and the caching headers are set correctly, you’ll only actually have the reporting endpoint on the backend re-requested when the previous report expires; so you’ll never do a “redundant” re-computation. And yet you don’t need to write a single scrap of rate-limiting code in the backend itself to protect that endpoint from being used to DDoS your system. It’s inherently protected by the caching. You get essentially the same semantics as if the backend itself was a worker running a scheduler that triggered the expensive computation and then pushed the result into an object store, which is then fronted by a CDN; but your backend doesn’t need to know anything about scheduling, or object stores, or any of that. It can be completely stateless, just doing some database/upstream-API queries in response to an HTTP request, building a response, and streaming it. It can be a Lambda, or a single non-framework PHP file, or whatever-you-like.
- deleted 6y ago
  [deleted]
- cornball 6y ago
  This comment touches on a lot of topics that I’m interested in learning more about but don’t have the vocabulary to google or find resources about. Any chance you could point me at a somewhere that I could read about these patterns in more depth? Handling expensive computations and preventing re-computation on the backend is something I don’t know how to solve effectively because there are several different approaches I can think of (in memory cache on the backend, persist the result and return it the next time it’s asked for, cache the response in front of the service, etc)
- ignoramous 6y ago
  Thanks. > Also, because CDNs try to be nearly-stateless, they don’t tend to be built with an architecture capable of fetching one “primary” copy of your canonical-store data and then mirroring it from there. True, though some CDNs let you replicate data up to 25MB in size globally; others support bigger sizes depending on your spends with them. > And yet you don’t need to write a single scrap of rate-limiting code in the backend itself to protect that endpoint from being used to DDoS your system. It’s inherently protected by the caching. Systems in steady state aren't what cause extended outages. The recovery phase can also DDoS your systems. For example, when a cache goes cold, it may over-power an underscaled datastore [0]. > You get essentially the same semantics as if the backend itself was a worker running a scheduler that triggered the expensive computation and then pushed the result into an object store, which is then fronted by a CDN. Agree, but running one's own high-availability infrastructure is hard for a small team. In some applications, even if scale isn't important, availability always mostly is. > It can be a Lambda, or a single non-framework PHP file, or whatever-you-like. Agree, a serverless function a CDN runs (Lambda@Edge, Workers, StackPath EdgeEngine etc) would accomplish a similar feat; and that's what we do for our data workloads that front S3 through Cloudflare Workers. [0] https://sre.google/sre-book/addressing-cascading-failures/ https://sre.google/sre-book/addressing-cascading-failures/
sam0x17 6y ago
CDNs are great unless the content can change / be deleted / contain state information, in which case are bets are off
- ignoramous 6y ago
  In our case, content can change, be deleted, and contain state information. One could accomplish a lot with a CDN and edge compute add-ons.

derefr 6y ago

CDNs aren’t intended as a “canonical store”; content can be invalidated from a CDN’s caches at any time, for any reason (e.g. because the CDN replaced one of their disk nodes), and the CDN expects to be able to re-fetch it from the origin. You need to maintain the canonical store yourself — usually in the form of an object store. (Also, because CDNs try to be nearly-stateless, they don’t tend to be built with an architecture capable of fetching one “primary” copy of your canonical-store data and then mirroring it from there; but rather they usually have each CDN node fetch its own copy directly from your origin. That can be expensive for you, if this data is being computed each time it’s fetched!) Your own HTTP reverse-proxy caching scheme, meanwhile, can be made durable, such that the cache is guaranteed to only re-fetch at explicit controlled intervals. In that sense, it can be the “canonical store”, replacing an object store — at least for the type of data that “expires.” This provides a very nice pipeline: you can write “reporting” code in your backend, exposed on a regular HTTP route, that does some very expensive computations and then just streams them out as an HTTP response; and then you can put your HTTP reverse-proxy cache in front of that route. As long as the cache is durable, and the caching headers are set correctly, you’ll only actually have the reporting endpoint on the backend re-requested when the previous report expires; so you’ll never do a “redundant” re-computation. And yet you don’t need to write a single scrap of rate-limiting code in the backend itself to protect that endpoint from being used to DDoS your system. It’s inherently protected by the caching. You get essentially the same semantics as if the backend itself was a worker running a scheduler that triggered the expensive computation and then pushed the result into an object store, which is then fronted by a CDN; but your backend doesn’t need to know anything about scheduling, or object stores, or any of that. It can be completely stateless, just doing some database/upstream-API queries in response to an HTTP request, building a response, and streaming it. It can be a Lambda, or a single non-framework PHP file, or whatever-you-like.

deleted 6y ago
[deleted]
cornball 6y ago
This comment touches on a lot of topics that I’m interested in learning more about but don’t have the vocabulary to google or find resources about. Any chance you could point me at a somewhere that I could read about these patterns in more depth? Handling expensive computations and preventing re-computation on the backend is something I don’t know how to solve effectively because there are several different approaches I can think of (in memory cache on the backend, persist the result and return it the next time it’s asked for, cache the response in front of the service, etc)
ignoramous 6y ago
Thanks. > Also, because CDNs try to be nearly-stateless, they don’t tend to be built with an architecture capable of fetching one “primary” copy of your canonical-store data and then mirroring it from there. True, though some CDNs let you replicate data up to 25MB in size globally; others support bigger sizes depending on your spends with them. > And yet you don’t need to write a single scrap of rate-limiting code in the backend itself to protect that endpoint from being used to DDoS your system. It’s inherently protected by the caching. Systems in steady state aren't what cause extended outages. The recovery phase can also DDoS your systems. For example, when a cache goes cold, it may over-power an underscaled datastore [0]. > You get essentially the same semantics as if the backend itself was a worker running a scheduler that triggered the expensive computation and then pushed the result into an object store, which is then fronted by a CDN. Agree, but running one's own high-availability infrastructure is hard for a small team. In some applications, even if scale isn't important, availability always mostly is. > It can be a Lambda, or a single non-framework PHP file, or whatever-you-like. Agree, a serverless function a CDN runs (Lambda@Edge, Workers, StackPath EdgeEngine etc) would accomplish a similar feat; and that's what we do for our data workloads that front S3 through Cloudflare Workers. [0] https://sre.google/sre-book/addressing-cascading-failures/ https://sre.google/sre-book/addressing-cascading-failures/

sam0x17 6y ago

CDNs are great unless the content can change / be deleted / contain state information, in which case are bets are off

ignoramous 6y ago
In our case, content can change, be deleted, and contain state information. One could accomplish a lot with a CDN and edge compute add-ons.