5 ms·

"If you have personalized pages (ie: majority logged-in users) this approach isn't going to work. " I've considered this problem, and am working on a solution

by nirvana 15y ago

"If you have personalized pages (ie: majority logged-in users) this approach isn't going to work. " I've considered this problem, and am working on a solution for nirvana[1]. The biggest challenge to this project has been to take a language (coffeescript) that is sequential and run it in a distributed environment, without the programmer having to know distributed programming. One of the techniques I'm applying is making a response (in this case, a web page) the result of a collection of components, which are rendered separately in the same context. (EG: The context is the headers of the request, plus the user record if the user is logged in, etc.) So, the request comes in, the components are loaded from the cache, they are executed (in parallel) all with a copy of the state, their results are aggregated and that result can run thru templating to produce a webpage that is returned. The idea then becomes, instead of executing the code for every component in every request, if the component has no context specific requirements (e.g.: it is the same for every user, it's a static element, or it's dynamic, and but doesn't need to be generated every time) .. then it can be flagged as cacheable. The caching would also have a staleness factor (Eg: 1m, 5m 10m).[2] My hope is that you can have pages that are custom per user, but that also contain heavy impact results (say a graph produced by an expensive operation), where the results come form cache, the static components come from cache, but the user specific parts are dynamically generated each request. This component approach not only lets the code be rendered in parallel, and often not even rendered, but instead pulled from cache, but it should allow for more convenient re-use of common elements and features across a site. I hadn't considered caching for just 1s, though. Will have to think about that. [1] Nirvana is CoffeeScript web development backed by erlang and Riak. Instantly distributed coffeescript. It will be open source, hopefully soon. Follow @nirvanacore on Twitter if you're interested in being notified. [2] Planned. There are some implications of this that will require tradeoffs, so initially it may just be a flag of Yes/No for "Cache for up to 1 minute." or some value like that.

aidenn0 15y ago
Something I have done in the past is put all the "loaded by everyone but not static" stuff in memcached. The application pushes to memcached on startup, and any POST that would change it causes the application backend to update the memcached entry(ies). This is a slightly different approach from the microcaching, but it has the advantage of being always consistent. I only do this when the stuff starts to obviously become a bottleneck, and I haven't done much with cloud hosting, where RAM seems to be more expensive than cpu cycles, so I don't know how well it would work there.
- sciurus 15y ago
  To relate this even more directly to the article, I believe W3 Total Cache can be configured to do this. http://wordpress.org/extend/plugins/w3-total-cache/ http://wordpress.org/extend/plugins/w3-total-cache/
- nirvana 15y ago
  Effectively, a Riak in-memory database is memcached, but distributed over all the nodes, with zero administration to grow or shrink it. So, if I've got 10 nodes, and each has 30MB set aside for this caching (or whatever) effectively its a 300MB cache. Adding another node makes it a 330MB cache. I'm avoiding the logic of deciding whether a POST would change it or not, by letting the components handle it. They can set a default TTL, and they can also, if they process any relevant data, force an update to the cache.
phillmv 15y ago
>Nirvana is CoffeeScript web development backed by erlang and Riak. OH COME ON, I would almost bet money @nirvanacore follows the great traditions set by @rolfscale and @hipsterhacker. That said!, good luck :D. >The idea then becomes, instead of executing the code for every component in every request, if the component has no context specific requirements (e.g.: it is the same for every user, it's a static element, or it's dynamic, and but doesn't need to be generated every time) .. then it can be flagged as cacheable. How are you going to win over everyone else's caching solution? Why are you going to be easier than cache[:foo] ||= Computation.lovers_do_it_expensively() ?
- nirvana 15y ago
  I'd love to find a place on the internet where I can talk about my project seriously with people. I'm the only engineer in our small startup, and building this platform to decrease the cost of development for myself going forward. I'm open sourcing it because there isn't a solution like this out here (or I'd just use that instead.) My focus is actually on zero-operations-dudes-on-payroll scalability up to medium sized services, not "it's gonna be huge man" scalability. I'm not using MongoDB because its "wicked fast!!!", I'm using Riak because "no ops guys needed, just add nodes." Seriously, anyone know such a place? I guess I'll have one when I make the first release, since I'll make a mailing list and issue tracker, etc. But right now, before the first release, when I'm thinking about serious architectural issues, that's the time I could use someone off of whom I could bounce ideas and get feedback, positive or negative. I mean, constructive feedback. That's why I made the comment on this post, because it's reasonably relevant to what I'm doing. In answer to your question, I have no interest in winning over anyone else's caching solution. I'm not building a web cache, I'm building a web development platform. This makes web development a whole lot easier for the kind of development I do. I expect it will be useful for others, but even if it isn't, the time invested in it is almost nearly recouped developing our MVP.
  - phillmv 15y ago
    Oh stop whining. My jest at the top of my post obscured my seriousness at the bottom. My point largely was, "I can't tell if you're being satirical or not". You're meshing three extremely new and very fancy technology stacks together and inventing a whole new framework to go with it. So… maybe you do want to figure out a good stock way to counter my skepticism. Waving your hands and saying "it's a web development platform!" feels unsatisfying :).
buro9 15y ago
Varnish supports ESI. What you've described is ESI. Just put Varnish in front of your servers and used the Expires headers of the responses to cache stuff accordingly. If you include user information in the cache hash of the personalised chunks of the page (i.e. in the URL or cookie) then you can cache that for a short time too. I do apply short term caching to a lot of things, mostly to protect against double requests for the same thing when there might be an expensive query behind it.
- nirvana 15y ago
  Thanks for the pointer. There's two advantages to what I'm doing over using Varnish in this way: 1. The component writer knows the needs of the component and can set the caching policy for it there. Other developers can then just include the component without knowing what its needs are. In a way, this means they don't have to design the page for caching.. they kinda get it for free. 2. There are several optimizations I can make using this method, and since it is intrinsic to the need to be able to run things in parallel (components have a clearly defined line between them another components) the caching ability is just a few extra lines of code and so relatively low cost for me to add.
- Cherian 15y ago
  I believe nginx has something similar called SSI http://wiki.nginx.org/HttpSsiModule http://wiki.nginx.org/HttpSsiModule
  - buro9 15y ago
    Everything has had SSI. In fact the way that the BBC used to make their web site in PERL on Apache used SSI for their "Most Read" section. ESI is different... Edge Side rather than Server Side. The benefit is that you don't need one server capable of doing everything. You can have a separate web service capable of returning the "Most Read" section on different technology, different machines, elsewhere on the network... and ESI will interleave the output of these web services at the edge of the network. Varnish supports ESI, and as one of the main reasons to use Varnish is caching you can reasonably expect (and be correct) that Varnish will cache ESI sections. So Varnish will receive a document, cache the whole of it, and use an entirely different cache policy on a section of it. It's all swings and roundabouts. I just have a strong preference towards not making any part of the solution complex by trying to make it do anything more than it need do, especially when other layers can be added that do that specific thing (caching in this instance) better.
tobyhede 15y ago
THAT SOUNDS AWESOME. Really interesting idea, I've been working with Riak and CoffeeScript and love the model. Adding distribution is ... inspired. Mad like a fox!
mkjones 15y ago
Cool approach - sounds a lot like what we call "Pagelets" at Facebook. Check out https://www.facebook.com/note.php?note_id=389414033919 https://www.facebook.com/note.php?note_id=389414033919 for a writeup about how we use them.
- nirvana 15y ago
  Thanks, reading that now. Riak has a pipeline service called riak_pipe. One of the things I've been thinking about is having applications define a pipe, and then sending requests thru the pipe. The pipe is inherently sequential, so it fits well with a sequential language like javascript (you'd just write fittings in the coffeescript/javascript and then at the app level define the pipeline as a series of fittings.) On the other hand, this seems potentially less parallelizable and so I won't know which is more performant until I can run some tests (And I'm not yet to the point where I can do that.) Anyway, reading the page you linked to now, will update this comment with what I learn from it. EDIT TO ADD: Have read a good chunk of that page now. THANK YOU! It's very interesting. I'd considered doing something like this already, but had put it off for later. At this point I'm working on just the server side, though I've saved the article and will check it out again when I can had some higher level functionality to nirvana. Thanks!
riffraff 15y ago
I believe this is somewhat similar to how Snippets work in Lift, where each snippet renders by itself and coordination is managed by actors.
troels 15y ago
I'm doing client side includes for personalised content. So, the main page is served with no personalised content and with all the heavy caching that this allows, and the html has place holders for the personalised content. Then, upon page load, an ajax request is fired that loads the user-specific content and injects it to the page. It works quite well and since this is specifically personalised content, I'm not too afraid of having to support non-js clients (Such as crawlers etc.).

by nirvana 15y ago

aidenn0 15y ago

Something I have done in the past is put all the "loaded by everyone but not static" stuff in memcached. The application pushes to memcached on startup, and any POST that would change it causes the application backend to update the memcached entry(ies). This is a slightly different approach from the microcaching, but it has the advantage of being always consistent. I only do this when the stuff starts to obviously become a bottleneck, and I haven't done much with cloud hosting, where RAM seems to be more expensive than cpu cycles, so I don't know how well it would work there.

sciurus 15y ago
To relate this even more directly to the article, I believe W3 Total Cache can be configured to do this. http://wordpress.org/extend/plugins/w3-total-cache/ http://wordpress.org/extend/plugins/w3-total-cache/
nirvana 15y ago
Effectively, a Riak in-memory database is memcached, but distributed over all the nodes, with zero administration to grow or shrink it. So, if I've got 10 nodes, and each has 30MB set aside for this caching (or whatever) effectively its a 300MB cache. Adding another node makes it a 330MB cache. I'm avoiding the logic of deciding whether a POST would change it or not, by letting the components handle it. They can set a default TTL, and they can also, if they process any relevant data, force an update to the cache.

phillmv 15y ago

>Nirvana is CoffeeScript web development backed by erlang and Riak. OH COME ON, I would almost bet money @nirvanacore follows the great traditions set by @rolfscale and @hipsterhacker. That said!, good luck :D. >The idea then becomes, instead of executing the code for every component in every request, if the component has no context specific requirements (e.g.: it is the same for every user, it's a static element, or it's dynamic, and but doesn't need to be generated every time) .. then it can be flagged as cacheable. How are you going to win over everyone else's caching solution? Why are you going to be easier than cache[:foo] ||= Computation.lovers_do_it_expensively() ?

nirvana 15y ago
I'd love to find a place on the internet where I can talk about my project seriously with people. I'm the only engineer in our small startup, and building this platform to decrease the cost of development for myself going forward. I'm open sourcing it because there isn't a solution like this out here (or I'd just use that instead.) My focus is actually on zero-operations-dudes-on-payroll scalability up to medium sized services, not "it's gonna be huge man" scalability. I'm not using MongoDB because its "wicked fast!!!", I'm using Riak because "no ops guys needed, just add nodes." Seriously, anyone know such a place? I guess I'll have one when I make the first release, since I'll make a mailing list and issue tracker, etc. But right now, before the first release, when I'm thinking about serious architectural issues, that's the time I could use someone off of whom I could bounce ideas and get feedback, positive or negative. I mean, constructive feedback. That's why I made the comment on this post, because it's reasonably relevant to what I'm doing. In answer to your question, I have no interest in winning over anyone else's caching solution. I'm not building a web cache, I'm building a web development platform. This makes web development a whole lot easier for the kind of development I do. I expect it will be useful for others, but even if it isn't, the time invested in it is almost nearly recouped developing our MVP.
- phillmv 15y ago
  Oh stop whining. My jest at the top of my post obscured my seriousness at the bottom. My point largely was, "I can't tell if you're being satirical or not". You're meshing three extremely new and very fancy technology stacks together and inventing a whole new framework to go with it. So… maybe you do want to figure out a good stock way to counter my skepticism. Waving your hands and saying "it's a web development platform!" feels unsatisfying :).

buro9 15y ago

Varnish supports ESI. What you've described is ESI. Just put Varnish in front of your servers and used the Expires headers of the responses to cache stuff accordingly. If you include user information in the cache hash of the personalised chunks of the page (i.e. in the URL or cookie) then you can cache that for a short time too. I do apply short term caching to a lot of things, mostly to protect against double requests for the same thing when there might be an expensive query behind it.

nirvana 15y ago
Thanks for the pointer. There's two advantages to what I'm doing over using Varnish in this way: 1. The component writer knows the needs of the component and can set the caching policy for it there. Other developers can then just include the component without knowing what its needs are. In a way, this means they don't have to design the page for caching.. they kinda get it for free. 2. There are several optimizations I can make using this method, and since it is intrinsic to the need to be able to run things in parallel (components have a clearly defined line between them another components) the caching ability is just a few extra lines of code and so relatively low cost for me to add.
Cherian 15y ago
I believe nginx has something similar called SSI http://wiki.nginx.org/HttpSsiModule http://wiki.nginx.org/HttpSsiModule
- buro9 15y ago
  Everything has had SSI. In fact the way that the BBC used to make their web site in PERL on Apache used SSI for their "Most Read" section. ESI is different... Edge Side rather than Server Side. The benefit is that you don't need one server capable of doing everything. You can have a separate web service capable of returning the "Most Read" section on different technology, different machines, elsewhere on the network... and ESI will interleave the output of these web services at the edge of the network. Varnish supports ESI, and as one of the main reasons to use Varnish is caching you can reasonably expect (and be correct) that Varnish will cache ESI sections. So Varnish will receive a document, cache the whole of it, and use an entirely different cache policy on a section of it. It's all swings and roundabouts. I just have a strong preference towards not making any part of the solution complex by trying to make it do anything more than it need do, especially when other layers can be added that do that specific thing (caching in this instance) better.

tobyhede 15y ago

THAT SOUNDS AWESOME. Really interesting idea, I've been working with Riak and CoffeeScript and love the model. Adding distribution is ... inspired. Mad like a fox!

mkjones 15y ago

Cool approach - sounds a lot like what we call "Pagelets" at Facebook. Check out https://www.facebook.com/note.php?note_id=389414033919 https://www.facebook.com/note.php?note_id=389414033919 for a writeup about how we use them.

nirvana 15y ago
Thanks, reading that now. Riak has a pipeline service called riak_pipe. One of the things I've been thinking about is having applications define a pipe, and then sending requests thru the pipe. The pipe is inherently sequential, so it fits well with a sequential language like javascript (you'd just write fittings in the coffeescript/javascript and then at the app level define the pipeline as a series of fittings.) On the other hand, this seems potentially less parallelizable and so I won't know which is more performant until I can run some tests (And I'm not yet to the point where I can do that.) Anyway, reading the page you linked to now, will update this comment with what I learn from it. EDIT TO ADD: Have read a good chunk of that page now. THANK YOU! It's very interesting. I'd considered doing something like this already, but had put it off for later. At this point I'm working on just the server side, though I've saved the article and will check it out again when I can had some higher level functionality to nirvana. Thanks!

riffraff 15y ago

I believe this is somewhat similar to how Snippets work in Lift, where each snippet renders by itself and coordination is managed by actors.

troels 15y ago

I'm doing client side includes for personalised content. So, the main page is served with no personalised content and with all the heavy caching that this allows, and the html has place holders for the personalised content. Then, upon page load, an ajax request is fired that loads the user-specific content and injects it to the page. It works quite well and since this is specifically personalised content, I'm not too afraid of having to support non-js clients (Such as crawlers etc.).