3 ms·

Seems to mainly be about Reddit. Didn't seem to answer the #1 question I had in my mind - how do they cache votes for logged-in users? Do they, say, cache a ge

by desu 17y ago

Seems to mainly be about Reddit. Didn't seem to answer the #1 question I had in my mind - how do they cache votes for logged-in users? Do they, say, cache a generic front page and then apply the user's prior voting choice via JS? Or do they cache the page only for the general public, and for logged-in users just cache their voting history and generate it on-the-fly? However they do it, they do a damn good job. I'm always impressed by how responsive Reddit is, along with the other highly dynamic / high load sites like Digg, etc. They manage to stay up and running fine even with all that going on; but just a link on their front page to someone's blog, which should be almost static, crushes that server into goo. Shows the power of good design!

joe_the_user 17y ago
This probably isn't how they do it but you could have a static page updates customizing information with AJAX.
bmelton 17y ago
In the past my strategy was basically that. Load a static page from cache wherever possible. If the user performs a vote, or submit, or what have you, write it to the database, but just perform the update on the user's local page in the DOM.
- desu 17y ago
  Yeah, that's what you'd do as the user makes a vote. But how would you load the page in the first place? The front page is different for each user, as they've likely voted up and down numerous links on it. You could load a generic page and then apply their votes by AJAX or something but that doesn't strike me as being any more efficient for the DB than just generating the page.
  - ajkirwin 17y ago
    Loading data from the db can be cheaper than actually RENDERING said data, so if you can push that clientside, you can save cycles.
  - bmelton 17y ago
    For the most part, we were serving a generic page, except for prior actions done by the user. The only major differences between your page and my page, for example, is that article 5 might not show vote links at all, if I had previously voted on them. For the most part, we accomplished this through cookies. If the cookie exists, read in a hashmap of activities you've recently performed, and update the page DOM based on those. If the cookie didn't exist, then go ahead and do a full query from the db, repop the cache and build the cookie. That said, we noticed that in a lot of cases, that wasn't ideal, and we started rendering partial templates -- so instead of loading a whole page from cache, we would load page parts, basically one part for each article. This worked well, as each template was basically a pre-rendered static file, and we only had two basic states to work with -- whether the user had voted on this before or not. Our new strategy then was to query just the indexed table for a user's recent votes, separately query an index for which articles to display, and load the templates for the to-be-displayed articles based upon the user's vote state. That allowed the page to render a lot faster, but kept the box under more load. Though we suspect it would have scaled well enough (had the project not eventually fizzled out), it wasn't ideal server-side, though delivered a vastly superior end-user experience. YMMV.
uggedal 17y ago
Use the source Luke: http://code.reddit.com/ http://code.reddit.com/
- desu 17y ago
  Thanks. I want to know the answer, but not so much that I'm willing to spend the several hours necessary to familiarise myself with a large foreign code base. I was hoping someone here could give a quick top-level explanation. From looking around, though, I think they're caching the user's votes and then just assembling the page on the fly from cache fragments. You could do that pretty quickly. Might be wrong.
jules 17y ago
You could load the user's votes into a cookie.
- desu 17y ago
  It would get too large quickly. Check this out for a good explanation of why you want to keep cookie sizes as small as possible: http://yuiblog.com/blog/2007/03/01/performance-research-part-3/ http://yuiblog.com/blog/2007/03/01/performance-research-part...
  - peoplerock 17y ago
    Thanks for that link. Interesting on the how ebay and myspace use cookie size with great abandon (v. Amazon & Google, for ex.). Leaves me wondering whether they know what it's costing them. And if so - whether they are realizing significant savings with other performance figures.

by desu 17y ago

joe_the_user 17y ago

This probably isn't how they do it but you could have a static page updates customizing information with AJAX.

bmelton 17y ago

In the past my strategy was basically that. Load a static page from cache wherever possible. If the user performs a vote, or submit, or what have you, write it to the database, but just perform the update on the user's local page in the DOM.

desu 17y ago
Yeah, that's what you'd do as the user makes a vote. But how would you load the page in the first place? The front page is different for each user, as they've likely voted up and down numerous links on it. You could load a generic page and then apply their votes by AJAX or something but that doesn't strike me as being any more efficient for the DB than just generating the page.
- ajkirwin 17y ago
  Loading data from the db can be cheaper than actually RENDERING said data, so if you can push that clientside, you can save cycles.
- bmelton 17y ago
  For the most part, we were serving a generic page, except for prior actions done by the user. The only major differences between your page and my page, for example, is that article 5 might not show vote links at all, if I had previously voted on them. For the most part, we accomplished this through cookies. If the cookie exists, read in a hashmap of activities you've recently performed, and update the page DOM based on those. If the cookie didn't exist, then go ahead and do a full query from the db, repop the cache and build the cookie. That said, we noticed that in a lot of cases, that wasn't ideal, and we started rendering partial templates -- so instead of loading a whole page from cache, we would load page parts, basically one part for each article. This worked well, as each template was basically a pre-rendered static file, and we only had two basic states to work with -- whether the user had voted on this before or not. Our new strategy then was to query just the indexed table for a user's recent votes, separately query an index for which articles to display, and load the templates for the to-be-displayed articles based upon the user's vote state. That allowed the page to render a lot faster, but kept the box under more load. Though we suspect it would have scaled well enough (had the project not eventually fizzled out), it wasn't ideal server-side, though delivered a vastly superior end-user experience. YMMV.

uggedal 17y ago

Use the source Luke: http://code.reddit.com/ http://code.reddit.com/

desu 17y ago
Thanks. I want to know the answer, but not so much that I'm willing to spend the several hours necessary to familiarise myself with a large foreign code base. I was hoping someone here could give a quick top-level explanation. From looking around, though, I think they're caching the user's votes and then just assembling the page on the fly from cache fragments. You could do that pretty quickly. Might be wrong.

jules 17y ago

You could load the user's votes into a cookie.

desu 17y ago
It would get too large quickly. Check this out for a good explanation of why you want to keep cookie sizes as small as possible: http://yuiblog.com/blog/2007/03/01/performance-research-part-3/ http://yuiblog.com/blog/2007/03/01/performance-research-part...
- peoplerock 17y ago
  Thanks for that link. Interesting on the how ebay and myspace use cookie size with great abandon (v. Amazon & Google, for ex.). Leaves me wondering whether they know what it's costing them. And if so - whether they are realizing significant savings with other performance figures.