6 ms·

Only disagreement (although I feel like I'm arguing w/ Linus about git) is don't memcache session data (lesson 5.) Memcache's 1mb max-block (exceeding that remo

by ecaron 14y ago

Only disagreement (although I feel like I'm arguing w/ Linus about git) is don't memcache session data (lesson 5.) Memcache's 1mb max-block (exceeding that removes too many performance perks to be considered viable) introduces a "I need to constantly worry about my sessions getting too big" mental overhead that isn't worth it. Go with Redis for storing session data.

batista 14y ago
Why would you need 1mb session data? Or 200K for that matter?
- ecaron 14y ago
  Because it happens. Typically in places where it shouldn't (e.g. bad programming) but sometimes it does (e.g. enormously complex active system state.) The point is that there are solutions offering equivalent speed without that barrier, so why select a technology that has limitations?
  - batista 14y ago
    "Because it happens" is not a very good technical answer. Anything can happen, even 1GB session data. But even anything approaching even half a MB of session data I would take as an architectural failure, and work to fix it, instead of letting it dictate what tools I use (ie. redis vs memcached). >The point is that there are solutions offering equivalent speed without that barrier, so why select a technology that has limitations? Because all decisions have tradeoffs and it more wise to depend on your particular use case to choose than to depend on a scenario (> 1MB session data) that can only happen with a seriously fucked up application design.
    - disordinary 14y ago
      You can easily get to 1mb if you follow the cache everything mantra that reddit is using here.
      - nostrademons 14y ago
        Why in sessions, though? Why not cache everything as uniquely-identified resources and store only the resource ID in the session? That way, resources can be shared across different sessions. I'm trying to think of a case where a user's actions (that's what's supposed to be in sessions, right?) can result in 1M of data and drawing a blank. Maybe if they're uploading images or video transiently, like in Google's Search-by-Image feature, but even that seems like a case where you should upload the image permanently and just store a reference to it in the session. Edit: Answered my own question: oftentimes it's useful to store a history of the user's recent actions in their session, because then you can build a more responsive, more tailored UI. You can save transient data they've already entered, avoid showing them promos they've already indicated no interest in, and potentially use it to predict useful things to show them next. Even then, though, it'd take 500 pageviews at 2K apiece (which is a very generous estimate) to hit a meg of data.
        Morg 14y ago
        NO ffs NO. There is absolutely no reason to store anything but the session ID in a session cookie. You can keep it in the js, so why use session ? multiple pages ? dude you're living in the past.
masklinn 14y ago
FWIW Reddit replaced memcached with cassandra in early 2010 for caches and transient stores.
- mdellabitta 14y ago
  Wow, now I'm glad I read comments before the article. This article is really out of date. They've come a long way since this was written.
  - masklinn 14y ago
    > This article is really out of date. Well yeah it's from early 2010, it's been 2 years. In January (2012)'s "state of the servers" post, the reddit blog noted traffic was 2 billion pageviews per month in December 2011, up from 870m or so in December 2010.
encoderer 14y ago
Even better: Don't use sessions. If there's data you're caching in a session, pre-compute it and store it in Redis (or other suitable NoSQL database). Going statless wherever possible almost always wins. (And I only say "almost" because I'm not smart enough to know if it really always wins. But for me, it has)
- irahul 14y ago
  > Even better: Don't use sessions. If there's data you're caching in a session, pre-compute it and store it in Redis (or other suitable NoSQL database). What do you mean by data you are caching in a session? Session is for maintaining state viz. logged in or not, language preference etc. How is storing in Redis(or any other data store) any different from storing in Memcache? > Going statless wherever possible almost always wins. Using sessions with a data store which can be accessed from multiple machines(no files or in-memory stores) is stateless. The simplest implementation will have a signed cookie with a session id. Your application will have a hook to load the user session from the backing data store before processing the request.
  - fooyc 14y ago
    > How is storing in Redis(or any other in-memory store) any different from storing in Memcache? Memcached is a cache, not a store. Memcached happily removes keys when it feels so Please see this post where memcached author recommends not to store sessions in memcached: http://dormando.livejournal.com/495593.html http://dormando.livejournal.com/495593.html
    - irahul 14y ago
      > Memcached happily removes keys when it feels so I know about memcache key eviction. For most of the applications, that is an acceptable compromise. For others, back up the sessions in the db every n seconds as mentioned in the article. Session is loaded every request and hitting the db every request is not something I want.
  - encoderer 14y ago

by ecaron 14y ago

batista 14y ago

Why would you need 1mb session data? Or 200K for that matter?

ecaron 14y ago
Because it happens. Typically in places where it shouldn't (e.g. bad programming) but sometimes it does (e.g. enormously complex active system state.) The point is that there are solutions offering equivalent speed without that barrier, so why select a technology that has limitations?
- batista 14y ago
  "Because it happens" is not a very good technical answer. Anything can happen, even 1GB session data. But even anything approaching even half a MB of session data I would take as an architectural failure, and work to fix it, instead of letting it dictate what tools I use (ie. redis vs memcached). >The point is that there are solutions offering equivalent speed without that barrier, so why select a technology that has limitations? Because all decisions have tradeoffs and it more wise to depend on your particular use case to choose than to depend on a scenario (> 1MB session data) that can only happen with a seriously fucked up application design.
  - disordinary 14y ago
    You can easily get to 1mb if you follow the cache everything mantra that reddit is using here.
    - nostrademons 14y ago
      Why in sessions, though? Why not cache everything as uniquely-identified resources and store only the resource ID in the session? That way, resources can be shared across different sessions. I'm trying to think of a case where a user's actions (that's what's supposed to be in sessions, right?) can result in 1M of data and drawing a blank. Maybe if they're uploading images or video transiently, like in Google's Search-by-Image feature, but even that seems like a case where you should upload the image permanently and just store a reference to it in the session. Edit: Answered my own question: oftentimes it's useful to store a history of the user's recent actions in their session, because then you can build a more responsive, more tailored UI. You can save transient data they've already entered, avoid showing them promos they've already indicated no interest in, and potentially use it to predict useful things to show them next. Even then, though, it'd take 500 pageviews at 2K apiece (which is a very generous estimate) to hit a meg of data.
      - Morg 14y ago
        NO ffs NO. There is absolutely no reason to store anything but the session ID in a session cookie. You can keep it in the js, so why use session ? multiple pages ? dude you're living in the past.

masklinn 14y ago

FWIW Reddit replaced memcached with cassandra in early 2010 for caches and transient stores.

mdellabitta 14y ago
Wow, now I'm glad I read comments before the article. This article is really out of date. They've come a long way since this was written.
- masklinn 14y ago
  > This article is really out of date. Well yeah it's from early 2010, it's been 2 years. In January (2012)'s "state of the servers" post, the reddit blog noted traffic was 2 billion pageviews per month in December 2011, up from 870m or so in December 2010.

encoderer 14y ago

Even better: Don't use sessions. If there's data you're caching in a session, pre-compute it and store it in Redis (or other suitable NoSQL database). Going statless wherever possible almost always wins. (And I only say "almost" because I'm not smart enough to know if it really always wins. But for me, it has)

irahul 14y ago
> Even better: Don't use sessions. If there's data you're caching in a session, pre-compute it and store it in Redis (or other suitable NoSQL database). What do you mean by data you are caching in a session? Session is for maintaining state viz. logged in or not, language preference etc. How is storing in Redis(or any other data store) any different from storing in Memcache? > Going statless wherever possible almost always wins. Using sessions with a data store which can be accessed from multiple machines(no files or in-memory stores) is stateless. The simplest implementation will have a signed cookie with a session id. Your application will have a hook to load the user session from the backing data store before processing the request.
- fooyc 14y ago
  > How is storing in Redis(or any other in-memory store) any different from storing in Memcache? Memcached is a cache, not a store. Memcached happily removes keys when it feels so Please see this post where memcached author recommends not to store sessions in memcached: http://dormando.livejournal.com/495593.html http://dormando.livejournal.com/495593.html
  - irahul 14y ago
    > Memcached happily removes keys when it feels so I know about memcache key eviction. For most of the applications, that is an acceptable compromise. For others, back up the sessions in the db every n seconds as mentioned in the article. Session is loaded every request and hitting the db every request is not something I want.
- encoderer 14y ago