3 ms·

> 200K/sec WOW! 1 TB+ of session data; wow! My back of the envelope arithmetic indicates 10 MB -- different ballpark. My first cut guess would be to have so

> 200K/sec WOW! 1 TB+ of session data; wow! My back of the envelope arithmetic indicates 10 MB -- different ballpark. My first cut guess would be to have something as simple as my code, see what the transaction rate was, and then start up enough separate instances plus some to give the desired data rate and have something to recover from too busy or a sick server. If you have found a product that can handle the 200K/sec and the 1+ TB, then, sure, just use that and get back to work more particular to your business. So, you answered my question: What is the interest in key-value stores? The answer being, at 200K/sec transactions and 1_TB of session state data, a key-value store becomes a challenging problem. I designed a small airplane -- you need a C5A or bigger. Okay, that's a good answer. I've heard of Cassandra but have not looked into it -- likely for my startup don't need it yet.

zbentley 8y ago
To add to this, there are a few other reasons why session/auth management is often a place where bespoke/home-rolled solutions are undesirable even at an early or low-volume stage. Not only are those areas tied to security (pretty hard to get a startup off the ground if a home-rolled session/auth system lets users access other users' data), but they're also among the first to break when subjected to unexpected traffic volume, and the hardest to replace. That's not a "what if your startup becomes the next Facebook?!?!" suggestion of premature optimization; these types of breakages happen early. If you went from 10s of MB in your store and 100s of hits per second to 100s and 1000s respectively, it's possible that previously unexpected conditions would occur. I've seen cache and auth systems buckle under 10x load long before the real heavy lifters (RDBMS and backend communication points with other webservices) felt any strain at all--but only in places with esoteric or hand-built cache and auth systems. Now, none of these may be problems for you, and I hope they're not. You may have coded with that type of growth in mind (and your mention of possible future sharding indicates you've thought about it--good on you). But the sheer ease with which Redis/Memcached/pick-a-KVS can be deployed and scaled, coupled with the difficulty of replacing all call-points to a session store (easy on a disciplined, low-developer-head-count stack; much harder on a stack that had to grow in traffic/LoC/engineers in a hurry--hooking into session management to track request state all over the place is something of a siren song), is, I suspect, an additional motivation besides "we have huge amounts of data/traffic" for the widespread popularizations of off-the-shelf key/value stores.
- graycat 8y ago
  WOW! Thanks. > hooking into session management to track request state all over the place is something of a siren song Yup. The scary part is to discover that occasionally, and too often, the site gets sick, and due to non-determinism, etc., I can't easily reproduce the behavior to analyze it, find the cause, and correct it. Using a log file, etc. to trace the work in detail of each user for some 60 second interval that showed the problem is likely a siren song, big muddy swamp, but, still, maybe one of only a few choices with any promise. I'll be thinking of such tracing as I continue to get my code ready for production. Right, turn on full tracing, run for two weeks, get a 20 TB log file, and discover that somehow with the tracing on don't get the problem!!!! Nightmare scenario!!! My home brew, DIY, roll your own, just some serialization, TCP/IP sockets, and two instances of a standard .NET collection class, session state store (server) has worked fine in my testing so far. The session state server has been one of the easiest parts of my code and work so far. Some TCP/IP, serialization and two instances of a collection class should not be considered a challenging development or debugging job!! But currently I'm doing some grunt work, ..., and racing to go live on the Internet. So, currently I have 0 users per second. When I get to 1 user a second, that will be a big day. You are correct: I'm a sole, solo founder, CEO, CFO, CIO, ..., software manager, server farm manager, ..., software installer, cable puller, etc. so, could easily change all my Web site software to use a different session state store. Uh, I've already designed the Web server software and session state store so that there need be no user affinity between users and a particular executing instance of the Web server software. Generally as you suggested, since I'm a solo developer, who still does good documentation, it's easy for me to make some likely changes, much easier than if I had rows and columns of developers filling floors of a 20 story building with 300,000 square feet of space. My planning for implementation now is for 1 to a few users a second, on average, 24 x 7. I do want to look at the whole server farm architecture and determine just what would happen, where my site would break, at, say, 100 users a second, from a DDoS attack, an article in the the tech press, something viral, etc. Maybe Cisco has a box that could help me. So, if the site is overloaded, then I want the users to get a simple message back apologizing for the overload or at least just a slow response. Basically I want to know where the performance bottlenecks and limitations are and have just one bottleneck that can handle the overload gracefully and throttle what reaches everything else. Heck, the electrical system has circuit breakers; electronics commonly has overload protection; my Web site should, too! For 10, 100, 1000 users a second, each of those steps is a factor of 10 more. Point: Factors of 10 are not just different quantitatively but different qualitatively. At 100 users a second, I'll be hiring for the 100 and also planning for 1000 and 10,000. I'll be thrilled at 1 user a second, but there is a chance that people will like my work and cause usage to grow to 1000 users a second. Good to know that some people already know how to handle 200K/second!!! Thanks!

zbentley 8y ago

To add to this, there are a few other reasons why session/auth management is often a place where bespoke/home-rolled solutions are undesirable even at an early or low-volume stage. Not only are those areas tied to security (pretty hard to get a startup off the ground if a home-rolled session/auth system lets users access other users' data), but they're also among the first to break when subjected to unexpected traffic volume, and the hardest to replace. That's not a "what if your startup becomes the next Facebook?!?!" suggestion of premature optimization; these types of breakages happen early. If you went from 10s of MB in your store and 100s of hits per second to 100s and 1000s respectively, it's possible that previously unexpected conditions would occur. I've seen cache and auth systems buckle under 10x load long before the real heavy lifters (RDBMS and backend communication points with other webservices) felt any strain at all--but only in places with esoteric or hand-built cache and auth systems. Now, none of these may be problems for you, and I hope they're not. You may have coded with that type of growth in mind (and your mention of possible future sharding indicates you've thought about it--good on you). But the sheer ease with which Redis/Memcached/pick-a-KVS can be deployed and scaled, coupled with the difficulty of replacing all call-points to a session store (easy on a disciplined, low-developer-head-count stack; much harder on a stack that had to grow in traffic/LoC/engineers in a hurry--hooking into session management to track request state all over the place is something of a siren song), is, I suspect, an additional motivation besides "we have huge amounts of data/traffic" for the widespread popularizations of off-the-shelf key/value stores.

graycat 8y ago
WOW! Thanks. > hooking into session management to track request state all over the place is something of a siren song Yup. The scary part is to discover that occasionally, and too often, the site gets sick, and due to non-determinism, etc., I can't easily reproduce the behavior to analyze it, find the cause, and correct it. Using a log file, etc. to trace the work in detail of each user for some 60 second interval that showed the problem is likely a siren song, big muddy swamp, but, still, maybe one of only a few choices with any promise. I'll be thinking of such tracing as I continue to get my code ready for production. Right, turn on full tracing, run for two weeks, get a 20 TB log file, and discover that somehow with the tracing on don't get the problem!!!! Nightmare scenario!!! My home brew, DIY, roll your own, just some serialization, TCP/IP sockets, and two instances of a standard .NET collection class, session state store (server) has worked fine in my testing so far. The session state server has been one of the easiest parts of my code and work so far. Some TCP/IP, serialization and two instances of a collection class should not be considered a challenging development or debugging job!! But currently I'm doing some grunt work, ..., and racing to go live on the Internet. So, currently I have 0 users per second. When I get to 1 user a second, that will be a big day. You are correct: I'm a sole, solo founder, CEO, CFO, CIO, ..., software manager, server farm manager, ..., software installer, cable puller, etc. so, could easily change all my Web site software to use a different session state store. Uh, I've already designed the Web server software and session state store so that there need be no user affinity between users and a particular executing instance of the Web server software. Generally as you suggested, since I'm a solo developer, who still does good documentation, it's easy for me to make some likely changes, much easier than if I had rows and columns of developers filling floors of a 20 story building with 300,000 square feet of space. My planning for implementation now is for 1 to a few users a second, on average, 24 x 7. I do want to look at the whole server farm architecture and determine just what would happen, where my site would break, at, say, 100 users a second, from a DDoS attack, an article in the the tech press, something viral, etc. Maybe Cisco has a box that could help me. So, if the site is overloaded, then I want the users to get a simple message back apologizing for the overload or at least just a slow response. Basically I want to know where the performance bottlenecks and limitations are and have just one bottleneck that can handle the overload gracefully and throttle what reaches everything else. Heck, the electrical system has circuit breakers; electronics commonly has overload protection; my Web site should, too! For 10, 100, 1000 users a second, each of those steps is a factor of 10 more. Point: Factors of 10 are not just different quantitatively but different qualitatively. At 100 users a second, I'll be hiring for the 100 and also planning for 1000 and 10,000. I'll be thrilled at 1 user a second, but there is a chance that people will like my work and cause usage to grow to 1000 users a second. Good to know that some people already know how to handle 200K/second!!! Thanks!