4 ms·
Nothing is wrong per se, but building it and keeping it was, and likely is, the wrong decision (unless you did it as an exercise or to learn). Suppose you had
by JOnAgain 8y ago
Nothing is wrong per se, but building it and keeping it was, and likely is, the wrong decision (unless you did it as an exercise or to learn).
Suppose you had decided to use Redis, Memcache, or any number other things that are available off the shelf. This would have let you get going faster, have more features, it'd be more scalable, and you would have a smaller code-footprint in your project (which is good from a maintainability perspective). Lots of web frameworks also provide integration with these stores out of the box with trivial code changes (i.e. with Rails, you just list Redis as your session store, provide the connection string to the instance, and you're done, Tomcat is about as easy, I feel reasonably certain this is true for all full featured web frameworks out there). We're talking a couple of lines of configuration here. This has the benefit of being written being battle tested by other people in production and with a variety of use cases. No guarantees it doesn't contain any bugs, but you're unlikely to be the one to encounter them first.
Also, your use case sounds relatively small. By contrast, at my last startup we were running tens of millions of cache hits per second with an ehCache/Memcache system (ehCache was a local hot-cache, cache misses went to Memcache). Other than some parameter tuning, it worked out of the box. It took me less than a day to have the whole thing working and set up for production use. Once it was set up, it never needed much attention except for the occasional Memcache param tweaks. For a vanilla use case like yours, you probably could have been running it in less than an hour and you'd never have to touch anything (with Rails and Heroku which I'm familiar with, it would take less than 10 minutes). We also get documentation and support for free (thank you stackoverflow, mailing list folks, and OS project maintainers).
So, I would wager that it took us less time to set up and maintain our fully-featured, massively scalable, distributed cache than you took to set up yours; and I'd also wager that our setup significantly outperforms yours. If we needed a feature in the future, we could also likely just add a config param vs. having to write it ourselves (5 min vs 5 days). And finally, we never had any tests for our caching or eviction management and we knock on wood never hit any bugs in either system.
Long story short, if I came across a project with a custom key-value store, it would have to be pretty darn specialized for me not to rip it out and replace with something standard.
So nothing sounds wrong with your design, but I would conjecture that the decision to design it in the first place was a mistake for any reason other than a learning experience.
- graycat 8y agoYou have some astounding performance numbers and experience. Thanks! Maybe it was a mistake, but I just started with Microsoft's .NET, ASP.NET, and IIS and no more than that as a framework. Maybe that was not so good. I had to find, download, read, abstract, and index 5000+ Web pages from MSDN -- a big delay and not so good. For the session state store, I just thought that it was an obvious, easy, routine, normal, expected little bit of programming based on TCP/IP, serialization, and collection classes. I'd heard of Redis but guessed that just reading the documentation enough to understand it and get it installed would be more work that what I did DIY. What I did was awfully easy. Yes, you have some good 5 minute solutions, but I didn't have the background to have the knowledge, infrastructure, and code that offered the 5 minute solutions -- I didn't know such things existed. No such things were clear chasing around among and reading Web pages at MSDN. Well, my session state store work is done well enough now for now and clearly easy enough for me to change to something else later if necessary. I'm learning, which was the main objective of my first post here. Thanks.