9 ms·
Remote, in-memory key-value stores: An idea whose time has come and gone? [pdf]
- stcredzero 8y agoInstead, data center services should be built using stateful application servers or custom in-memory stores with domain-specific APIs, which offer higher performance than RInKs at lower cost. I'm in the midst of implementing a stateful application server-cluster with custom in-memory stores with domain-specific APIs. Basically, it's a way of implementing a game loop and a multiplayer game by specifying lambdas going against in-memory entity databases and in memory spatial databases. However, to do this, I have gone from using PostgresSQL to the Badger fast key-value store as a database. I'm mainly motivated by the desire to keep everything in golang and eliminate all outside dependencies. That said, what I'm doing is increasing the granularity of the server processes. The server processes are composed of Actor-like things which are responding to async input queues. So my experience both confirms and contradicts some of the the things said in the paper.
- kitotik 8y agoI’m looking more and more at Erlang/OTP for many of these same reasons. Ets/Dets/Mnesia seems to address a ton of typical stateful app use cases right out of the box, and it was built for the Actor model with mass quantities of fine-grained processes you mentioned. It’s cool to hear a similar approach can be done in golang without a ton of external deps.
- jrumbut 8y agoWould you mind sharing what made you go with a stateful application server? I've never found myself in the situation of wanting to go that direction.
- stcredzero 8y agoWould you mind sharing what made you go with a stateful application server? I'm implementing a MMO game loop. Even though I am exposing the API as what amounts to a bunch of lambdas, under the covers, I am holding onto state and mutating it, basically for performance. EDIT: To put it another way, there is a level of abstraction at which my system is stateless. To implement that with enough performance, I have to hold onto state, so I can deliver data to the implementation of those abstractions in a timely fashion.
- jrumbut 8y agoInteresting perspective, thanks!
- zemo 8y agothe main function of a multiplayer game server is to contain the game state. Depending on the game, it may also run a simulation. You couldn't, for example, write a database record every time someone shot a gun in an FPS, the whole thing would come to a crawl. And game clients generally only care about the present. Only the current game state is of interest to the client in most situations; the past is rarely queried. The outliers here are things that are naturally transactional and persistent like character progression and item ownership; those things you'd stick in a database.
- jrumbut 8y agoMakes sense thanks!
- zemo 8y agoha, I ... also am working on a stateful game server in Go, the multiplayer servers that power Jackbox Games' line of games. I don't currently need disk persistence though, I just do it all in memory. It's also way easier than an MMO because the game is entirely instanced, so you can just partition the data by the game instance.
- estsauver 8y agoJust wanted to say playing Jackbox with my family has been really enjoyable, and it's really great how easy it is to get everyone set up and started.
- neeleshs 8y agoIt's a balance between how inprocess stores bring instability to the main application vs cost of n/w & marshalling. Out of memory errors crashing the application, instead of providing a degraded experience etc. Answer as usual is, "it depends".We moved from inprocess to remote, because we valued application stability more than raw performance.
- kahnjw 8y agoQuestion: for data keyed by user, did you find you'd actually be better off using an distributed in-process cache than a remote cache? I find that hard to believe but maybe I'm just not up to speed on how advanced these caching solutions are.
- neeleshs 8y agoOur use case was not for data keyed in by user, but system generated events that were cached in memory (on a remote cache) for deduplication. We started with distributed in-process cache (Apache Ignite), but found out that failure modes of the cache were not acceptable for the core application. Hence we moved the cache out, to Redis. App behaves far better now.
- klodolph 8y agoThis is a misleading title: it’s talking specifically about remote in-memory key-value stores. I thought that this would be against e.g. Badger.
- nostrademons 8y agoThe stateful application server is actually a common architecture for initial prototypes of new apps, because of its simplicity. Hacker News, Mailinator, Plenty of Fish, and IIRC StackOverflow were all built with this architecture. Where it really falls down is when you have a team of developers all trying to add new features. Code changes a lot more frequently than data. With stateful app servers, every time you deploy new code you need complex and error-prone routines to bring the new process up, have it read the existing state, make sure it's consistent with the new features, and if not migrate it to the new format. With RInKs, a huge benefit is that you can take the application up and down and all persistent state is already there, and usually the RInK itself handles persistence & error-checking (because it enforces a common data model itself). For minor UI tweaks and per-request features (which are a large portion of changes in the early part of the product lifecycle), you don't need to do a migration at all, the new version just magically appears. For more complex data format changes, you make it so that the application code can read both the old and new formats, converts old to new if necessary, and writes back only in the new format, and then you can drop the old format once it is no longer present in the persistent store. This lets you do the upgrade piecemeal, and avoids corrupting your entire data store if you do it wrong (because the architecture functions as a canary on every deployment, and you can rollback with zero to a handful of corrupted records if things aren't working properly). The other situation where stateful app servers are useful is when scalability is much more important than change velocity. This may be why Google is coming out with this paper now. This would never have flown within Google when I was there (2009-2014), but I've heard that the focus on maintenance, performance, and architectural improvements is now much greater than that on new features, so it may be appropriate for the Google of today.
- sonnyblarney 8y ago"every time you deploy new code you need complex and error-prone routines to bring the new process up, have it read the existing state, make sure it's consistent with the new features, and if not migrate it to the new format." Pardon my ignorance ... but wouldn't a stateful model that used mem as true 'cache' not avoid that problem? As if to say the in-memory store is not 'pre loaded' persey, rather, it's loaded incrementally on a on-demand basis from the app itself? So 'rebooting' has the effect of flushing the cache thats built up over time after reboot, hopefully seamlessly? Or am I (probably) missing something?
- akkartik 8y agoSounds like their argument hinges on the efficacy of systems like https://eng.uber.com/intro-to-ringpop https://eng.uber.com/intro-to-ringpop
- sytse 8y agoYes, Ringpop was reference number 8 in the article. It is interesting that https://github.com/uber/ringpop-go https://github.com/uber/ringpop-go says: "This project is no longer under active development" So the time of Redis has passed but the successor has passed away as well?
- roskilli 8y agoBasically it's been deprecated in favor of using projects that are more mature like Apache Helix (but conversely more operationally intense to get started, requires Zookeeper and a bunch of things on top if you want to use it in a non-Java setting, i.e. with Go).
- Tharkun 8y agoI've really grown to dislike ZooKeeper. While incredibly useful, it's pretty awful. It's nearly impossible to secure in any non-trivial setup. And it's inane requirement to reserve a physical disk [1] for its transaction log is almost impossible to fulfil with virtualized hardware. [1] From the manual: > ZooKeeper's transaction log must be on a dedicated device.
- hadsed 8y agoIs this not the assumption of many SQL databases that make certain guarantees about complying with ACID? I'm not a DB expert but feels like something I've read before.
- tatersolid 8y agoNot since the 1990s
- graycat 8y agoI don't get it: Key-value stores is a topic that just won't go away. I was interested in key-value stores, did a little work, solved the problem for my needs, and don't care anymore. Sure maybe there can be some high end, challenging aspects, but what I did was dirt simple, childishly simple to do, blindingly fast, with plenty of capacity -- maybe will cover 75% of the need for everyone? Heck, if we want something with really deep functionality, we are just back to relational database, right? So, I wanted a key-value store for session state for users of the Web site of my startup. When a user sends a page back to my server, it has in a "hidden field" the value of an encrypted token with identifies the user's logical session. And I have in VB.NET a class for the data I want to keep on that user, their session state. So, in my VB.Net Web server code, I have defined some classes: (1) An allocated instance of one class sends a new or updated key-value pair to the key-value server. (2) An allocated instance of another class requests from the key-value server the value that corresponds to a key. (3) An allocated instance of a third class gets the value back from the key-value server. For communications, just take an instance of a class, serialize it (thank you VB.NET), copy it to a byte array, and send that via old TCP/IP sockets. Then the receiver gets the byte array and does a deserialization to get the corresponding object instance. The key-value server is based almost entirely on just two instances of a standard .Net collection class. One instance has the key-value pairs where, right, the key in the pair is the key in the collection class. The other instance has a time-date pair as key and for a value a key from the key value store. Here there is the time-date when the corresponding key-value was last written or read and is used for session time outs. So, for session time outs, just read this time-date and session key instance in ascending order on the time-date key and for any session time-outs delete that collection class element and also the corresponding element in the other collection class. It's a few hundred lines of code and was a simple exercise in TCPIP, object instance de/serialization, and collection class usage. I also have code to permit sending some system management commands to the key-value server. It runs as just a console application. The code is single threaded, and I'm counting on just the standard TCP/IP FIFO queue to hold the backlog of work to be done. All the data is held in main memory so should be blindingly fast. Some simple arithmetic indicates that the main memory needed is not very large for even a quite busy Web server computer. The session state server can serve any reasonable number of Web server computers, and a particular user doesn't need session affinity with a particular Web server. Looked simple and easy to me. How much of the need will something so simple satisfy? What needs will such a simple solution not satisfy?
- nullwasamistake 8y agoI'm astounded at the number of developers in my own company that reach for Redis when they just need a cache. It's a huge waste of effort for far worse performance and less reliability than something like Java's Caffiene. Stateful servers run into huge problems when other people try to access them from non-static ip (usually sticky sessions are IP based). We, at least in apps I work on, stuff all the shared state into JWT tokens to avoid this. Everything else that needs to be cached goes into an in-memory in-app store. If you're using a "fast" language to start with, you won't need more than a dozen servers unless you're at Google scale so cache hit rates are good even though each instance maintains it's own. Again if you're using a "fast" language memory overhead from in app storage isn't much more than putting things in a dedicated KV store. There's really no point to it when the performance drop is usually 20x. We use DropWizard and Vert.X extensively, and most of our endpoints can handle many thousands of requests per second. The ones that use the cache instead of hitting the DB every time hit upwards of 1 million requests/sec. You're very lucky to get 50k with any remote KV store in the same app. Most load balancer strategies send many requests in a row from a client to the same backend server anyways for TCP performance reasons. The only reason for a KV cache is if you're running a slow language like Ruby, PHP, Python, where the memory overhead for objects is large and the speed of a remote cache is roughly the same as one in memory. If you find yourself needing a cache for good enough performance you shouldn't be using these languages in the first place. In memory caches don't work well when you need 100+ server instances to handle the load anyways
- zemo 8y ago> Stateful servers run into huge problems when other people try to access them from non-static ip (usually sticky sessions are IP based). We, at least in apps I work on, stuff all the shared state into JWT tokens to avoid this. They're not saying that an application server should never query another server, they're saying that the server that functions as a state container should host a domain-specific api for manipulating the state. See figure 2 b; the frontend and stateful application server layers are separate layers.
- deleted 8y ago[deleted]
- 8y ago
- happythought 8y agoAtul Adya and Daniel Myers work on Slicer[1] so they have some bias. [1] https://www.usenix.org/system/files/conference/osdi16/osdi16-adya.pdf https://www.usenix.org/system/files/conference/osdi16/osdi16...
- dbmikus 8y agoWould this just be like using RocksDB or something like that with extensions to auto-shard and balance between instances? That's what it looks like to me, but I could be misunderstanding it.
- analyst74 8y agoAnother benefit in-memory cache vs remote key-value store is the reliability. Redis/memcached servers are super reliable, but at scale, you'll run into non-trivial amount of failures when communicating to those servers, even a sub-second blip in network connections could result in hundreds of failures. Now you have to deal with retry, ensure your application logic is failure tolerant, worry about timeout, consider impact of temporary delays in redis to your application server due to skyrocketing threads waiting for results and all that fun stuff.
- alexchamberlain 8y agoAt scale, don't you have the same problems with your in memory caching solution (specifically in the distribution and coordination parts)?
- graycat 8y agoNice. Sounds like the clients should be willing to retry any such remote communications due if only due to little blue ET men crawling through the server farm LAN cables.
- philihp 8y agoThe author has missed a very very important feature: Redis as a distributed lock manager. Redis’s SETNX is a very fast and very cheap solution to resolve race conditions and dining philosopher problems among shards and nodes in large clusters.
- saurabhsharan 8y agoI assume you've read https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html https://martin.kleppmann.com/2016/02/08/how-to-do-distribute...
- yenwel 8y agoEvent sourced persistent actors in cluster mode is where it's at!
- d--b 8y agoThe title is annoying on purpose... This is a different, more complex and more-efficient-in-some-cases approach.
- netik 8y agoArticle reads like an overview of 2008 technologies, but offers no actual, new solution to the existing solution that fuels Twitter and other social media sites.
- pwpwp 8y agoFunny that this doesn't mention parse-free data structures like FlatBuffers or Cap'n Proto that don't need unmarshalling. https://google.github.io/flatbuffers/ https://google.github.io/flatbuffers/ https://capnproto.org/ https://capnproto.org/
- jdalsgaard 8y agoLove it; "if you fetch it over a socket, it ain't no cache"
- hinkley 8y agoWhen I'm explaining KV to a junior dev, I do it something like this: Memcached got big during a confluence of two problems that have both been solved now. First, we had a bunch of big, semi-stateful web applications built in programming languages with GC, and server memory got bigger than you could reliably collect without long pauses. Pushing some of your data into a separate memory space raises the ceiling considerably. Secondly, during this ten year window, ethernet cards were half an order of magnitude faster than hard drives. Putting stuff on another machine could be faster than sending it to swap, a memory mapped file, or some more sophisticated data store (like a database). We don't have to struggle with these now, and half the time we avoid the first one altogether. They still have lots of places they are used, but you are way better off working to cache inbound requests instead of outbound requests. That lets you move a bunch of caching to the edge of your network, or to the user agent.
- bakhy 8y agoI'm currently working on a side project, a distributed in-process key/value store for .NET Standard, which might be interesting in this context. I hope you'll pardon the shameless plug, but I'm interested in getting feedback. Particularly, what to focus on, what use cases, missing features... And of course, if you see any issues with the approach. https://github.com/jbakic/Shielded.Gossip https://github.com/jbakic/Shielded.Gossip It needs more work, but I think the eventually consistent part could already be useful for a simple distributed cache layer atop a classic relational DB.
- PunksATawnyFill 8y agoIt's "key/value" store.