7 ms·
Redis on Acid: 0.5M ops/sec, 1ms latency and ACID compliance
- jjawssd 9y agoAll I see with uBlock is a black screen. I have to whitelist over a dozen ad trackers to see the content. Several ad scripts pull in even more scripts from external domains. Try it yourself.
- janpieterz 9y agouBlock Origin working fine here. 2 requests blocked.
- jjawssd 9y agoAre you using the uBlock defaults? I have blocked everything above "Regions, languages" in the 3rd-party filters tab except the experimental filter.
- pravula 9y agoWorks fine for me.
- TimNN 9y agoI think the problem is that some block list includes `:root .scroll-top` as a blocked Element, which triggers for the `<body>` element on this page.
- Mister_Snuggles 9y agouMatrix blocks all of the trackers and renders this page seemingly perfectly.
- deleted 9y ago[deleted]
- segmondy 9y agoVery impressive, especially how it doesn't seem to matter much on the read/write ratio. I've only used redis to cache, do folks really use it as a DB?
- edem 9y agoI'd also like to know what else is it used for.
- tgtweak 9y agoHigh traffic atomic push/pop between multiple clients (often used for jobs like kue.js) write coalescing - caching individual writes for later bulk inserts into a db to help insert rates. Simple counts and counters which can be incremented and then purged/reset on interval. Simple Pub/sub for medium throughout systems. Many more applications than simple key/value.
- ilkkao 9y agoI'm using Redis as the main data store for my semi-serious chat app project. For better or worse, all tables, including the userdb is stored in Redis. In general, an extra layer on top of Redis is a must to make it even simple searchable database. Especially indices can't be ad-hoc implemented with separate Redis commands. Only transactions or Lua snippets that update both data and related indices atomically can avoid data corruption. For my own use, I wrote: https://github.com/ilkkao/rigidDB https://github.com/ilkkao/rigidDB It supports indices and strict schema for the data. I think somebody has described Redis at some point as a database-SDK. Makes sense to me.
- le-mark 9y agoI used redis' lua scripting to implement a per user/account cache; I wanted to provide a memcached instance per account, but also enforce a limit on cache size so a single account couldn't cache GBs of data and detoriate the service for everyone. I used hashes to track items and their sizes per account, and simply calculated the available size of all live objects on insert.
- neomantra 9y ago
- patkai 9y agoI don't like to admit but I still don't get it, could I use Redis as a main webapp backend?
- znpy 9y agoLong story short, "ACID" is what a datastore has to provide in order to be eligible to be called a "proper database". So yeah, I guess you could use redis as main datastore.
- mayank 9y agoYou definitely could, with some pretty big caveats: * you can tolerate some data loss in case of system crashes/power failures/etc, because redis only flushes data to disk periodically [1] * your dataset can fit in RAM, since redis is an in-memory datastore [1] https://redis.io/topics/persistence https://redis.io/topics/persistence
- kristoff_it 9y agoUsing AOF you have better durability, you can read about it in the link you provided.
- mayank 9y agoAs far as I know, AOF is still not the default setting. It's important to point this out, or otherwise Redis will suffer the same fate as early MongoDB, which had a similar default persistence model which many users didn't fully understand.
- immad 9y agoRedis won't function till it has loaded all data from disk to memory. So if your webapp doesn't have too much data then possibly. Also since Redis is memory backed you need more RAM then data. This can get very costly. Another annoyance is that Redis is single process single threaded so you really have to avoid running long running queries unless you do extensive manual sharding. (Disclaimer: it's been a few years since I had to think about these constraints so maybe some are removed in more recent versions of Redis)
- znpy 9y agoQuestions I would like to see answered: - How does oss redis compare to enterprise redis ? - What are the differences when running the same benchmarks with both oss redis and enterprise redis ? - what is the marginal utility of an additional cpu core/thread ? that is, what happens if I run those benchmarks on an AMD ThreadRipper ?
- koolba 9y agoI'm curious on all of these as well. Also, is this a first party (ie antirez) piece of software? If not, has he blessed calling something "Redis Enterprise"?
- dvirsky 9y agoRe the first question - enterprise Redis includes, besides the management utilities you can expect from such a product - a different cluster architecture (that actually predates Redis cluster) and cluster manager; It doesn't require a cluster aware client as it uses a proxy tier (that also does implicit pipelining and other stuff to increase throughput); And the next version of it will include some extra features like flash support, extra modules, etc. Re the second and third questions - I'm afraid I cannot answer. I hope the author of the post will reply tomorrow morning (it's the middle of the night here).
- 9y ago
- sgmansfield 9y agoIt's worth noting that they're running this benchmark on an absolutely enormous (and expensive) instance: https://aws.amazon.com/ec2/instance-types/x1/ https://aws.amazon.com/ec2/instance-types/x1/
- mkj 9y agoKeep reading - AWS latency was too high so the benchmark was run on some bare Dell hardware.
- deleted 9y ago[deleted]
- nosefouratyou 9y agoI would think that something like hyperdex/warp[1] would be better for that use case. [1] http://hyperdex.org/ http://hyperdex.org/ [2] https://www.cs.cornell.edu/people/egs/papers/hyperdex-sigcomm.pdf https://www.cs.cornell.edu/people/egs/papers/hyperdex-sigcom... [3] http://rescrv.net/papers/warp-tech-report.pdf http://rescrv.net/papers/warp-tech-report.pdf
- lobster_johnson 9y agoHyperdex is no longer maintained [1]. While the technology is impressive, the author seems to have lost interest, and is now working on something called Consus [2]. Hyperdex's problem all along was that the author — a very talented developer from what I can tell — seems more invested in his projects from the perspective academic research (he's at Cornell) than in delivering a practical, living open source project. He tried to form a company around Hyperdex (the transactional "Warp" add-on thing was commercial) even though nobody seemed to be using it; and he was the sole developer. Unfortunately, as interesting as Consus is, history seems to be repeating itself there. But yeah, Hyperdex seemed to have real potential at one point. It was the only NoSQL K/V store (at the time) that had transactions. [1] https://github.com/rescrv/HyperDex/issues/233 https://github.com/rescrv/HyperDex/issues/233 [2] https://github.com/rescrv/Consus https://github.com/rescrv/Consus
- misframer 9y ago> It was the only NoSQL K/V store (at the time) that had transactions. What about FoundationDB?
- lobster_johnson 9y agoNot open source, though.
- misframer 9y agoWarp wasn't open source either.
- cjhanks 9y agoI guess I don't understand who requires less than 1ms latency on ACID writes in a system accessible only through socket interfaces. Even so - isn't this benchmark simply pushing the requirement of fast disk syncs onto a fast flash drive designed for DMA? I mean, okay.. I guess... did customers actually think the networking wasn't the latency culprit?
- b34r 9y agoWhy wouldn't they say 500kops/s? Using a small number like 0.5 is just bad marketing.
- overcast 9y agoBecause anyone who actually understands any of that, knows it's an impressive number regardless. They aren't "marketing" this to the general public.
- aaronblohowiak 9y ago"almost always" atomicity means "not atomic". (if i guess correctly, there is no stage/commit phase for data mutations in the aof persistence, so i believe incremental changes are written to the AOF as the script runs so you can have partially-applied updates on restore if your redis process dies in the middle of executing a watch/multi/exec or lua script.) also, instances die all the time.
- gopalv 9y ago> also, instances die all the time This where the durability argument has a problem as well. At a first glance, this looks like an un-replicated system, which means that the loss of an instance is an availability nightmare. The worrying quote from the article for me is this one >> With Redis Enterprise, it is easy to create a master-only cluster and have all shards run on the same node Another node has to be brought in and attached to the same network disk to restore access to that key range? 500k+ ops/sec is nothing to laugh about on a single node with 1:1 read-write ratios, however the fragility of this system is concerning. Half a decade ago, I was working with row-level atomic ops in ZBase/Membase (set-with-cas[1]), which gets away with using replication instead of an ssd backing the durability of operations + an fsync - the 99% latencies were at 3-4ms, but the scalability and availability were baked in. [1] - https://github.com/zbase/documentation/wiki/Data-Integrity https://github.com/zbase/documentation/wiki/Data-Integrity
- lpinca 9y agoI'm not sure how the append only mode works but I tried to run a script (appendfsync directive set to always) which sets a new key in an endless loop and kill the server in the middle of the execution. No changes are written to the AOF so I guess the operation is atomic.
- antirez 9y agoHello, I'm not sure of the setup Redis Labs is using, but vanilla Redis AOF does not allow partially-applied updates. Not for Redis transactions (MULTI/EXEC) nor with scripting. The same happens in the replication channel. In order to enforce this, Redis goes a very long way to avoid partial applications of Lua scripts, more info are in the EVAL page and in the -BUSY error in case of scripts not returning that already made writes.
- latch 9y agoFWIW, I just get a blank page in FF with uBlock running. I believe it's the "naked-social-share" which various 3rd party extensions (Fanboy's) is blocking.
- dis-sys 9y agoStopped reading after I saw the following statement - “All or nothing” property is almost always achieved, excluding cases like... Almost always achieved? That actually means it is "never really achieved", fixed for you. It is just sad that cheap marketing materials like this one are keep being pushed to the front page of NH.