5 ms·
Benchmarks done on only one metric are often misleading (and in commercial circumstances usually intentionally!). Would love to see visually what trade offs Dra
by baobob 4y ago
Benchmarks done on only one metric are often misleading (and in commercial circumstances usually intentionally!). Would love to see visually what trade offs Dragonfly is making to achieve the numbers from the chart at the top of the README. If excellent technical work means there really is no trade off, that would also be a great reason to chart it visually.
Also as a Redis replacement, it's not clear what durability is offered, and for most Redis use cases this is close to the first question
- YarickR2 4y agoTradeoff is inability to run on anything besides recent Linux kernels
- tyingq 4y agoI suppose because of io_uring? It does sound like extendible hashing might have downsides in some scenarios also.
- miohtama 4y agoDoes anyone run production Redis/memcached outside Linux any case?
- genewitch 4y agoProbably not many in production, but the "recent kernel" is going to be the more important part. I'm not hip to how much new stuff is backport-able, so this may preclude Ubuntu 20.04, for instance. You lose the "LTS" part if you compile your own kernel, if you manage to make it functional at all. Note: I never use kernel modules due to the issues rhel/debian and I have had with such things in the distant past.
- 10000truths 4y agoThe kernel has its own LTS releases, and running Linux in the cloud is a well tested use case. Any library not available in your distro of choice can be built from source and tracked manually, which usually works well enough.
- arinlen 4y ago> I'm not hip to how much new stuff is backport-able, so this may preclude Ubuntu 20.04, for instance. You lose the "LTS" part if you compile your own kernel, if you manage to make it functional at all. I'm not sure that concern is justified. It seems io_uring was pushed as part of the 5.1 linux kernel release, and Ubuntu 20.04 LTS seems to have been shipped with 5.4. https://packages.ubuntu.com/search?keywords=linux-image-generic https://packages.ubuntu.com/search?keywords=linux-image-gene... Also, a quick Google search pointed to io_uring patches for Ubuntu 18.04 LTS.
- citrin_ru 4y agoRedis/memcached are used in production on FreeBSD servers (and other BSD systems).
- deleted 4y ago[deleted]
- Symmetry 4y agoSad that you can't run dragonflydb on DragonFlyBSD.
- thayne 4y agoPresumably, most of the performance benefit comes from using linux io_uring, which limits it to using recent linux kernels. Of course, it is also possible there are situations where it doesn't perform as well.
- romange 4y agoHow you would suggest to demonstrate visually that there are no trade-offs? The tradeoff the way I see it - one needs to implement 200 Redis commands from scratch. Besides, I think DF has a marginally higher 50th percentile latency. Say, if Redis has 0.3ms for 50th percentile, DF can have 0.4ms because it uses message passing for inter-thread communication. 99th percentiles are better in DF for the same throughput because DF uses more cpu power which reduces variance under load. Re-durability - what durability is offerred by Redis? AOF ? We will provide similar durability guarantees with better performance than AOF. We already provide snapshotting that can be 30-50 faster than of Redis.
- pcthrowaway 4y agoDoes Dragonfly have a timeseries extension? Or does it support extensions?
- romange 4y agoIt does not. If you refer to Redis Modules we are not planning to support those - we can just implement same features within DF.
- baobob 4y agoThere is usually a tradeoff between latency and throughput, although I'm not so sure this would be true for your innovation, since you've eliminated a whole chunk of fixed overhead from the system (syscall batching). However, batching in general often implies added latency If I recall ScyllaDB has some excellent examples of demonstrating this particular tradeoff visually. A simple option would be a scatter plot where X = latency, Y = load or similar, with points coloured according to the system under test. Probably there is a better option, but this would likely be enough to sell me at least
- romange 4y agoWell in that case, I hope to have an good answer for you. Right below the benchmark graph I deliberately put a table that shows 99th percentile lantencies at *peak* throughput as reported by memtier_benchmark. 99th latency percentile of Dragonfly at its peak throughput. I put it here as well op r6g c6gn c7g set 0.8ms 1ms 1ms get 0.9ms 0.9ms 0.8ms setex 0.9ms 1.1ms 1.3ms