3 ms·

Language / framework and DB choice does matter for performance. A lot. https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=db https://www.teche

Language / framework and DB choice does matter for performance. A lot. https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=db https://www.techempower.com/benchmarks/#section=data-r18&hw=... As you can see, the fastest Python implementation is just 11% of the top solution in Rust. And your key/value store choice matters a lot. Redis is well known, but it's much slower than Tarantool or LMDB. Which is slower than RocksDB or Aerospike (though take this claim with a grain of salt, they can perform differently under different loads (writes, reads, updates, deletes) and different number of concurrent requests).

enitihas 7y ago
Redis is slower than LMDB? This sounds very hard to believe, since redis operates purely in memory, and all the data structures are designed with that in mind. LMDB does disk writes for every key value set operation. So I find it hard to believe redis is slower than LMDB
- james_s_tayler 7y ago
  Redis writes it's replication log to disk.
  - enitihas 7y ago
    That happens on a configured time interval, not on every request unlike lmdb.
- kjeetgill 7y ago
  I don't know for sure, but I'd imagine it's dominated by network. Which kinda the point of the article: benchmarks need detail and context to interpret well.
kiadimoondi 7y ago
Some of those DBs solve different use cases than others. I wouldn't use Redis as an embedded DB (unless benchmarking indicated it fit my needs better), as the user of that software benefits most in using it with multi-machine access in mind. Embedded use cases are where RocksDB, LMDB, TokyoDB/KyotoDB, SQLite, etc. come into play. Keeping the use case in mind (and it's possible evolution) will help pick the best tool for the job.
jugg1es 7y ago
do you have evidence showing redis performance vs tarantool and lmdb? I'd be interested to see that.
kjeetgill 7y ago
I found myself completely nodding along to the first half of your comment only to find myself scratching my head to the last half. I'm unfamiliar with Tarantool, but comparing a "server" database like redis with embedded ones like lmdb or rocksdb is so strange. It's comparing different engines to other whole cars. NB: For anyone not in the know, "server" vs embedded here is about how your processes communicate with the DB, not about the type of hardware they're suitable to run on. An embedded database tends to be more of a library that attaches to a file or directory, often from a single process or occasionally from multiple processes on a single host. "Server" databases tend to be connected to from processes on a number of servers over the network. I keep using "server" in quotes here because I don't think I've seen that word used to make the distinction vs embedded databases reliable in literature.
jstrong 7y ago
I don't think your "11%" is the best representation of the data. black sheep (python): 101,508 req/sec actix (rust): 886,499 req/sec yes, 101508/886499 = 0.11 but your "x is % of y" doesn't seem to capture the relationship very well here IMO. you could also say, the rust library processes 773% more requests per sec. or, the python library processes 89% fewer requests per sec. (using: percent change = d2/d1-1) personally I prefer multiples in terms of the larger thing. actix does over 8x the requests per/sec as the python lib.

enitihas 7y ago

Redis is slower than LMDB? This sounds very hard to believe, since redis operates purely in memory, and all the data structures are designed with that in mind. LMDB does disk writes for every key value set operation. So I find it hard to believe redis is slower than LMDB

james_s_tayler 7y ago
Redis writes it's replication log to disk.
- enitihas 7y ago
  That happens on a configured time interval, not on every request unlike lmdb.
kjeetgill 7y ago
I don't know for sure, but I'd imagine it's dominated by network. Which kinda the point of the article: benchmarks need detail and context to interpret well.

kiadimoondi 7y ago

Some of those DBs solve different use cases than others. I wouldn't use Redis as an embedded DB (unless benchmarking indicated it fit my needs better), as the user of that software benefits most in using it with multi-machine access in mind. Embedded use cases are where RocksDB, LMDB, TokyoDB/KyotoDB, SQLite, etc. come into play. Keeping the use case in mind (and it's possible evolution) will help pick the best tool for the job.

jugg1es 7y ago

do you have evidence showing redis performance vs tarantool and lmdb? I'd be interested to see that.

kjeetgill 7y ago

I found myself completely nodding along to the first half of your comment only to find myself scratching my head to the last half. I'm unfamiliar with Tarantool, but comparing a "server" database like redis with embedded ones like lmdb or rocksdb is so strange. It's comparing different engines to other whole cars. NB: For anyone not in the know, "server" vs embedded here is about how your processes communicate with the DB, not about the type of hardware they're suitable to run on. An embedded database tends to be more of a library that attaches to a file or directory, often from a single process or occasionally from multiple processes on a single host. "Server" databases tend to be connected to from processes on a number of servers over the network. I keep using "server" in quotes here because I don't think I've seen that word used to make the distinction vs embedded databases reliable in literature.

jstrong 7y ago

I don't think your "11%" is the best representation of the data. black sheep (python): 101,508 req/sec actix (rust): 886,499 req/sec yes, 101508/886499 = 0.11 but your "x is % of y" doesn't seem to capture the relationship very well here IMO. you could also say, the rust library processes 773% more requests per sec. or, the python library processes 89% fewer requests per sec. (using: percent change = d2/d1-1) personally I prefer multiples in terms of the larger thing. actix does over 8x the requests per/sec as the python lib.