3 ms·
A previous version of MeiliSearch was using RocksDB and we were having a lot of trouble using it, a lot of setup to do to make sure that we were not killed by t
by Kerollmops 5y ago
A previous version of MeiliSearch was using RocksDB and we were having a lot of trouble using it, a lot of setup to do to make sure that we were not killed by the OS due to OOM or even fixing a lot of strange segfault by patching the RocksDB library itself...
RocksDB doesn't support transaction but views in the database, which means that if you are indexing, writing into the database and that any event makes your program to stop unexpectedly, you can't just start your program and use the data like this as it could be corrupted.
This is why at MeiliSearch we prefer using LMDB, even in case of an unexpected crash, a reboot is instant and valid, you just need to restart the indexing you were previously doing and can serve requests to the users with the previous version of the database.
Also as you can see [0], the benchmarks between LMDB and RocksDB is very clear. I understand that it is maybe not reading the database that takes time on your side but it is on our side, combined with the set operations between sets of internal documents ids.
[0]: http://www.lmdb.tech/bench/optanessd/imdt.html http://www.lmdb.tech/bench/optanessd/imdt.html
- karterk 5y agoI see. Perhaps, because we're using the native CPP client we faced no such crash issues with RocksDB. We have also handled transactions at a higher layer. All the best with MeiliSearch!