4 ms·

Honestly, I'm still not sure, why would I use something like RocksDB instead or in addition to plain PostgreSQL/MongoDB/Redis instances. I don't work with a lo

by Adiqq 4y ago

Honestly, I'm still not sure, why would I use something like RocksDB instead or in addition to plain PostgreSQL/MongoDB/Redis instances. I don't work with a lot of data, but typically my decisions base on basic factors and purpose: PostgreSQL - SQL, structured data, cannot scale horizontally MongoDB - NoSQL, unstructured data Redis - key-value, distributed cache I get it that you can replace storage engine and you can theoretically get more performance, but in practice compatibility and standardization is more important, because a lot of products (including third-party) will already use PostgreSQL/MongoDB/Redis, so it's no-brainer to use it as well for your solution. However for me to pick RocksDB or some other, new, shining database/storage engine, there would have to be more compelling reasons.

jzelinskie 4y ago
Unless you are building a database, these embedded KV store libraries are less likely to be the best solution the job. If you are considering them for an app that isn't a database, you should also take a long, hard look at SQLite first. What's also interesting is the trend of newer distributed "database systems" like Vitess[0] or SpiceDB[1] that forego embedded KV stores and instead reuse existing SQL databases as their "embedded database". Vitess leverages MySQL and SpiceDB leverages MySQL, PostgreSQL, CockroachDB, or Spanner. Systems built this way get to leverage many high-level features from existing databases systems such that they can focus on innovating in even higher-level functionality. In the case of Vitess, it's scaling, distributing, and schema management of MySQL. In the case of SpiceDB, it's building a database specifically optimized for querying access control data in a way that can coordinate with causality across multiple services. [0]: https://github.com/vitessio/vitess https://github.com/vitessio/vitess [1]: https://github.com/authzed/spicedb https://github.com/authzed/spicedb
- foota 4y ago
  Me: that's Zanzibar, innit? Their GitHub repo: based on Zanzibar
zarzavat 4y ago
In your list RocksDB is most like Redis, but even faster because the data doesn't have to leave the process. Think of it as a high performance sports car like a Ferrari. It's not good at taking the kids to school or buying groceries. But if you need to prioritise performance at the expense of all other considerations then it's exactly what you need.
Xeoncross 4y ago
Like S3 or Redis, RocksDB is much more performant when you don't need the query engine and want to have highly compact storage with fast lookups and high write throughput. Storage engines are different levels of complexity based on the query requirements. Simple K/V stores can run circles around Postgres/MySQL as long as you don't need the extra features.

by Adiqq 4y ago

jzelinskie 4y ago

Unless you are building a database, these embedded KV store libraries are less likely to be the best solution the job. If you are considering them for an app that isn't a database, you should also take a long, hard look at SQLite first. What's also interesting is the trend of newer distributed "database systems" like Vitess[0] or SpiceDB[1] that forego embedded KV stores and instead reuse existing SQL databases as their "embedded database". Vitess leverages MySQL and SpiceDB leverages MySQL, PostgreSQL, CockroachDB, or Spanner. Systems built this way get to leverage many high-level features from existing databases systems such that they can focus on innovating in even higher-level functionality. In the case of Vitess, it's scaling, distributing, and schema management of MySQL. In the case of SpiceDB, it's building a database specifically optimized for querying access control data in a way that can coordinate with causality across multiple services. [0]: https://github.com/vitessio/vitess https://github.com/vitessio/vitess [1]: https://github.com/authzed/spicedb https://github.com/authzed/spicedb

foota 4y ago
Me: that's Zanzibar, innit? Their GitHub repo: based on Zanzibar

zarzavat 4y ago

In your list RocksDB is most like Redis, but even faster because the data doesn't have to leave the process. Think of it as a high performance sports car like a Ferrari. It's not good at taking the kids to school or buying groceries. But if you need to prioritise performance at the expense of all other considerations then it's exactly what you need.

Xeoncross 4y ago

Like S3 or Redis, RocksDB is much more performant when you don't need the query engine and want to have highly compact storage with fast lookups and high write throughput. Storage engines are different levels of complexity based on the query requirements. Simple K/V stores can run circles around Postgres/MySQL as long as you don't need the extra features.