3 ms·

In a nutshell, you trade in the various guarantees relational databases provide, like transactions, relational integrity, and high levels of queryability for hi

by jerf 3y ago

In a nutshell, you trade in the various guarantees relational databases provide, like transactions, relational integrity, and high levels of queryability for higher performance on what a key/value store can do, simplicity of API, and some things that can be easier for the system to provide precisely because it is offering fewer guarantees. Particularly distribution... SQL is hard to distribute precisely because of the guarantees it offers. When to use it? You must be sure you have a case where the additional guarantees relationship databases provide are not necessary, and you either need the simplicity of usage or deployment. Or in rare cases, speed, but whereas most people seem to act as if this the main reason, I consider it a relatively poor reason. A relational DB with its guarantees turned off (i.e., no relations, no transactions, tables with just a key and a value) perform fairly closely to a key/value store for a wide range of scaling needs. There is a top end where this matters, but fewer programmers need this than think they need this. Still, it is a valid concern, and if you don't need relational guarantees it may let you scale down the instance size. Most of the cases I see where key/value stores are a good choice relate more to the simplicity than performance. I love me my Postgres, but nothing compares to the simplicity of just tossing up a Memcache somewhere and solving my problem, if that's what my problem calls for. No schema, no migrations, an API so simple my local programming language may well simply integrate it with my native associative array syntax... if you are careful to use it only where you aren't going to need relational functionality the bang for the buck can be very nice.

jmartrican 3y ago
I'm a big fan of relational DBs. But I do still have a Redis instance running in my environment. I know that my relational DB can easily do what Redis is doing but there are two big reasons that I will disclose below for why I still use Redis. 1) The system that is reading the data from Redis is a front-end system. I have concerns with security. As a front-end system, it can be accessed from the Internet and potentially hacked. As such, I make sure this system does not have access to the DB. 2) Preserve DB connections, size (affects costs of storing backups and time to hydrate), CPU, and memory. I would really like to keep my DB as trim and spry as possible. I am using traditional relational DB and not distributed. So in an effort to keep my complexity in DB management down, I offload some work to Redis. I could have created a second relational DB that has low security requirements and no relational object mapping, but I didnt think of that till now (I will explore this further in the next few days). In my case the data that is stored in Redis is accessed a lot and consists of data from multiple tables, so it was a good pick for moving to Redis. Performance wise, I suspect relational DB could perform jus as fast, but again I want to offload traffic off of the DB to not have to grow it or go distributed. If I was a better DB admin, I could probably created views or other relational DB features. But I felt it was easy to just let my API backend code (which has access to data that is also not in DB or might need to be formatted or calculated) construct the final data object then store a copy in Redis, whenever the data is updated.
zinodaur 3y ago
Great writeup. I use a KV store for work, but only because the alternative for our data size (>1PB, >1 trillion keys), sharded sql, is totally awful. Distributed KV stores like foundationDB, dynamoDB actually do offer transactions , which is a huge win and I think their main selling point. I guess I'm like you - I really don't understand why someone who had another choice would opt in to a KV store (barring something like memcache, or something really high performance using an embedded KV store).

by jerf 3y ago

jmartrican 3y ago

I'm a big fan of relational DBs. But I do still have a Redis instance running in my environment. I know that my relational DB can easily do what Redis is doing but there are two big reasons that I will disclose below for why I still use Redis. 1) The system that is reading the data from Redis is a front-end system. I have concerns with security. As a front-end system, it can be accessed from the Internet and potentially hacked. As such, I make sure this system does not have access to the DB. 2) Preserve DB connections, size (affects costs of storing backups and time to hydrate), CPU, and memory. I would really like to keep my DB as trim and spry as possible. I am using traditional relational DB and not distributed. So in an effort to keep my complexity in DB management down, I offload some work to Redis. I could have created a second relational DB that has low security requirements and no relational object mapping, but I didnt think of that till now (I will explore this further in the next few days). In my case the data that is stored in Redis is accessed a lot and consists of data from multiple tables, so it was a good pick for moving to Redis. Performance wise, I suspect relational DB could perform jus as fast, but again I want to offload traffic off of the DB to not have to grow it or go distributed. If I was a better DB admin, I could probably created views or other relational DB features. But I felt it was easy to just let my API backend code (which has access to data that is also not in DB or might need to be formatted or calculated) construct the final data object then store a copy in Redis, whenever the data is updated.

zinodaur 3y ago

Great writeup. I use a KV store for work, but only because the alternative for our data size (>1PB, >1 trillion keys), sharded sql, is totally awful. Distributed KV stores like foundationDB, dynamoDB actually do offer transactions , which is a huge win and I think their main selling point. I guess I'm like you - I really don't understand why someone who had another choice would opt in to a KV store (barring something like memcache, or something really high performance using an embedded KV store).