5 ms·

Can anyone explain why this might be exciting? (genuinely curious)

by netfl0 7y ago

thanatos_dem 7y ago
Ever wanted a database with all the operational complexity of Kafka, and more? Well now you can have it! It’s exciting as someone more in the SRE side, because you’d need to pay be an obscene amount to support this.
- slowmovintarget 7y ago
  It's like the bulked-up Cartman of key-value stores. Edit: Nope, it's a relational DB
- diehunde 7y ago
  Isn't that "side" just a bunch of clicks on a wizard on some cloud provider?
solatic 7y ago
There are companies that fit the following operational niche: * Need to run Kafka (for domain-specific reasons) * Need a relational database, but not necessarily a large or performant one A project that lets you "buy a Kafka cluster, get the relational database free" is strictly simpler and cheaper to operate (in theory...) compared to operating both a Kafka cluster and a traditional relational database.
adamnemecek 7y ago
It uses kafka as an operational primitive. Kafka is somewhat complicated but it also solves real problems. I can see a fast, distributed db built on top of kafka.
- qaq 7y ago
  There fast distributed DBs and they are generally built on top of distributed ordered key-value stores. Many of them do not require to run external components like zookeeper. What is the advantage of running one on top of Kafka?
  - Godel_unicode 7y ago
    The main benefit I have heard is that some people already have Kafka.
    - qaq 7y ago
      in what sense is it a benefit?
random023987 7y ago
If you want an asymmetric persistence layer where all writes are documents placed in a Kafka topic, and all reads are via relational queries. It's a distributed, eventually consistent, highly available ledger that can be queried with SQL.
- Scarbutt 7y ago
  Is not eventually consistent, it uses https://omid.incubator.apache.org/ https://omid.incubator.apache.org/.
manigandham 7y ago
It's exciting in the sense that you can apply all these open-source projects to create a very slick lightweight relational database that can replicate changes globally across unlimited instances by using a Kafka topic as the WAL log. Don't actually use this in production. Use a real database that does what you want, and use the native replication features. If you need to replicate to heterogenous or alternative sources or just want a stream of all changes then you can use Kafka with any number of existing database connectors that automatically read and export changes.

Can anyone explain why this might be exciting? (genuinely curious)

by netfl0 7y ago

Can anyone explain why this might be exciting? (genuinely curious)

Ever wanted a database with all the operational complexity of Kafka, and more? Well now you can have it! It’s exciting as someone more in the SRE side, because you’d need to pay be an obscene amount to support this.

slowmovintarget 7y ago
It's like the bulked-up Cartman of key-value stores. Edit: Nope, it's a relational DB
diehunde 7y ago
Isn't that "side" just a bunch of clicks on a wizard on some cloud provider?

solatic 7y ago

There are companies that fit the following operational niche: * Need to run Kafka (for domain-specific reasons) * Need a relational database, but not necessarily a large or performant one A project that lets you "buy a Kafka cluster, get the relational database free" is strictly simpler and cheaper to operate (in theory...) compared to operating both a Kafka cluster and a traditional relational database.

adamnemecek 7y ago

It uses kafka as an operational primitive. Kafka is somewhat complicated but it also solves real problems. I can see a fast, distributed db built on top of kafka.

qaq 7y ago
There fast distributed DBs and they are generally built on top of distributed ordered key-value stores. Many of them do not require to run external components like zookeeper. What is the advantage of running one on top of Kafka?
- Godel_unicode 7y ago
  The main benefit I have heard is that some people already have Kafka.
  - qaq 7y ago
    in what sense is it a benefit?

random023987 7y ago

If you want an asymmetric persistence layer where all writes are documents placed in a Kafka topic, and all reads are via relational queries. It's a distributed, eventually consistent, highly available ledger that can be queried with SQL.

Scarbutt 7y ago
Is not eventually consistent, it uses https://omid.incubator.apache.org/ https://omid.incubator.apache.org/.

manigandham 7y ago

It's exciting in the sense that you can apply all these open-source projects to create a very slick lightweight relational database that can replicate changes globally across unlimited instances by using a Kafka topic as the WAL log. Don't actually use this in production. Use a real database that does what you want, and use the native replication features. If you need to replicate to heterogenous or alternative sources or just want a stream of all changes then you can use Kafka with any number of existing database connectors that automatically read and export changes.