3 ms·
KCache is an in-memory Java cache where all writes are sent to a Kafka topic and then applied locally by any instances listening to that topic, providing an eve
by manigandham 7y ago
KCache is an in-memory Java cache where all writes are sent to a Kafka topic and then applied locally by any instances listening to that topic, providing an eventually-consistent cache. Great for scaling a small dataset that can fit on a single-node with easy replication and a full log of all updates in the Kafka topic.
KarelDB adds relational querying, schema/table management, transactions, and a SQL interface on top of KCache by using various Apache open-source projects (Calcite, Omid, Avatica, Avro). Here's a blog post from the author that has more details: https://yokota.blog/2019/09/23/building-a-relational-database-using-kafka/ https://yokota.blog/2019/09/23/building-a-relational-databas...
A typical database (like Postgres) or cache (like Redis) has a WAL where all writes are applied and this log is also used for replicas. KCache and KarelDB are interesting experiments in doing this from scratch using open-source components, although using Kafka as a durable log of system-agnostic changes is already very common. In fact, it's probably the most used feature of Kafka Connect with various database connectors and changefeed streams.
- ako 7y agoSounds like Kareldb would make a good alternative to rocksdb, when combining tables and streams in Kafka streaming?