3 ms·
I agree, the Confluent team paint a very clear big picture. You may be interested to take a look at Crux [0], a general purpose document database with bitempora
by refset 7y ago
I agree, the Confluent team paint a very clear big picture. You may be interested to take a look at Crux [0], a general purpose document database with bitemporal graph queries, implemented as an ES/CQRS system directly on top of Kafka (or a local KV store).
We created Crux because we found ourselves routinely needing bitemporal functionality when building immutable systems that are capable of ingesting and interpreting out-of-order/late-arriving events [1].
So far we have decided against using Kafka Streams to keep our log-storage options very pluggable, but we pretty much implement the same mechanics.
Disclosure: product manager for Crux :)
[0] https://juxt.pro/crux/index.html https://juxt.pro/crux/index.html
[1] https://juxt.pro/blog/posts/introducing-crux.html https://juxt.pro/blog/posts/introducing-crux.html
- dmitryminkovsky 7y agoThank you! I look forward to checking this out. I'd say the picture painted by Confluent is much rosier than reality, but it definitely is the clearest, most actionable picture to date.
- refset 7y agoTo be specific, the picture I am thinking about in particular is described in a slide titled "Local consistency points in the absence of Global Consistency" (from this talk: https://youtu.be/VYOMmwkdSig?t=1817 https://youtu.be/VYOMmwkdSig?t=1817), where it shows how each local consistency point is ideally implemented using ES/CQRS. I also agree that the tools Confluent are currently providing us with today do not make this trivial. Edit: the other big picture Confluent quote I like is "You are not building microservices, you are building an inside-out database" (Tim Berglund, Confluent, 2018) which is a perfect answer to this other quote: "The hardest part of microservices is your data" (Christian Posta, Red Hat, 2017)