4 ms·
Agreed. To expand on your point, I believe this is pretty close to exactly what Amazon did to build Aurora, which is probably the most exciting development I've
by yowlingcat 7y ago
Agreed. To expand on your point, I believe this is pretty close to exactly what Amazon did to build Aurora, which is probably the most exciting development I've seen as an engineering leader in the past decade. Suddenly, the one crucial, stateful component of my system that could ever present sharding issues at scale became something I could pay a premium to not have to think about (to a point -- believe it's still 64TB for Aurora).
What's extra cool about this for me is how it illustrates to me how far open source components that build on the Apache Kafka ecosystem have come, to the degree that each fairly complex component of relational databases that expose a full RDBMS system from the basic building block of a log have been fully developed (to some production capacity) in the Kafka ecosystem. That's amazing! What does this mean?
It means that you could theoretically (please don't hurt me) implement a relational DB backed by a twitter DB log, or a blockchain, or any other event log that you can't necessarily use traditionally as a WAL. It also introduces a lot of really interesting possibilities for data integration. Definitely has my mind reeling a bit.