3 ms·

Why would I want a relational database living in Kafka? MySQL or Postgres are great at what they do.

by xmpir 7y ago

mothsonasloth 7y ago
Why would I want DOOM running on a 148x64 LCD Canon printer screen? Because I can - https://www.youtube.com/watch?v=NPWi5yJK3zo https://www.youtube.com/watch?v=NPWi5yJK3zo
xjoins 7y ago
I came here to ask this question. It's a really solid technical article. But apart from the sake of doing it, why would anyone ever want to? I'm not suggesting there's no scenario in which one would want to build a SQL database using Kafka, I'm just asking if anybody knows of one.
- wefarrell 7y ago
  One use case for Kafka is log aggregation. Using SQL to analyze logs would be one use case.
- dominotw 7y ago
  > SQL database what is your definition of sql database?
lmm 7y ago
A lot of the low-level behaviour is hard to control, at least in terms of having a well-known public interface. E.g. transaction isolation level is database-global, limited control over when updates to indices happen, limited direct control over MVCC. The internals of MySQL or Postgres look a lot like Kafka, but the event-transforming parts are hidden inside a black box.
mamcx 7y ago
I love PG and rdbms in general, but certainly exist SO MANY things that could be improved with RDBMS. 1- RDBMS implement an imperfect view of the relational model. Is like to say Java is the only OO inplemented in the world. 2- SQL is the standard... and is not practical to say to eliminate it... BUT RDBMS are constrained for the subpar language SQL is. Modern additions are nice... but that is to say that postcript is a nice way to do apps. Also, a much better language could expand the role of RDBMS. This is not weird. I live this way when working in FoxPro: - UI on fox lang. - DB on fox lang. - Reports on fox lang. - Scripting on fox lang. - Web on fox lang - Triggers on fox lang - OO on fox lang. And all that still with clean separation of logic and components. 3- RDBMS were made by a certain mindset and use case of the 80s. Still so good that can be valid today, but it could get extended. JSON support is just a tiny example of that. 4- Why you can't do "SELECT .. FROM index"? Why you need to create a table to get an index or FTS? Is weird limitation. The relational model is fine too for KV stores. ---- "Relational database" is a database made on top of the relational MODEL. You could get MANY IMPLEMENTATIONS of that with different use cases. Similar how the functional MODEL and the OO MODEL is not frozen with a single language... that is not that good for app development. With a better lang, you could eliminate massive ORMs in a nice way! P.D: Remember, this IS NOT teory. Is proven. This is how was with the dbase family of langs!

Why would I want a relational database living in Kafka? MySQL or Postgres are great at what they do.

by xmpir 7y ago

Why would I want a relational database living in Kafka? MySQL or Postgres are great at what they do.

Why would I want DOOM running on a 148x64 LCD Canon printer screen? Because I can - https://www.youtube.com/watch?v=NPWi5yJK3zo https://www.youtube.com/watch?v=NPWi5yJK3zo

xjoins 7y ago

I came here to ask this question. It's a really solid technical article. But apart from the sake of doing it, why would anyone ever want to? I'm not suggesting there's no scenario in which one would want to build a SQL database using Kafka, I'm just asking if anybody knows of one.

wefarrell 7y ago
One use case for Kafka is log aggregation. Using SQL to analyze logs would be one use case.
dominotw 7y ago
> SQL database what is your definition of sql database?

lmm 7y ago

A lot of the low-level behaviour is hard to control, at least in terms of having a well-known public interface. E.g. transaction isolation level is database-global, limited control over when updates to indices happen, limited direct control over MVCC. The internals of MySQL or Postgres look a lot like Kafka, but the event-transforming parts are hidden inside a black box.

mamcx 7y ago

I love PG and rdbms in general, but certainly exist SO MANY things that could be improved with RDBMS. 1- RDBMS implement an imperfect view of the relational model. Is like to say Java is the only OO inplemented in the world. 2- SQL is the standard... and is not practical to say to eliminate it... BUT RDBMS are constrained for the subpar language SQL is. Modern additions are nice... but that is to say that postcript is a nice way to do apps. Also, a much better language could expand the role of RDBMS. This is not weird. I live this way when working in FoxPro: - UI on fox lang. - DB on fox lang. - Reports on fox lang. - Scripting on fox lang. - Web on fox lang - Triggers on fox lang - OO on fox lang. And all that still with clean separation of logic and components. 3- RDBMS were made by a certain mindset and use case of the 80s. Still so good that can be valid today, but it could get extended. JSON support is just a tiny example of that. 4- Why you can't do "SELECT .. FROM index"? Why you need to create a table to get an index or FTS? Is weird limitation. The relational model is fine too for KV stores. ---- "Relational database" is a database made on top of the relational MODEL. You could get MANY IMPLEMENTATIONS of that with different use cases. Similar how the functional MODEL and the OO MODEL is not frozen with a single language... that is not that good for app development. With a better lang, you could eliminate massive ORMs in a nice way! P.D: Remember, this IS NOT teory. Is proven. This is how was with the dbase family of langs!