4 ms·

Ok. I admit using Kafka as DB is not straight forward but just stating it doesn't provide ACID functionality is not enough. The example they give is very simpl

by je42 6y ago

Ok. I admit using Kafka as DB is not straight forward but just stating it doesn't provide ACID functionality is not enough. The example they give is very simplistic. With the correct design of kafka topics and events the problem of the example can be fixed. And according to oracle https://www.oracle.com/database/what-is-database/ https://www.oracle.com/database/what-is-database/ : > A database is an organized collection of structured information, or data, typically stored electronically in a computer system. So Kafka clearly fits that definition.

halbritt 6y ago
The author presumes that every use case requires a transactional database. ACID is nice, especially if it's needed, but generally not needed, especially in many streaming data applications for which Kafka is most suitable.
satisfaction 6y ago
I'm guilty of using it as a DB, my home weather station writes to kafka topics, sometimes the postgres instance is down for months, no problems letting the kafka topic store the data until i get around to rebooting pg and restarting the connector.
dathinab 6y ago
Didn't some newspaper use Kafka to store the newspapers they released in order or something similar? (I think it was the New York Times, maybe??). Honestly as long as you don't use it as a general purpose database, it might very well be the best choice for your use-case.
- je42 6y ago
  Indeed. Known the tools and apply the one that has least cons and most pros ;). Also, a good starter to for knowing how to use a Streaming Store like Kafka as DB is the video Database inside-out. https://www.youtube.com/watch?v=fU9hR3kiOK0 https://www.youtube.com/watch?v=fU9hR3kiOK0
- barnabask 6y ago
  2017 article: https://open.nytimes.com/publishing-with-apache-kafka-at-the-new-york-times-7f0e3b7d2077 https://open.nytimes.com/publishing-with-apache-kafka-at-the...
- lmm 6y ago
  I'd actually say Kafka makes a better general-purpose database. SQL databases are only appropriate for a pretty narrow range of problems.

by je42 6y ago

halbritt 6y ago

The author presumes that every use case requires a transactional database. ACID is nice, especially if it's needed, but generally not needed, especially in many streaming data applications for which Kafka is most suitable.

satisfaction 6y ago

I'm guilty of using it as a DB, my home weather station writes to kafka topics, sometimes the postgres instance is down for months, no problems letting the kafka topic store the data until i get around to rebooting pg and restarting the connector.

dathinab 6y ago

Didn't some newspaper use Kafka to store the newspapers they released in order or something similar? (I think it was the New York Times, maybe??). Honestly as long as you don't use it as a general purpose database, it might very well be the best choice for your use-case.

je42 6y ago
Indeed. Known the tools and apply the one that has least cons and most pros ;). Also, a good starter to for knowing how to use a Streaming Store like Kafka as DB is the video Database inside-out. https://www.youtube.com/watch?v=fU9hR3kiOK0 https://www.youtube.com/watch?v=fU9hR3kiOK0
barnabask 6y ago
2017 article: https://open.nytimes.com/publishing-with-apache-kafka-at-the-new-york-times-7f0e3b7d2077 https://open.nytimes.com/publishing-with-apache-kafka-at-the...
lmm 6y ago
I'd actually say Kafka makes a better general-purpose database. SQL databases are only appropriate for a pretty narrow range of problems.