4 ms·
The "traditional" Kafka infrastructure (i.e. LinkedIn) uses Kafka as the source of truth. Everything gets written directly into Kafka and makes its way from th
by luhn 10y ago
The "traditional" Kafka infrastructure (i.e. LinkedIn) uses Kafka as the source of truth. Everything gets written directly into Kafka and makes its way from there. This is a powerful concept, but involves giving up a lot of advantages of a RDBMS.
This is the first time I've heard of the approach WePay uses and I really like it. Now MySQL is the source of truth and Debezium pipes it into Kafka, allowing you to leverage Kafka while still using MySQL as your primary data store. (Or Postgres, which Debezium includes support for.)
- swah 10y agotl;dr but so the simple operations like "update my birthdate" happen with sync MySQL operations?
- luhn 10y agoYep. All their applications always write directly to MySQL.
- happyslobro 10y agoI guess if that became a bottleneck, they could set up a queue for write requests. Maybe even something more sophisticated that automatically batches similar writes that pile up in the queue. They could use something like Kafka for this...