3 ms·
interesting, in my case I need the events to be inserted in a SQL db, even the real time ones. For instance, I receive Contacts data from Hubspot in realtime, I
by kfk 4y ago
interesting, in my case I need the events to be inserted in a SQL db, even the real time ones. For instance, I receive Contacts data from Hubspot in realtime, I send those contacts over to Salesforce and I store them in Postgres. Why Postgres? Because we want to keep a history of the contacts, plus we will need to have 1 source of truth for customers data to fulfill various data privacy requirements. How about Kafka for such a use case? Let's say I receive maybe 10,000 contacts per day.
- geoduck14 4y agoNot your OP, but someone who has been thinking about this. Use Kafka as the super fast layer to connect producers and consumers. Have it write to S3/Blob. Then insert into your dB layer for "cold" access. Inserting into a dB takes longer than pub/sub, and you don't want to mess up the publishing of data.