2 ms·
> processing billions of Kafka events per day Except that the burden is on all clients to coordinate to avoid processing an event more than once since Kakfa is
by throwwgisgreat 11mo ago
> processing billions of Kafka events per day
Except that the burden is on all clients to coordinate to avoid processing an event more than once since Kakfa is a brainless invention just dumping data forever into a serial log.
- williamdclt 11mo agoI'm not sure what you're talking about. Do you mean different consumers within the same consumer group? There's no technology out there that will guarantee exactly-once delivery, it's simply impossible in a world where networks aren't magically 100% reliable. SQS, RedPanda, RabbitMQ, NATS... you call it, your client will always need idempotency.
- mrkeen 11mo agoThat is called a 'consumer group' which has been a part of Kafka for 15 years. The author is suggesting to avoid this solution and roll your own instead.