3 ms·
If you have some mapping from data to units of work, can't you just use callbacks to make sure the data isn't acked until the units of work complete successfull
by thebean11 6y ago
If you have some mapping from data to units of work, can't you just use callbacks to make sure the data isn't acked until the units of work complete successfully? I use kafka at work and so far haven't really run into this as an issue.
In the bit you quoted, I'd assume the local work queue has callback objects.
- kerblang 6y agoSorry for late answer: A Kafka ack is not a signal that an individual message is complete, but that the message index pointer can move forwards. So if you read 10 tasks and run them in parallel, you can't just ack a random selection of 5 as they complete. You could work out a system to ack the entire bunch when the last one finishes though. All of this assumes you're ok with more-than-once delivery, of course.