You've made very good points about smart clients, but at some point one has to ponder if it's worth it or one should just not use kafka in the first place. I'v

You've made very good points about smart clients, but at some point one has to ponder if it's worth it or one should just not use kafka in the first place. I've seen databases used as messaging queues and if it was up to me, I'd never do that. It's usually "but we already have kafka + db, why burden ourselves with another messaging technology?", which is fair. > You can stream messages into a buffer and process them in parallel, and commit the low watermark offset whenever it changes, as described above. I've implemented this in .NET with Channels and saturate the CPUs with no problem. That is very nice -- certainly seems better than just batch processing of kafka messages, but you're still just kicking the can down the road. How large do allow the buffer to become and what do you do when it's getting too large? You probably use a DLQ. Don't get me wrong, I think the buffer idea probably works most of the time.