3 ms·
Kafka can buffer indefinitely (so long as you give it the disk space) and you can throttle to your heart's content within the consumer's loop or just consumer.p
by abraxas 5y ago
Kafka can buffer indefinitely (so long as you give it the disk space) and you can throttle to your heart's content within the consumer's loop or just consumer.pause().
I don't think you know the basics of Kafka's API or how it works internally.
- RhodesianHunter 5y agoThis person doesn't appear to know the basics of a single thing they're maligning, but certainly isn't letting that slow them down.
- mgaunard 5y agoThe fact that you think this is a proper way to deal with a queue is quite concerning. You completely missed the point. It's the producer that needs to be paused, and the reason for that is that memory is not infinite. You cannot just keep buffering until the consumer catches up, because it may never catch up.
- jeffbee 5y agoWhile it is true that Kafka does not guarantee long-term steady state behavior, it is modeled as an impedance adapter of infinite capacity. It presents no impedance to the producer, has storage that is much larger than the arrival rate, and drains to the consumer as available. Any feedback between the consumer and producer has to happen out of band, which is fine.
- mgaunard 5y agoAnd that is one of thr major defects I pointed out. It's not "fine".