3 ms·
Is Kafka good for processing hundreads of GB size messages per second? Can also a Kafka Stream (output) be input to another Kafka stream dynamically?
by zekrioca 9y ago
Is Kafka good for processing hundreads of GB size messages per second? Can also a Kafka Stream (output) be input to another Kafka stream dynamically?
- spiffytech 9y agoKafka expects messages to be under 1MiB. If you need larger messages, they recommend you store the payload somewhere else and use Kafka to pass around pointers to the payload. So e.g., pass around an S3 URL.