3 ms·
Benchmarking Streaming Computation Engines at Yahoo
- estefan 11y agoThis is the first mention I've seen of flink on HN.
- luckydata 11y agothere were a few before but not many.
- kod 11y agoIf I'm reading this correctly, the Kafka topic only had 5 partitions, but they had 10 workers. With the Spark direct stream, kafka partitions are 1:1 with spark partitions, which means at most half of the workers would be doing work without a shuffle. Seems like a pretty basic oversight that should be addressed.