5 ms·
I'll have to benchmark Spark 2.0 against Flink, it seems like it could be faster than Flink now. It does depend on if the dataset they used was living in memory
by faizshah 10y ago
I'll have to benchmark Spark 2.0 against Flink, it seems like it could be faster than Flink now. It does depend on if the dataset they used was living in memory before they ran the benchmark, but it's still some pretty impressive numbers and some of the optimizations they made with Tungsten sound similar to what Flink was doing.
- capkutay 10y agoReally depends on the use case. If you're trying to do streaming, spark will introduce some latency because it's based on micro-batching. Flink and Storm are better for low-latency scenarios.
- faizshah 10y agoI agree that Flink is better for low latency, but I was also under the impression Flink's DataSet API was faster for batch processing than Spark. Now that they're purporting Spark 2.0 may be up to 10x faster than 1.6 I'll have to take another look.