3 ms·
I can't figure out what their modified version of the Yahoo Benchmark is. Does it still involve Kafka? Does it still involve Redis? Did they only generate tuple
by spooneybarger 8y ago
I can't figure out what their modified version of the Yahoo Benchmark is. Does it still involve Kafka? Does it still involve Redis? Did they only generate tuples in-memory for SABER and the other systems pull from Kafka?
The Yahoo Streaming Benchmark isn't really much of a streaming benchmark, its a "reading things off of kakfa, deserializing JSON, sticking things in Redis benchmark". Really not a good benchmark for understanding the system at test.
- grt17 8y agoIn previous blog posts, first Flink and then Spark removed the interactions with Redis and Kafka and started generating data in-memory, in order to measure their system's performance. The same applies here. Sure, the benchmark isn't great, as mentioned in the post, but this is what people are still using to evaluate their systems.
- frankmcsherry 8y agoNo, they did not only generate tuples in-memory for SABER. From the linked Flink blog post: > Databricks made a few modifications to the original benchmark, all of which are explained in their own post: > Removing Redis from step 5 > Generating data in memory for Flink and Spark; generating data via Spark and writing it to Kafka for Kafka Streams > Writing data out to Kafka instead of to Redis I think the authors know that the benchmark is weak. The blog post isn't (to my reading) claiming that it is proof that SABER is great so much as that the distributed stream processors need better evidence that they should exist (vs the Databricks Structured Streaming SIGMOD 2018 paper, in which the YSB benchmark is (I think) the only quantitative evaluation they do).
- spooneybarger 8y agoThanks. I missed the link to what the modifications were.