3 ms·
I think both length of code is a side effect that results from the primary argument, and "cant do event time" is one of the symptoms. Neither is a primary argum
by vgt 11y ago
I think both length of code is a side effect that results from the primary argument, and "cant do event time" is one of the symptoms. Neither is a primary argument in the blog post.
The primary argument is demonstrated through color coding different logical bits, which end up being clearly portable and elegantly distinct in dataflow.
This is demonstrated in two ways:
1. The "juicy value add" code that does the aggregation is labeled yellow, and doesn't change across all the samples with Dataflow. With Spark, it needs to be rewritten for every use case. Similarly, for all colors.
2. In Dataflow all the colors are separate. This makes expressing your logic easier. In Spark, the colors mix in dramatic ways with every demonstrated use case.
As Tyler said, all this is described in the blog post itself, but I don't blame you for missing it, since it's a really long post :)