4 ms·

Accoring to the proposal (https://wiki.apache.org/incubator/BeamProposal https://wiki.apache.org/incubator/BeamProposal), OSS impl. of streaming is on the way,

• teraflop 11y ago
  Huh, I can't find anything to that effect in the proposal. It does mention the existence of runners for Spark and Flink, but doesn't say they'll be getting streaming support. I had assumed it was unlikely for the same reason that this article talks about (lack of support for first-class processing by event time). But assuming it's true. it's very welcome news! I'll be keeping a close eye on future releases.
  • takidau 11y ago
    Flink's event-time support is coming along nicely. Their first round of true event-time support came in November (https://flink.apache.org/news/2015/11/16/release-0.10.0.html https://flink.apache.org/news/2015/11/16/release-0.10.0.html), and much more is on the way. Flink will be an excellent platform for Beam, both batch and streaming. As I understand it, Spark has event-time support coming soon as well. I think basic stuff is landing in 1.7. Not sure precisely what they have planned, but I can only imagine that Spark will also become an excellent platform for executing streaming Beam pipelines in due time. In the meantime, the streaming runner for Spark can either target those features which Spark does support well (i.e., processing-time windowing, in this case), or try to emulate those it doesn't (such as how it was done in the article).