jkff

4 ms·

1. 1.▲
   by jkff 6y ago

   Hey - I'm Eugene, cofounder of the parent organization of this project (Work On Climate). We left Google to work on climate. One of the first things we've discovered after talking to lots of people in climate tech is that almost a
2. 2.▲
   by jkff 11y ago

   Hi, I'm also an engineer on the Dataflow team. This is a very exciting project! However it is still constrained by Spark's limitations outlined in this post in terms of what Dataflow pipelines it can run. For example, currently it
3. 3.▲
   by jkff 11y ago

   Hi! I'm an engineer on the Dataflow team. I'd love to understand the issue you mentioned about size of intermediate files, and help make Dataflow work well for you. Could you send a note to dataflow-feedback@google.com and elabora
4. 4.▲
   by jkff 11y ago

   Hi! Dataflow team member here. On maturity: Dataflow is built by the people who over the past 12 years created MapReduce, FlumeJava and FlumeC++, Pregel and Millwheel, based on the sum of experience obtained from all of these. It shares mos