4 ms·

I'm not into the sample code using static imports. Static imports makes it hard to copy examples, especially if you don't include the imports statements. It's

I'm not into the sample code using static imports. Static imports makes it hard to copy examples, especially if you don't include the imports statements. It's a bit unfair to compare stream API to similar APIs because it's so universally applicable. For example, Python (and I suspect groovy) impose some implementations and limitations that aren't imposed with streams.

PaulHoule 4y ago
There is a very big design space for "Stream" APIs. Microsoft's LINQ for instance can compile a stream operation into a SQL statement and JooQ does the same. That system offers query optimization and efficient joins that depend on the query system having complete visibility into the queries. indexes built ahead of time, etc. Another extreme is a system like https://www.reactive-streams.org/ https://www.reactive-streams.org/ that are especially good for apply a filter and map and other operations to a stream of real time events, e.g. instead of having a pull operation such as a for-loop over an Iterable, items go into the system from a stream. I've worked on systems that use the later kind of streaming to run batch jobs and you can get great performance (780% speedup with 8 cpus) on crazy heterogenous workloads. You do have to be careful though to shut the system down or flush it out or otherwise you get wrong answers. Frequently those frameworks don't shut themselves down properly unless you implement clean shutdown yourself. The point is that operators like "filter" and "map" and the rest are so powerful because they are portable between the minimal pidove up to a Hadoop cluster. Every implementation though has details, makes different decisions, has different limitations. For instance the way group by is done in Python and Java are completely different. A "Tuple" doesn't make as much sense in Java as it does in Python because it's no fun to "get(7)" and get just an Object.

PaulHoule 4y ago

There is a very big design space for "Stream" APIs. Microsoft's LINQ for instance can compile a stream operation into a SQL statement and JooQ does the same. That system offers query optimization and efficient joins that depend on the query system having complete visibility into the queries. indexes built ahead of time, etc. Another extreme is a system like https://www.reactive-streams.org/ https://www.reactive-streams.org/ that are especially good for apply a filter and map and other operations to a stream of real time events, e.g. instead of having a pull operation such as a for-loop over an Iterable, items go into the system from a stream. I've worked on systems that use the later kind of streaming to run batch jobs and you can get great performance (780% speedup with 8 cpus) on crazy heterogenous workloads. You do have to be careful though to shut the system down or flush it out or otherwise you get wrong answers. Frequently those frameworks don't shut themselves down properly unless you implement clean shutdown yourself. The point is that operators like "filter" and "map" and the rest are so powerful because they are portable between the minimal pidove up to a Hadoop cluster. Every implementation though has details, makes different decisions, has different limitations. For instance the way group by is done in Python and Java are completely different. A "Tuple" doesn't make as much sense in Java as it does in Python because it's no fun to "get(7)" and get just an Object.