14 ms·
Java JEP 461: Stream Gatherers
- psYchotic 3y agoThe author of the JEP, Viktor Klang, held a nice talk about this at Devoxx. If you're interested, you can watch the talk here: https://www.youtube.com/watch?v=8fMFa6OqlY8 https://www.youtube.com/watch?v=8fMFa6OqlY8
- exabrial 3y agoI really wish I convince our ownership these conferences are worth going to. I’d love to be able to ask questions and learn first hand from the people developing these.
- mjburgess 3y agolong numberOfWords = Stream.of("the", "", "fox", "jumps", "over", "the", "", "dog") // (1) .filter(Predicate.not(String::isEmpty)) // (2) .collect(Collectors.counting()); // (3) This programming style is both expressive and efficient. Err... I assume the author here is aware that it is neither, but perhaps struggles for something else to say in motivating this API. It's certainly an amusing remark. I'd prefer either an imperative, for(w <- words) if(!w.empty) count++ or, an ergonomic dsl, words.filter( w -> !w.empty ).count The strange namespacing of functional primitives, glued into an api via 'gather', 'collect' etc. is neither expressive not efficient. Though I concede it may be a necessary route for a JEP
- bmacho 3y agoSeems expressive and efficient to me.
- marginalia_nu 3y agoExpressive, sure. Efficient? Not so much. It's easy to find cases where performing an operation using the streams API comes at a 90% performance hit.
- PhilipRoman 3y agoTSMT Fingers crossed that the upcoming Valhalla project can improve the situation w.r.t. number of allocations. I've noticed a general trend where languages without support for proper compile time metaprogramming (and the corresponding optimizations) often create uglier code (despite being higher level) to avoid the performance hit. For example the mutable Vec2D type in various physics related libraries. Last time I tested it was still faster to implement various string operations using Unsafe, than with standard APIs. I want to be able to trust my compiler/JIT.
- qsort 3y agoThe do notation would be ideal here, but it's probably a bit too much to ask...
- Traubenfuchs 3y agoIs it that fun to shit on Java with cherrypicked worst-case and unfair examples? You could use any libraries StringUtils.hasText (any serious app has one) to avoid the Predicate.not and do static imports (StringUtils, Collectors) to avoid explicitedly naming the classes containg those static methods... Also your example misses the string collection creation part. Ultimately we should get, put on one line like your code: words.stream().filter(hasText).collect(counting());
- mrkeen 3y ago> Is it that fun to shit on Java with cherrypicked worst-case and unfair examples? The person doing the shitting did not pick the examples.
- d35007 3y agoThe kids hate Java, and that's ok. No sense arguing with them. I am happy a tool I use frequently is getting better. I am also happy there are other tools out there.
- mike_hearn 3y agoThat example could be easily written like this: var num = words.stream().filter(w -> !w.isEmpty()).count(); This is shorter and closer to what you're expecting. The reason it's written the long way is because of the labels. The prior sentence is: A stream pipeline consists of three parts: a source of elements, any number of intermediate operations, and a terminal operation. For example: ... so the goal here is to make the different components of the pipeline clear conceptually for the explanation of what's to come, not to write the shortest code possible. It therefore uses code that makes the underlying abstractions explicit rather than the sugar that avoids typing. The reference to "expressive and efficient" meanwhile is likely about the fact that Java can parallelize streams in many cases (efficient). As for "expressive", well, that's somewhat debatable but for sure there are some cases where the functional pipeline style is easier to read and more expressive than the imperative style. In my own code I use both because indeed especially for short operations over lists and the like, sometimes a for loop is just clearer to my eye. But it's a matter of taste. BTW the same code written in Kotlin: val num = words.filter { !it.isEmpty() }.count() Very similar. You may slightly prefer the syntax or dislike the "magical" auto-naming of the lambda parameter, I've seen it argued both ways. ---- https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/stream/Stream.html#count https://docs.oracle.com/en/java/javase/21/docs/api/java.base...()
- kaba0 3y agoI think Scala is hands down the best collections/stream library, I especially like their _ shorthands: val num = words.count(!_.isEmpty())
- Marazan 3y agoI think that is a horrifying crime against man and nature. If I wanted to write PERL I would do so.
- kaba0 3y agoWhat is hard to understand about it? _ is universally used as a placeholder, it also usable in pattern matching. Scala can be used to make some perl-y <!! operators, but it is not like that at all, and makes for very readable, understandable code.
- SkiFire13 3y agoNote that Java also allows you to write that as words.filter(w -> !w.isEmpty()).count() And they just end up calling the same methods. The author probably used "expressive" and "efficient" to refer to the underlying APIs for the fact that you can compose operations without fully evaluating the whole stream until the terminal one.
- deleted 3y ago[deleted]
- dullcrisp 3y agoIf you look closely you’ll see that the given code is equivalent to your second example, though admittedly the Java syntax is a bit clumsy.
- another-dave 3y ago> evaluation begins only when a terminal operation is invoked. In this example, line (1) creates a stream, but does not evaluate it, line (2) sets up an intermediate filter operation but still does not evaluate the stream, and finally the terminal collect operation on line (3) evaluates the entire stream pipeline. It's quite clear, reading beyond just the code, that the original author is talking about _computational_ efficiency. It feels like you missed the point — that streams are only evaluated lazily as needed — in order to poke fun at the (toy) example. The Java code can (and in a real-world application would), be made more consise (e.g. via static imports) but the author is deliberately being more verbose to be more explicit in what's going on.
- mrkeen 3y ago> It feels like you missed the point — that streams are only evaluated lazily as needed — in order to poke fun at the (toy) example. GP may be closer to the point than you realise, even if unintentionally so. I mentioned Streams failing to be lazy in another comment, and predictably got down-votes rather than corrections. So anyway... https://bugs.java.com/bugdatabase/view_bug?bug_id=8079264 https://bugs.java.com/bugdatabase/view_bug?bug_id=8079264 Submitted: 2015-05-01 https://bugs.java.com/bugdatabase/view_bug?bug_id=8149614 https://bugs.java.com/bugdatabase/view_bug?bug_id=8149614 Submitted: 2016-02-10 https://bugs.java.com/bugdatabase/view_bug?bug_id=8155217 https://bugs.java.com/bugdatabase/view_bug?bug_id=8155217 Submitted: 2016-04-25 https://bugs.java.com/bugdatabase/view_bug?bug_id=8189234 https://bugs.java.com/bugdatabase/view_bug?bug_id=8189234 Submitted: 2017-10-11 https://bugs.java.com/bugdatabase/view_bug?bug_id=8196106 https://bugs.java.com/bugdatabase/view_bug?bug_id=8196106 Submitted: 2018-01-24 https://bugs.java.com/bugdatabase/view_bug?bug_id=8229983 https://bugs.java.com/bugdatabase/view_bug?bug_id=8229983 Submitted: 2019-08-21 https://bugs.java.com/bugdatabase/view_bug?bug_id=8267758 https://bugs.java.com/bugdatabase/view_bug?bug_id=8267758 Submitted: 2021-05-25 I wonder if 2024 will be the year of the lazy Stream.
- another-dave 3y agoI think the point that you are making — that even if Streams are meant to be efficient on paper, but reality is different because of bugs in implementation — is an interesting one that moves on the discussion! Surprised to hear you were downvoted for it. Personally, I don't think that's what the other comment was getting at though, as they were commenting on efficiency regards to syntax.
- oaiey 3y agoefficient compared to what? Previous Java: Yes. C++: Yes. C#: maybe not. Functional Programming Language X with 2 dozen language feature supporting it: Definitely not. But this is the Java Enhancement Process, and within this language AND the fact that this is specification/documentation example, this is efficient.
- PaulHoule 3y agoStreams is too complex for what it does and it doesn’t even parallelize well. Here is something that does roughly the same thing but I think is way better See https://github.com/paulhoule/pidove https://github.com/paulhoule/pidove https://central.sonatype.com/artifact/com.ontology2/pidove https://central.sonatype.com/artifact/com.ontology2/pidove
- nogridbag 3y agoThis is far less readable than Java streams to me. sum(filter(x->x%2==0,List.of(5,3,4,19,75,6))); Streams are formatted linearly similar to Clojure threading macros.
- PaulHoule 3y agoThe advantage of the static import DSL is that it is more composable: anyone else can define operators that interoperate with my operators without the complexity of the scheme linked above not to mention more generality. This goes not just for the low level operators like "filter" that you might want to supplement but the many more operators you can write that are implemented out of mine. I prototyped an API that works in the same direction as streams by having something like https://paulhoule.github.io/pidove/apidocs/com/ontology2/pidove/iterables/TidyIterable.html https://paulhoule.github.io/pidove/apidocs/com/ontology2/pid... in that it wraps all Iterables returned by my methods and has not just the teardown facility but also all the operators attached as instance methods which lets you write the chaining style you ask for that I know is in demand. If I was going to go any further on pidove it would have involved more use of code generation and this system https://github.com/paulhoule/ferocity https://github.com/paulhoule/ferocity which was supposed to be a code generator for writing code generators, and it could code generate stubs that would let you write expression trees in Java as S-expressions and build them up into methods and either compile the code to real Java source code or execute the methods by evaluating the expression tree in place. Like common LISP you can write syntactic macros in that that metalanguage because an Expression<Expression<X>> can be evaluated at compile time, one of quite a few concepts like "quoting" that I encountered in that spike. The idea was ferocity would get to the point where it synthesizes a more complete and perfect pidove.
- StevePerkins 3y agoIn fairness, you are omitting the list instantiation, and collecting only a count rather than an assignment of results. Apples to apples, your alternative algorithm would be implemented like this: words.stream().filter(w -> !w.isEmpty()).count() Which is hardly any longer, and arguably more readable intent with the English word "filter".
- benibela 3y agoMy favorite is XPath: let $numberOfWords := count($words[.]) or let $numberOfWords := $words[.] => count()
- qsort 3y agoIf you're like me and your first thought was "why can't we just use flatMap and mapMulti, the summary table in the linked article convincingly explains why: https://cr.openjdk.org/~vklang/Gatherers.html https://cr.openjdk.org/~vklang/Gatherers.html
- mjburgess 3y agoHow to implement flatMap(mapper) public final static <T,R> Gatherer<T, ?, R> flatMap(Function<? super T, ? extends Stream<R>> mapper){ return Gatherer.of( () -> (Void)null, (nothing, element, downstream) -> { try(Stream<? extends R> s = mapper.apply(element)) { return s == null || s.sequential().allMatch(downstream::flush); } }, (l,r) -> l, (nothing, downstream) -> {} ); } Better title: how scala won the war but lost every battle
- keepamovin 3y agoStream Gatherers Nice name. I think of: Those Moisture Farmers of Mos Eisley have certainly updated their methods. They are even approaching the Dew Gatherers of Arrakis in efficiency.
- Almondsetat 3y agoJust last week I got really frustrated at the Streams API because the .collect did almost perfectly what I wanted but didn't return a stream
- deleted 3y ago[deleted]
- dionian 3y agocollecting an entire stream means reading the entire stream into a fixed data structure, so it makes sense the result is not a stream.
- marwis 3y agoI wish there was something similar to F# |> operator for using arbitrary functions in call chains. It would be useful in more cases and have far simpler signature/interface.
- kirmerzlikin 3y agoI really like the idea but I don't understand the naming. Why "gatherer"? Doesn't "gather" mean more or less the same as "collect"?
- mrkeen 3y agoAnd 'fold'. And 'reduce'.
- danbruc 3y agoI would say in computing gather usually appears together with scatter and they are used when one thing gets split up into several things (scatter) and then they get collected back together (gather) into one thing. Breaking up one big task into several smaller ones, sending them somewhere for processing, and then collecting the results back together to combine them into one final result would be the prototypical example of scatter gather operations. Also in SIMD processing you can come across scatter and gather operations, there they essentially break apart or assemble the data vectors in memory.
- ivolimmen 3y agoCoincidentally I just watch the talk on this from Nikolai: https://www.youtube.com/watch?v=epgJm2dZTSg https://www.youtube.com/watch?v=epgJm2dZTSg. This explains what it is. His talk: https://www.youtube.com/watch?v=pNQ5OXMXDbY https://www.youtube.com/watch?v=pNQ5OXMXDbY shows how to use/implement it.
- neonsunset 3y agoAh, Java adding yet another C# feature 15 years later.
- marginalia_nu 3y agoWell yes. This is Java's stated development strategy. Let other languages evaluate new features, and implement them once they're demonstrated to be good.
- mrkeen 3y agoThe unstated part is implementing them poorly. Optionals that throw NPEs, Streams that operate on more than 1 item when called with .limit(1). Futures that don't cancel. Lambdas that don't play nicely with exceptions. Non-extendable streams.
- DarkNova6 3y agoOptionals only throw NPE if you’re using them wrong. You are not wrong about cancelabel futures, though.
- marginalia_nu 3y agoEither way it feels like optionals are going to be deprecated real quick now we have sealed interfaces and record matching. They let you get rid of so much of the unpleasantness in dealing with this API. Instead you can public sealed interface Maybe<T> { <T2> Maybe<T2> map(Function<T, T2> mapper); <T2> Maybe<T2> flatMap(Function<T, Maybe<T2>> mapper); T elseGet(T fallback); record Just<T>(T value) implements Maybe<T> { public <T2> Maybe<T2> map(Function<T, T2> mapper) { return new Just<T2>(mapper.apply(value)); } public <T2> Maybe<T2> flatMap(Function<T, Maybe<T2>> mapper) { return mapper.apply(value); } public T elseGet(T fallback) { return value; } } record Empty<T>() implements Maybe<T> { public <T2> Maybe<T2> map(Function<T, T2> mapper) { return new Empty<>(); } public <T2> Maybe<T2> flatMap(Function<T, Maybe<T2>> mapper) { return new Empty<>(); } public T elseGet(T fallback) { return fallback; } } } public void demo() { Maybe<String> foo = new Maybe.Empty<String>(); System.out.println( switch (foo) { case Maybe.Just(String val) -> "Hello " + val; case Maybe.Empty() -> "Fine, leave me hanging"; } ); };
- manjalyc 3y agoA lot of comments here focus on the readability, conciseness, and expressiveness of Java streams compared to other languages. IMO they are missing the point and are just reiterating the same complaints everyone has about the Java language in x different ways. Java streams bring real benefits compared to other mature languages. My favorite is that sequential streams can be efficiently parallelized with a single operation, .parallel(), and sequentialized back, .sequential(), on any stream without having to configure a single knob manually (although you certainly can), an equivalent I am unaware of in any other matured language. These make operations such as .collect() and its mutable reductions leverage multiple threads for effectively 0 additional programming time. edit: A lot of people are focusing on my favorite feature of parallelizing or serializing a stream with a single command, which apparently you can also do in C#, that was just an example guys. Other cool things you can do with Java streams natively now is leverage virtual threads (take that C#), use them asynchronously with completable futures, define how elements are accessed/gathered in streams via spliterators, etc. Streams in Java are very composable (not just as in composition, but as in utility), and enable leveraging nearly every other part of the language natively. In other mature languages, streams are very rigid and feel non-composable. I'm not saying that everything is impossible in other languages, but that in Java streams feel like a first-class citizen.
- neonsunset 3y agoC# had PLINQ (aka arr.AsParallel()) since the time immemorial. It also works quite well with picking the right parallelization strategy and is very similar in its use to Rust Rayon's .par_iter().
- ynik 3y agoThat's a common feature nowadays, not unique to Java. C# has had `enumerable.AsParallel()` since .NET 4.0 (2010). Rust has rayon `input.par_iter()`.
- manjalyc 3y agoDoes C# allow you to convert a sequential stream into a parallel stream, or do streams have to be parallelized when initiated? Genuinely asking, I do not know C#
- Dudester230602 3y agoWhy waste such a powerful term as "stream" on good old trivial 'list comprehensions'? https://en.wikipedia.org/wiki/List_comprehension https://en.wikipedia.org/wiki/List_comprehension
- xienze 3y agoBecause it applies to things other than lists?
- Dudester230602 3y agoAre you making an assumption based on some specific "lists" or are Java 'streams' infinite? https://en.wikipedia.org/wiki/List_(abstract_data_type) https://en.wikipedia.org/wiki/List_(abstract_data_type) https://en.wikipedia.org/wiki/Stream_(abstract_data_type) https://en.wikipedia.org/wiki/Stream_(abstract_data_type)
- mrkeen 3y ago> or are Java 'streams' infinite? They are in toy examples. I don't follow your original point 100%, but I will say that 90% of the time when I use Streams, I'm really just wishing List had map and filter, etc.
- tpm 3y agoJava Streams apply to Collections, which is a broader abstraction than a list.
- Sankozi 3y ago"A list comprehension is a syntactic construct available in some programming languages for creating a list based on existing lists." (https://en.wikipedia.org/wiki/List_comprehension https://en.wikipedia.org/wiki/List_comprehension) Stream's terminal operations contains much more options than just "creating a list".
- shepherdjerred 3y agoStreams work on any Java Collection, such as Maps.
- spreiti 3y agoLooking forward to this JEP. This is, IMO, how Streams should have been implemented from the get go. All intermediate operations can now be expressed with Gatherers rather than having dedicated methods that can never cover all intermediate operations people come up with.
- stuff4ben 3y agoFor not having seriously touched Java in over 10 years since Java 8, I was able to still follow along this JEP. I suppose I could pick it up again if I had to. If my 401k runs out when I retire in 10-15 years, I will always know I can still code some shitty Java for some shitty company and make ends meet.
- jfengel 3y agoPrecisely. That has been Java's design goal from the beginning. It's not flashy or exciting or quick. It's a language for writing code that will keep on cranking along for decades, and maintained by multiple generations of developers. They resisted adding any significant features at all for many years. They did gradually add some elements from languages like Scala and Kotlin, once they had proven that those features made really significant improvements to real code rather than just looking good in isolation. It's the Honda Civic of languages. It's not especially fun and it doesn't keep up with the times. You just get in and go so that you can get something done.
- MrBuddyCasino 3y ago> It's the Honda Civic of languages That’s Golang. Java 21 is Toyota: unmatched stability, continuity and bang-for-buck in a deceptively simple shell that hides some of the most advanced tech there is.
- shepherdjerred 3y ago> It's not flashy or exciting or quick. Java's performance is fine for the vast majority of applications. Similar languages perform about as well. The JVM is an incredible piece of technology. > It's not especially fun I love Java, even though I don't write it much anymore. The ecosystem is incredibly mature. There are hundreds if not thousands of high quality libraries.
- paulddraper 3y ago> It's the Honda Civic of languages. Honda Civic doesn't devoure RAM.
- 3y ago
- zb3 3y agoWhat's wrong with imperative versions of these examples? Is imperative code not allowed or something?
- SomeDaysBe 3y agoStreams can tend to be more readable and easier to write for the right problem set. This is because it is super easy to compose basic functions together which are then invoked by the Stream API. It also makes it really easy to parallelize operations.
- smrtinsert 3y agoFor code coverage purposes I love the fact that there's only one branch of code to focus on. A lot of my code with streams instantly starts with a return statement and then has a 5-10 lines of stream operation invocations. Very clean stuff.
- jvmboi 3y agoStreams are fine and Java is clearly evolving in a good direction but what makes me tear out my hairs is that they needed 7 years to add a toList() method to streams. This is so typical for the way that the Java language evolved. Ergonomics always came last with Java and if it's not clunky and painful to use at first then the Java stewards don't want it. Maybe in another 7 years we will get filter() and map() directly on Collection.
- samus 3y agoJava has always preferred introducing features late over introducing something that they later can't get rid off anymore. Whether something is truly useful can often be said only in hinsight. Streams are in the standard library after all. To enable more rapid iteration, it would have been smarter to place it in a separate package of a `jdk.*` module (like `jdk.httpserver`) for which weaker compatibility guarantees apply. But that would completely divorce it from the other collection classes. Fixing that would require adding extension methods to Java.
- jvmboi 3y agoIs it really a new feature to have the most traveled path be comfortable? I wouldn't agree. I think whoever is in charge of Java should be conservative and careful but what I would just wish is for them to give the developer experience some priority.
- smrtinsert 3y agoI'd rather they defer decisions until they are sure. Libraries came out immediately to help with verbosity complaints around streams. I'd rather each step they take be sure-footed.
- jiehong 3y agoThey haven’t referenced clojure transducers, and I find that surprising. Wouldn’t that be a bit of a nicer interface?
- shepherdjerred 3y agoStreams are one of my favorite feature in Java, and this looks like a great step in making them even more flexible.
- zactato 3y agoThere's a lot of haters in this thread. You don't need to use Java. I don't use Java day to day anymore, but I'm excited to see the language evolve. I would highly recommend watching some of the talk from the Java architects. They're very smart and have thought a lot about how to design languages. Java obviously prioritizes different things than newer language, so decisions are always made in the context of maintain backwards compatibility.
- randomnoun 3y agoSo yes, naming is one of those qualitative things, but the term 'Gatherer' doesn't sound great to my ears, and introducing a new 'Integrator' term to represent the Gatherer equivalent of a Collector's accumulator doesn't seem necessary either. In terms of streams, it seems like this is a fancier 'pipe' ( in unix terms ), so I would have called it .pipe() instead of .gather() ... having a series of .pipe()s seems more intuitive than having a series of .gather()s on a stream.
- pazderski 3y agoThe example to find suspicious temperature changes (btw Kelvin does not use ° anymore since 1968) is unnecessary complex. List<List<Reading>> findSuspicious(Stream<Reading> source) { var suspicious = new ArrayList<List<Reading>>(); Reading previous = null; for (Reading next : source.toList()) { if (previous != null && isSuspicious(previous, next)) suspicious.add(List.of(previous, next)); previous = next; } return suspicious; } Doesn't look bad to me. Actually the streaming example here is quite underwhelming. It still needs a separate check for the first "iteration" (window.size() == 2). While reading the classic for-loop example I expected that especially that is something the streaming approach can handle better. In my opinion people overdo streams. I'm not sure I ever saw someone actually using the parallelization which is one of the stronger features of streams. Instead I get entire method bodies with a lot going on all shoved in one line which is for good reason discouraged anywhere else. And streams are tedious to debug. I really dislike when people (auto)refactor a 'for(var x : list) { ... }' into a 'list.stream.forEach(x -> { ... })' without any other benefit/change. Just this week I encountered a quite crazy case where someone wrote a IntStream.iterate(0, i -> i < getSize(), i -> i + 1).mapToObj(this::getObjectByIndex).toList() Two anonymous classes and who knows how many extra method calls just to not write a simple for-loop where you even could initialize the list with the known size.