10 ms·

Many Go programmers would claim this is cleaner and more understandable, unfortunately I'm not one of them.

by kubb 2y ago

int_19h 2y ago
The bigger problem is that it's not efficiently composable. If your caller needs to additionally filter `out`, that's another loop with a copy.
- deleted 2y ago
  [deleted]
- thegeekpirate 2y ago
  The proper term is deforestation https://en.wikipedia.org/wiki/Deforestation_(computer_science) https://en.wikipedia.org/wiki/Deforestation_(computer_scienc..., and I have seen Go libraries which do this
  - nemo1618 2y ago
    ah, that's what it's called! I did this in my compile-to-Go language, but I didn't know the name for it: https://github.com/lukechampine/ply?tab=readme-ov-file#supported-optimizations https://github.com/lukechampine/ply?tab=readme-ov-file#suppo...
- jerf 2y ago
  In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow. As a sibling notes, it is possible to "fix" that, but the fix is not even remotely free. It is quite complicated to do it generically. (Although there are other benefits; the most natural approach to that largely fixes my grumblings about refactoring I mentioned in another post.) Plus, in a for loop approach, it is not true that the caller may need another loop with a copy. They may just loop over the result and skip over the things they don't need. They only need a copy if they are going to pass that on to something else. A drum I can not stop banging on is that you can not just take a neat technique out of one language and slam it into another without examining the end result to make sure that you haven't murdered the cost/benefits tradeoff. You can slam together all the maps and filters and reduces you want in Haskell, and applicatives and monads and all the fun, due to a combination of the laziness and various safe optimizations like loop fusion. In an eager context that lacks loop fusion, going .Map().Map().Map().Map() has radically different performance implications. For instance, "take 10 $ map f list" in Haskell will only call "f" 10 times. .Map().Take(10) in most implementations will create the full array, however large it is, and slice ten off the end after that. In imperative languages, contrary to frequent claims from the functional programming crowd, for loops are actually often better in practice. The solution to their pathologies is to be aware of them and not do them. But it is far, far easier to get good performance out of a for loop in an imperative language than to contort one's code into a pale parody of functional programming.
  - akkad33 2y ago
    > In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow. Which imperative language? In Java and rust, two languages I know, all these operations are lazy until the final collect. So no copy is made
    - nine_k 2y ago
      Nor in Python. Nor in C++20's std::views. The very point of iterators us to proceed by one element, avoiding intermediate copies of collections. One case where the for-loop would be much more efficient is a simple transformation like incrementing each element of an array, or adding two arrays element-wise. A serious C compiler could unroll such a loop into several vector instructions which work on several elements at once. Maybe even LLVM can recognize something like that in Go code.
    - jerf 2y ago
      That's why I specified the code. If you're writing .Map().Map().Map().Map(), you are usually getting a lot of intermediate arrays. If you have a .Collect() call, you're in the deforestation branch. This has its own issues with stressing the inliner (turning simple, direct for loops over large collections into traversals that include several indirect method calls per item in addition to the payload is pretty easy), but that's still generally better than stressing the RAM. Rust's map doesn't operate on arrays at all from what I can see but operates on iterators directly. This is good and generally more correct. However there's a lot of languages that don't support that. Rust is generally also going to be more reliable about compiling it all away than a lot of other languages where it will be really easy to spill over what the inliner can handle. Those long compile times in Rust do have their benefits. There's also languages that sort of split the difference, e.g., it is not that difficult in Python to use itertools and generators to correctly write something that will not generate a lot of intermediate arrays, but it is also easy to write a series of calls and list comprehensions to write otherwise equivalent code that will create a lot of intermediate arrays. I expect as we continue to build new languages over time they're going to all look like Rust here. It's pretty obvious that conceiving of loops as iteration over some sequence is the way to go. However, that is a result that we got to precisely because of our experiences with a lot of languages that don't support it as well, or support it inconsistently, or as is actually quite common, the language nominally supports it but the ecosystem tends to assume concrete values a lot more often than it should, and all these languages are still around. Writing in this style correctly in imperative code is more difficult than a lot of people jumping up and down about how we should rewrite all our loops as maps and filters tend to account for. It can be done, but it's often harder than it looks, in at least one of the writing and the performance if not both, and the harder it is, the more the costs stack up on the costs side, and the harder the costs/benefits analysis becomes. And I still don't like how it refactors in most cases.
margalabargala 2y ago
The original version relied on no fewer than five different imported functions from a package that, if you're not already familiar with them, you would have to go read to see exactly what they do. The updated version has just one such function. A function gets less readable as you have to go context switch to more and more different functions to understand what it's doing. Yes, the `slices` functions are named such that you can make a decent educated guess as to what they're doing, but if there's a problem you're trying to debug that doesn't save you from having to dive into each one.
- deleted 2y ago
  [deleted]
- scubbo 2y ago
  > A function gets less readable as you have to go context switch to more and more different functions to understand what it's doing. This is true. A function also gets less readable every time you have to mentally translate from low-level implementation code, to a high-/human-level description of "what is this actually doing?" (e.g. `val := a[len(a)-idx-1]` - "ohhh, ok, that's iterating in reverse"). Extracting out common patterns to methods with well-known names like `Filter` or `Map` or `ForEach` short-circuits that repeated parsing. > that doesn't save you from having to dive into each one. 99% of the time, it really does. If you can't trust something as basic and well-defined as `Map` to do what you expect it to do, you've chosen a poor library on which to take a dependency.
  - margalabargala 2y ago
    > 99% of the time, it really does. If you can't trust something as basic and well-defined as `Map` to do what you expect it to do, you've chosen a poor library on which to take a dependency. The original comment said: it = slices.Map(it) I understand Map taking an array and a function, as in the article. I don't understand Map taking simply an array. Then I went to look it up, and turns out that slices.Map does not in fact exist. https://pkg.go.dev/slices https://pkg.go.dev/slices Personally I feel like this exchange is a vivid illustration of exactly my original point.
coin 2y ago
Go programming style is like programming in 2005 before functional programming because widespread (Google Guava, Clojure). If you showed functional programming to programmers in 2005 many of them would find it difficult to read.

Many Go programmers would claim this is cleaner and more understandable, unfortunately I'm not one of them.

by kubb 2y ago

Many Go programmers would claim this is cleaner and more understandable, unfortunately I'm not one of them.

The bigger problem is that it's not efficiently composable. If your caller needs to additionally filter `out`, that's another loop with a copy.

deleted 2y ago
[deleted]
thegeekpirate 2y ago
The proper term is deforestation https://en.wikipedia.org/wiki/Deforestation_(computer_science) https://en.wikipedia.org/wiki/Deforestation_(computer_scienc..., and I have seen Go libraries which do this
- nemo1618 2y ago
  ah, that's what it's called! I did this in my compile-to-Go language, but I didn't know the name for it: https://github.com/lukechampine/ply?tab=readme-ov-file#supported-optimizations https://github.com/lukechampine/ply?tab=readme-ov-file#suppo...
jerf 2y ago
In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow. As a sibling notes, it is possible to "fix" that, but the fix is not even remotely free. It is quite complicated to do it generically. (Although there are other benefits; the most natural approach to that largely fixes my grumblings about refactoring I mentioned in another post.) Plus, in a for loop approach, it is not true that the caller may need another loop with a copy. They may just loop over the result and skip over the things they don't need. They only need a copy if they are going to pass that on to something else. A drum I can not stop banging on is that you can not just take a neat technique out of one language and slam it into another without examining the end result to make sure that you haven't murdered the cost/benefits tradeoff. You can slam together all the maps and filters and reduces you want in Haskell, and applicatives and monads and all the fun, due to a combination of the laziness and various safe optimizations like loop fusion. In an eager context that lacks loop fusion, going .Map().Map().Map().Map() has radically different performance implications. For instance, "take 10 $ map f list" in Haskell will only call "f" 10 times. .Map().Take(10) in most implementations will create the full array, however large it is, and slice ten off the end after that. In imperative languages, contrary to frequent claims from the functional programming crowd, for loops are actually often better in practice. The solution to their pathologies is to be aware of them and not do them. But it is far, far easier to get good performance out of a for loop in an imperative language than to contort one's code into a pale parody of functional programming.
- akkad33 2y ago
  > In most imperative languages, writing .Map().Map().Filter().Map() is another full copy for each call anyhow. Which imperative language? In Java and rust, two languages I know, all these operations are lazy until the final collect. So no copy is made
  - nine_k 2y ago
    Nor in Python. Nor in C++20's std::views. The very point of iterators us to proceed by one element, avoiding intermediate copies of collections. One case where the for-loop would be much more efficient is a simple transformation like incrementing each element of an array, or adding two arrays element-wise. A serious C compiler could unroll such a loop into several vector instructions which work on several elements at once. Maybe even LLVM can recognize something like that in Go code.
  - jerf 2y ago
    That's why I specified the code. If you're writing .Map().Map().Map().Map(), you are usually getting a lot of intermediate arrays. If you have a .Collect() call, you're in the deforestation branch. This has its own issues with stressing the inliner (turning simple, direct for loops over large collections into traversals that include several indirect method calls per item in addition to the payload is pretty easy), but that's still generally better than stressing the RAM. Rust's map doesn't operate on arrays at all from what I can see but operates on iterators directly. This is good and generally more correct. However there's a lot of languages that don't support that. Rust is generally also going to be more reliable about compiling it all away than a lot of other languages where it will be really easy to spill over what the inliner can handle. Those long compile times in Rust do have their benefits. There's also languages that sort of split the difference, e.g., it is not that difficult in Python to use itertools and generators to correctly write something that will not generate a lot of intermediate arrays, but it is also easy to write a series of calls and list comprehensions to write otherwise equivalent code that will create a lot of intermediate arrays. I expect as we continue to build new languages over time they're going to all look like Rust here. It's pretty obvious that conceiving of loops as iteration over some sequence is the way to go. However, that is a result that we got to precisely because of our experiences with a lot of languages that don't support it as well, or support it inconsistently, or as is actually quite common, the language nominally supports it but the ecosystem tends to assume concrete values a lot more often than it should, and all these languages are still around. Writing in this style correctly in imperative code is more difficult than a lot of people jumping up and down about how we should rewrite all our loops as maps and filters tend to account for. It can be done, but it's often harder than it looks, in at least one of the writing and the performance if not both, and the harder it is, the more the costs stack up on the costs side, and the harder the costs/benefits analysis becomes. And I still don't like how it refactors in most cases.

margalabargala 2y ago

The original version relied on no fewer than five different imported functions from a package that, if you're not already familiar with them, you would have to go read to see exactly what they do. The updated version has just one such function. A function gets less readable as you have to go context switch to more and more different functions to understand what it's doing. Yes, the `slices` functions are named such that you can make a decent educated guess as to what they're doing, but if there's a problem you're trying to debug that doesn't save you from having to dive into each one.

deleted 2y ago
[deleted]
scubbo 2y ago
> A function gets less readable as you have to go context switch to more and more different functions to understand what it's doing. This is true. A function also gets less readable every time you have to mentally translate from low-level implementation code, to a high-/human-level description of "what is this actually doing?" (e.g. `val := a[len(a)-idx-1]` - "ohhh, ok, that's iterating in reverse"). Extracting out common patterns to methods with well-known names like `Filter` or `Map` or `ForEach` short-circuits that repeated parsing. > that doesn't save you from having to dive into each one. 99% of the time, it really does. If you can't trust something as basic and well-defined as `Map` to do what you expect it to do, you've chosen a poor library on which to take a dependency.
- margalabargala 2y ago
  > 99% of the time, it really does. If you can't trust something as basic and well-defined as `Map` to do what you expect it to do, you've chosen a poor library on which to take a dependency. The original comment said: it = slices.Map(it) I understand Map taking an array and a function, as in the article. I don't understand Map taking simply an array. Then I went to look it up, and turns out that slices.Map does not in fact exist. https://pkg.go.dev/slices https://pkg.go.dev/slices Personally I feel like this exchange is a vivid illustration of exactly my original point.

coin 2y ago

Go programming style is like programming in 2005 before functional programming because widespread (Google Guava, Clojure). If you showed functional programming to programmers in 2005 many of them would find it difficult to read.