7 ms·

Maybe a heretical opinion here, but I like writing loops much more than using higher order functions like map/fold/filter, even though I have quite a bit of exp

Maybe a heretical opinion here, but I like writing loops much more than using higher order functions like map/fold/filter, even though I have quite a bit of experience with ML family languages by now. These are my reasons: 1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. 2) Loops are a single construct you learn once and apply everywhere, while HOFs are a huge and bewildering zoo. 3) Loops can do many things that HOFs cannot. You can do break/continue/return in the loop body. You can iterate over two collections at once, skipping elements of one collection depending on what you see in the other. And so on. 4) Loops scale to from simple use cases to complicated ones gradually and continuously. With HOFs, when your use case changes slightly (e.g. you add a counter), you often need to go all the way up and use a different HOF. 5) Loops make the time and space complexity much more obvious. With HOFs, you often get nasty surprises. For example, see the tail-recursive vs non-tail-recursive implementations of "map" in OCaml, or the foldr vs foldl vs foldl' situation in Haskell. 6) Loops are much easier to understand from a machine point of view. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C. In a nutshell, loops are easier to learn, easier to read, easier to write, and easier to execute.

orangecat 11y ago
Loops can do many things that HOFs cannot. That's an advantage of HOFs; when you see one, you know exactly what it does, compared to loops where anything can happen either intentionally or accidentally. It's a form of the principle of least power. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C. C uses HOFs in many places, like the compare function in qsort.
- vanderZwan 11y ago
  > when you see one, you know exactly what it does, compared to loops where anything can happen either intentionally or accidentally. That is debatable, which is shown by the parent comment decrying the "bewildering zoo" of HOFs. I could flip this argument around and say that for loops have all complexity explicitly written out using fairly simple low-level constructs, whereas with HOFs I have to remember the semantics of each of them. And it would be equally unconvincing.
michaelfeathers 11y ago
The problem I have with loops is that they are like fly-paper. Once you have a loop iterating over a collection it's tempting to add more and more to it, mixing responsibilities in run-on code. This isn't theoretical, I see code like this all of the time. It's nearly impossible to abuse HOFs that way.
- cousin_it 11y ago
  Yeah, I agree. "Fly-paper" sounds a bit judgmental though. For a more neutral metaphor, we could say that HOFs are like Lego bricks, and loops are like clay. Both have their benefits and drawbacks. I've been fascinated by HOFs for many years, but my personal projects are filled with "loop abuse" and mixed responsibilities, all soft and malleable, just the way I like it. If I had to dismantle half the thing just to change the color of one brick, I'd give up pretty quickly.
  - michaelfeathers 11y ago
    That sort of change is easy - at least you have bricks. Clay just oozes all over itself. Whatever works for you. If it's just your code, you have enough control to keep loops from going bad. Large multi-programmer projects seem far more prone to that problem.
  - agumonkey 11y ago
    I second gp comment. HOF and FP in general constraints programming so much it leads to decoupled logic. Sometimes too much, but it's a good way to balance imperative statements allowing too much state.
  - reagency 11y ago
    Flypaper accumulates more bugs than clay.
- s_kilk 11y ago
  Yeah, I see this all the time. Giant 600 line functions made of nested for loops, with all sorts of shit going on at each layer. The author will insist it's basically impossible to abstract any of it, that the for-loops are the only way to capture the essence of the problem. Then you spend two days tracking down a bug in there and discover that it's all actually some combination of map, reduce and a fold with a closure at heart.
DannoHung 11y ago
> 1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. Haskell generally won't parallelize unless you tell it to. HOFs are not specifically for parallelization, however the cost associated with switching from a serial HOF to a parallel HOF can be very, very low. The same is absolutely not true for loops. There are some interesting things available with SIMD macros, but that's not parallelization in the way you're referring to it. > 2) Loops are a single construct you learn once and apply everywhere, while HOFs are a huge and bewildering zoo. This is sort of true, but as noted in a sibling comment, this is not necessarily a good thing. Each loop construct can behave differently. The invariant could change, the initial index might be different, the increment is arbitrary. None of those are things you are supposed to do, but they are all stepps taken used to obtain a behavior. When you are using HOF's, each one has a specific purpose and behavior and they can be understood in isolation and then in as many or few compositions as desired > 3) Loops can do many things that HOFs cannot. You can do break/continue/return in the loop body. You can iterate over two collections at once, skipping elements of one collection depending on what you see in the other. And so on. I'll agree on this point, mostly. They are strictly more powerful when you want to do something complicated. You do pay for the power with complexity. My one quibble is that it isn't necessarily that hard to do co-iteration, and let's be honest, most co-iteration is done 1 for 1. > 4) Loops scale to from simple use cases to complicated ones gradually and continuously. With HOFs, when your use case changes slightly (e.g. you add a counter), you often need to go all the way up and use a different HOF. I don't see this as even a minor issue. In fact, it's kind of the point. If you need to do something different, create a new composition of functions and as long as they have the same type signatures and the operation is what you now want, there's no problem at all. > 5) Loops make the time and space complexity much more obvious. With HOFs, you often get nasty surprises. For example, see the tail-recursive vs non-tail-recursive implementations of "map" in OCaml, or the foldr vs foldl vs foldl' situation in Haskell. Except when they don't, as in the case where you are doing manipulations of the index or loop invariants; one of the things you cited as a benefit of loops. I don't know about OCaml, but foldr and foldl are strictly different behaviors. One folds from the left, the other folds from the right, which is seriously important if you're using a non-associative operation. Now, maybe you want to say this is backing your point up, but write a for loop that does a non-associative left fold over a single linked list and we'll talk about obvious complexity. To make an aside, are you maybe thinking about this in terms of Haskell's lists vs C arrays? The fact that the data structures are so, so different makes a huge impact on how you work with them. foldl vs foldl' has nothing to do with HOF in particular, but Haskell's non-strict evaluation semantics. If you want to argue about strictness vs non-strictness though, that's a totally valid thing where I don't think there's any single dominant viewpoint. Non-strictness has certain benefits, but it's not necessarily worth it, especially if you can ask for limited non-strictness with otherwise strict semantics. 6) Loops are much easier to understand from a machine point of view. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C. You don't need a language with closures. Closures are definitely a great thing to have in general, but all you need is first class functions. You can write HOF's in C if you want to. It's a pain in the ass because function pointers are annoying as all hell, but you can absolutely do it. Overall, I'll say that there's probably a point to be made that there's a bit of overhead in learning all the different and varied HOF's that work with a function, but I don't really think it's that much larger than becoming familiar with any given pattern you can implement in a loop construct. The difference being that you implement the loop construct every time you want to use it. With a HOF, you just refer to the one you want to use.
runT1ME 11y ago
>1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. Maybe in your world, but not in Scala. Happens all the time in scala, if you're using a HOF on any monadic context it's trivial to add additional context to allow for parallelism without changing the underlying structure of any subsequent code.
- javajosh 11y ago
  My hobby: probabilistically down-voting jargon-laden comments on HN. Mere jargon gets a 50% chance of downvote (I flip a coin); that shoots to 100% when someone uses jargon and the word "trivial".
  - runT1ME 11y ago
    So, in addition to knowingly violating HN etiquette you are bragging about your ignorance? Tell me why what I said isn't 'trivial'? If you aren't familiar with the concepts, then how could you even know if what I said was simple or not?
    - javajosh 11y ago
      >if you're using a HOF on any monadic context it's trivial to add additional context to allow for parallelism without changing the underlying structure of any subsequent code. If that's not head-up-your-ass jargon, then I don't know what is. And I'll continue to down-vote stuff like that. Writing is about communicating, not demonstrating how smart you are. Abusing me as ignorant tells me you have little interest in communicating, and confirms you were posturing. And how do you know what I am or am not familiar with? Jargon is orthogonal to content.
      - runT1ME 11y ago
        >To me, docker can be thought of as a process wrapper. The executable is called an image, and the running process called a container. That's from a comment of yours. It's meaningless to me because I don't know much about deployment/dev-ops etc. Now I could lash out out of some insecurity and say that you're just using "jargon", or I could take some responsibility and look up those words.
munificent 11y ago
Personally, I like both for loops and higher-order functions over sequences. Loops can be more powerful, but they force the reader into an imperative mindset where they have to mentally step through the code to understand what it's doing. That makes sense when the imperative model is the natural way to understand the code, but it can be overkill when there is always a simple high-level way to understand the operation. Higher-order functions give you that. If I'm filtering a sequence "filter()" makes that fast and easy to read. But once I start chaining more than a few together, or start passing large lambdas to them, they lose that higher level reasoning and aren't worth it. One way to look at it is, "How will the reader most easily think about this code?" If it's as a series of high-level transformations, use HOFs. If it's as imperative code, use a loop. Another way is, "Does this code need to do control flow?" If so, use a statement. That's what they're good at. Otherwise, HOFs work fine. I do find myself converting code from one form to another as it changes over time. I wish that transition cost was lower but I don't find it to be a big deal.
codygman 11y ago
> The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. It's just a s/map/parMap/g away. Sometimes parallelization isn't worth it though. Are you talking about automatic parallelization as something that parallelizes anything possible or that also analyzes whether it's a good idea?
cpprototypes 11y ago
The functional style eliminates types of bugs that can happen with for loops and makes the code easier to read. For example, consider the task of removing names that start with "B" from a list. This is the for loop way: List<String> names = Lists.newArrayList("Alice", "Tom", "Bob", "Brandon", "John", "James", "Ben"); List<String> bNames = new ArrayList<>(); for (String name : names) { if (name.startsWith("B")) { bNames.add(name); } } names.removeAll(bNames); And the functional way: names = names.stream().filter(name -> !name.startsWith("B")).collect(Collectors.toList()); The basic problem with the for loop is that you need create a temporary list to do the work. It's low level coding that requires telling the computer exactly where to put the temporary results, then what to do with it. I've seen bugs where developers accidentally return the wrong list (such as bNames in this example) or later modify the wrong one. When the code becomes more complicated, there's often a lot of these temporary variables that greatly reduce code readability and allow subtle bugs to occur.
- newgame 11y ago
  What about iterating from the end of the list in this particular case. This way you don't need a temporary list for (int i = names.size()-1; i >= 0; i--) { String name = names.get(i); if (name.startsWith("B")) names.remove(i); } Admittedly, needing to use an explicit index counter is not as nice (and more prone to errors) as using the other for syntax. But one could imagine a language with e.g. macros that made the backwards-looping syntax more intuitive (I assume a single-threaded situation and an ArrayList). Your general point still stands though.
  - wcummings 11y ago
    Both implementations are incorrect as they needlessly mutate the input list: List<String> namesNotStartingWithB = new ArrayList<>(); for (String name : names) { if (!name.startsWith("B")) { namesNotStartingWithB.add(name); } } Imagine a scenario where you're filtering one of the arguments to a method: the input will be mutated with no indication to the caller (or in the method signature), causing bugs, iterator invalidation etc.
cle 11y ago
Personally, I find functional styles to be easier to understand. Transforming data by composing functions is easier for me to understand because each step generally stands by itself, with no state to keep track of across steps. This generally leads to less buggy code. Also, I find that manually managing control flow leads to a lot of bugs. If I have tricky for-loop with lots of continues and breaks, the complexity leads to subtle bugs. I try to avoid those features as much as possible, deferring control flow to functional composition. In general, there are less edge cases to manage with a functional style (control flow, null collections, etc.). There are definitely performance reasons to use traditional for-loops, but the vast majority of the time it's appropriate to sacrifice performance for stability and readability.

orangecat 11y ago

Loops can do many things that HOFs cannot. That's an advantage of HOFs; when you see one, you know exactly what it does, compared to loops where anything can happen either intentionally or accidentally. It's a form of the principle of least power. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C. C uses HOFs in many places, like the compare function in qsort.

vanderZwan 11y ago
> when you see one, you know exactly what it does, compared to loops where anything can happen either intentionally or accidentally. That is debatable, which is shown by the parent comment decrying the "bewildering zoo" of HOFs. I could flip this argument around and say that for loops have all complexity explicitly written out using fairly simple low-level constructs, whereas with HOFs I have to remember the semantics of each of them. And it would be equally unconvincing.

michaelfeathers 11y ago

The problem I have with loops is that they are like fly-paper. Once you have a loop iterating over a collection it's tempting to add more and more to it, mixing responsibilities in run-on code. This isn't theoretical, I see code like this all of the time. It's nearly impossible to abuse HOFs that way.

cousin_it 11y ago
Yeah, I agree. "Fly-paper" sounds a bit judgmental though. For a more neutral metaphor, we could say that HOFs are like Lego bricks, and loops are like clay. Both have their benefits and drawbacks. I've been fascinated by HOFs for many years, but my personal projects are filled with "loop abuse" and mixed responsibilities, all soft and malleable, just the way I like it. If I had to dismantle half the thing just to change the color of one brick, I'd give up pretty quickly.
- michaelfeathers 11y ago
  That sort of change is easy - at least you have bricks. Clay just oozes all over itself. Whatever works for you. If it's just your code, you have enough control to keep loops from going bad. Large multi-programmer projects seem far more prone to that problem.
- agumonkey 11y ago
  I second gp comment. HOF and FP in general constraints programming so much it leads to decoupled logic. Sometimes too much, but it's a good way to balance imperative statements allowing too much state.
- reagency 11y ago
  Flypaper accumulates more bugs than clay.
s_kilk 11y ago
Yeah, I see this all the time. Giant 600 line functions made of nested for loops, with all sorts of shit going on at each layer. The author will insist it's basically impossible to abstract any of it, that the for-loops are the only way to capture the essence of the problem. Then you spend two days tracking down a bug in there and discover that it's all actually some combination of map, reduce and a fold with a closure at heart.

DannoHung 11y ago

> 1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. Haskell generally won't parallelize unless you tell it to. HOFs are not specifically for parallelization, however the cost associated with switching from a serial HOF to a parallel HOF can be very, very low. The same is absolutely not true for loops. There are some interesting things available with SIMD macros, but that's not parallelization in the way you're referring to it. > 2) Loops are a single construct you learn once and apply everywhere, while HOFs are a huge and bewildering zoo. This is sort of true, but as noted in a sibling comment, this is not necessarily a good thing. Each loop construct can behave differently. The invariant could change, the initial index might be different, the increment is arbitrary. None of those are things you are supposed to do, but they are all stepps taken used to obtain a behavior. When you are using HOF's, each one has a specific purpose and behavior and they can be understood in isolation and then in as many or few compositions as desired > 3) Loops can do many things that HOFs cannot. You can do break/continue/return in the loop body. You can iterate over two collections at once, skipping elements of one collection depending on what you see in the other. And so on. I'll agree on this point, mostly. They are strictly more powerful when you want to do something complicated. You do pay for the power with complexity. My one quibble is that it isn't necessarily that hard to do co-iteration, and let's be honest, most co-iteration is done 1 for 1. > 4) Loops scale to from simple use cases to complicated ones gradually and continuously. With HOFs, when your use case changes slightly (e.g. you add a counter), you often need to go all the way up and use a different HOF. I don't see this as even a minor issue. In fact, it's kind of the point. If you need to do something different, create a new composition of functions and as long as they have the same type signatures and the operation is what you now want, there's no problem at all. > 5) Loops make the time and space complexity much more obvious. With HOFs, you often get nasty surprises. For example, see the tail-recursive vs non-tail-recursive implementations of "map" in OCaml, or the foldr vs foldl vs foldl' situation in Haskell. Except when they don't, as in the case where you are doing manipulations of the index or loop invariants; one of the things you cited as a benefit of loops. I don't know about OCaml, but foldr and foldl are strictly different behaviors. One folds from the left, the other folds from the right, which is seriously important if you're using a non-associative operation. Now, maybe you want to say this is backing your point up, but write a for loop that does a non-associative left fold over a single linked list and we'll talk about obvious complexity. To make an aside, are you maybe thinking about this in terms of Haskell's lists vs C arrays? The fact that the data structures are so, so different makes a huge impact on how you work with them. foldl vs foldl' has nothing to do with HOF in particular, but Haskell's non-strict evaluation semantics. If you want to argue about strictness vs non-strictness though, that's a totally valid thing where I don't think there's any single dominant viewpoint. Non-strictness has certain benefits, but it's not necessarily worth it, especially if you can ask for limited non-strictness with otherwise strict semantics. 6) Loops are much easier to understand from a machine point of view. For HOFs, you need a language with closures, and in some cases GC as well. Loops, on the other hand, can be done in C. You don't need a language with closures. Closures are definitely a great thing to have in general, but all you need is first class functions. You can write HOF's in C if you want to. It's a pain in the ass because function pointers are annoying as all hell, but you can absolutely do it. Overall, I'll say that there's probably a point to be made that there's a bit of overhead in learning all the different and varied HOF's that work with a function, but I don't really think it's that much larger than becoming familiar with any given pattern you can implement in a loop construct. The difference being that you implement the loop construct every time you want to use it. With a HOF, you just refer to the one you want to use.

runT1ME 11y ago

>1) The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. Maybe in your world, but not in Scala. Happens all the time in scala, if you're using a HOF on any monadic context it's trivial to add additional context to allow for parallelism without changing the underlying structure of any subsequent code.

javajosh 11y ago
My hobby: probabilistically down-voting jargon-laden comments on HN. Mere jargon gets a 50% chance of downvote (I flip a coin); that shoots to 100% when someone uses jargon and the word "trivial".
- runT1ME 11y ago
  So, in addition to knowingly violating HN etiquette you are bragging about your ignorance? Tell me why what I said isn't 'trivial'? If you aren't familiar with the concepts, then how could you even know if what I said was simple or not?
  - javajosh 11y ago
    >if you're using a HOF on any monadic context it's trivial to add additional context to allow for parallelism without changing the underlying structure of any subsequent code. If that's not head-up-your-ass jargon, then I don't know what is. And I'll continue to down-vote stuff like that. Writing is about communicating, not demonstrating how smart you are. Abusing me as ignorant tells me you have little interest in communicating, and confirms you were posturing. And how do you know what I am or am not familiar with? Jargon is orthogonal to content.
    - runT1ME 11y ago
      >To me, docker can be thought of as a process wrapper. The executable is called an image, and the running process called a container. That's from a comment of yours. It's meaningless to me because I don't know much about deployment/dev-ops etc. Now I could lash out out of some insecurity and say that you're just using "jargon", or I could take some responsibility and look up those words.

munificent 11y ago

Personally, I like both for loops and higher-order functions over sequences. Loops can be more powerful, but they force the reader into an imperative mindset where they have to mentally step through the code to understand what it's doing. That makes sense when the imperative model is the natural way to understand the code, but it can be overkill when there is always a simple high-level way to understand the operation. Higher-order functions give you that. If I'm filtering a sequence "filter()" makes that fast and easy to read. But once I start chaining more than a few together, or start passing large lambdas to them, they lose that higher level reasoning and aren't worth it. One way to look at it is, "How will the reader most easily think about this code?" If it's as a series of high-level transformations, use HOFs. If it's as imperative code, use a loop. Another way is, "Does this code need to do control flow?" If so, use a statement. That's what they're good at. Otherwise, HOFs work fine. I do find myself converting code from one form to another as it changes over time. I wish that transition cost was lower but I don't find it to be a big deal.

codygman 11y ago

> The automatic parallelization opportunities of HOFs don't happen in practice nearly as often as you'd expect. Sorry, but it's true. Haskell's "map" doesn't parallelize automatically. It's just a s/map/parMap/g away. Sometimes parallelization isn't worth it though. Are you talking about automatic parallelization as something that parallelizes anything possible or that also analyzes whether it's a good idea?

cpprototypes 11y ago

The functional style eliminates types of bugs that can happen with for loops and makes the code easier to read. For example, consider the task of removing names that start with "B" from a list. This is the for loop way: List<String> names = Lists.newArrayList("Alice", "Tom", "Bob", "Brandon", "John", "James", "Ben"); List<String> bNames = new ArrayList<>(); for (String name : names) { if (name.startsWith("B")) { bNames.add(name); } } names.removeAll(bNames); And the functional way: names = names.stream().filter(name -> !name.startsWith("B")).collect(Collectors.toList()); The basic problem with the for loop is that you need create a temporary list to do the work. It's low level coding that requires telling the computer exactly where to put the temporary results, then what to do with it. I've seen bugs where developers accidentally return the wrong list (such as bNames in this example) or later modify the wrong one. When the code becomes more complicated, there's often a lot of these temporary variables that greatly reduce code readability and allow subtle bugs to occur.

newgame 11y ago
What about iterating from the end of the list in this particular case. This way you don't need a temporary list for (int i = names.size()-1; i >= 0; i--) { String name = names.get(i); if (name.startsWith("B")) names.remove(i); } Admittedly, needing to use an explicit index counter is not as nice (and more prone to errors) as using the other for syntax. But one could imagine a language with e.g. macros that made the backwards-looping syntax more intuitive (I assume a single-threaded situation and an ArrayList). Your general point still stands though.
- wcummings 11y ago
  Both implementations are incorrect as they needlessly mutate the input list: List<String> namesNotStartingWithB = new ArrayList<>(); for (String name : names) { if (!name.startsWith("B")) { namesNotStartingWithB.add(name); } } Imagine a scenario where you're filtering one of the arguments to a method: the input will be mutated with no indication to the caller (or in the method signature), causing bugs, iterator invalidation etc.

cle 11y ago

Personally, I find functional styles to be easier to understand. Transforming data by composing functions is easier for me to understand because each step generally stands by itself, with no state to keep track of across steps. This generally leads to less buggy code. Also, I find that manually managing control flow leads to a lot of bugs. If I have tricky for-loop with lots of continues and breaks, the complexity leads to subtle bugs. I try to avoid those features as much as possible, deferring control flow to functional composition. In general, there are less edge cases to manage with a functional style (control flow, null collections, etc.). There are definitely performance reasons to use traditional for-loops, but the vast majority of the time it's appropriate to sacrifice performance for stability and readability.