14 ms·

I don't get why someone would rather write a for loop than use declarative data syntax. If I want to get the names of all administrators doing `users.Where(user

by inglor 12y ago

I don't get why someone would rather write a for loop than use declarative data syntax. If I want to get the names of all administrators doing `users.Where(user => user.isAdmin()).Select(user => user.Name)` is so much nicer than using a for loop - or maybe he's suggesting we start writing FOR loops instead of SQL for our databases too?

bunderbunder 12y ago
In most languages, the for loop ends up being faster. Sometimes noticeably so. Me, I generally start with declarative syntax, but the profiler frequently tells me to go back and change it. Being a systems programmer, wonder if it's easier for him to just use the for loop as a default. Performance demands are always high in systems programming (because there'll be a whole stack of additional software standing on top of your code and depending on it for performance), so one might end up needing to use for loops often enough that it's easier to just use them all the time for consistency's sake.
- drcode 12y ago
  That's the essence of Clojure's transducers http://clojure.org/transducers http://clojure.org/transducers ...they give you the declarative syntax of filter/reduce/etc but can have the same evaluation strategy as for loops, with comparable performance.
  - adrusi 12y ago
    That's not why clojure has transducers. You can use a mapping transducer to express map, but it's actually going to have slightly worse performance than regular map. There's nothing inherently slow about higher order collection functions, rust has them and they compile to the same machine code as the equivalent loop construct. Its just that most languages implement them on the wrong data structures. Functional languages implement them on lazy lists, which is good, but has overhead. A lot of languages implement them on arrays, which is bad because then it needs to process the whole array at once, and allocate all the memory, even though the next transformation in the pipeline doesn't need all that memory. Rust implements them on iterators, which have all the sane benefits as lazy lists, but fit better into a performance-focused imperative language.
    - bunderbunder 12y ago
      .NET implements them on iterators, too, but that's not an inherently fast approach. .NET iterators are objects that implement a specific interface, and iterating over them involves two function calls per element - one to move to the next item, and one to retrieve the current item. Those two function calls introduce an overhead that may be significant in a "tight loop", and that may not exist in a hand-coded for loop. Perhaps .NET's optimizer is smart enough to get things down to a for loop equivalent in some cases, but in general I haven't seen much evidence of that happening.
      - pcwalton 12y ago
        > .NET iterators are objects that implement a specific interface, and iterating over them involves two function calls per element - one to move to the next item, and one to retrieve the current item. Any optimizing compiler will inline those functions. "Avoid function calls for performance" hasn't been relevant optimization advice for at least a decade.
        inglor 12y ago
        In particular - the Microsoft .NET JIT will gladly make them as fast as loops in most cases - there might be overhead involving closures but that's mostly taken care of.
        bunderbunder 12y ago
        A good optimizer can inline those functions, but that doesn't necessarily mean it will. .NET JIT is conservative in what it decides to inline. There's even a special hint you can use to ask it to be more aggressive about inlining a method. You can't necessarily apply that hint to every function, though. Anonymous ones, for example.
        pcwalton 12y ago
        > A good optimizer can inline those functions, but that doesn't necessarily mean it will. If the optimizer isn't choosing to inline where it is a performance win, it's not a good optimizer. Especially for trivial functions like Current() and MoveNext(), which are pretty much the textbook example of functions where inlining really helps (not so much because of the function call overhead, but because it enables so many intraprocedural loop optimizations).
- deleted 12y ago
  [deleted]
- bjz_ 12y ago
  I'm not sure what languages you are referring to, but Rust and D's iterator methods compile to the same code that you would write if you were using raw loops.
- pcwalton 12y ago
  > In most languages, the for loop ends up being faster. Sometimes noticeably so. Me, I generally start with declarative syntax, but the profiler frequently tells me to go back and change it. The optimization techniques to make higher-order functions compile down into the same code as a for loop are well-known. All you have to do is inline, constant propagate, and maybe SROA. Every optimizing compiler I know of, even JavaScript, has no problem doing this.
  - Veedrac 12y ago
    FWIW, PyPy has a hard time with this since Python's iterators are really heavyweight.
svisser 12y ago
It's easy to dismiss an opinion but that doesn't mean it's wrong. For example, if you like filter/reduce you may criticize languages that don't encourage it (Go, Python, etc). But often this leads to copying ideas from one language resulting in code that's hard to maintain in another language which encourages different ways of expressing the same logic.
- metaphorm 12y ago
  filter and reduce are both builtins in Python and I see nothing discouraging anyone from taking full advantage of them.
  - svisser 12y ago
    In Python 3 reduce was removed and you'd have to use functools.reduce. Similarly lambda functions are a bit of a mistake, in Guido's opinion, as well as his preference to use list comprehensions over filter(). I don't expect to see more functional programming constructs anytime soon in Python. That being said, list comprehensions work quite well in Haskell, Python, etc.
  - deleted 12y ago
    [deleted]
  - kasabali 12y ago
    GvR discourages them if that's a strong enough reason for you :)
    - metaphorm 12y ago
      its not, actually. he seems to discourage them on the grounds that a list comprehension would be better. however, that's just alternate syntax for a logically equivalent operation. personally, I use whichever one is more clear to me based on context. I appreciate having the option. Maybe you say that's un-Pythonic because "there should be just one way to do things", but I'm not particularly dogmatic and not persuaded by that.
seanmcdirmid 12y ago
Debugging. Try debugging a Select/Where stream in C#; it is a PITA. I often find myself unwinding my list comprehensions into for loops because I need to debug. There is a reason Haskell's type system has to be so strong :)
- eropple 12y ago
  I was pretty confused by this post for a minute until I remembered I've been on VS2015 since the first preview release. This problem is greatly reduced there.
  - seanmcdirmid 12y ago
    I saw that in June, and I'm interested in how effective it is, and what the usability characteristics are (it looked complex at the time, but it was an early demo). But going from 2010 to 2013 was a painful, and I don't want to throw this in a VM, so I'll just wait for RTM.
    - eropple 12y ago
      2013 to 2015 is a much smaller jump. Unless you are doing weirder things than me (and I mean, that's kind of hard!), I'd bet you'd be party. YMMV, of course.
      - seanmcdirmid 12y ago
        I use a plugin that makes curly braces very small, so that my C# code looks more like Python. it seems to break on every version upgrade, and I can't stand normal sized C# curly braces anymore.
        eropple 12y ago
        I'd honestly be surprised if that broke this time around. Every extension I've tried in VS2015 worked fine. Some I had to fight with (VSIX manifests with too-restrictive versions), but it's mostly just drop-in-and-go. They do run side-by-side just fine, too, FWIW.
mhd 12y ago
Wasn't one of Pike's earliest published whitepapers basically stating that "most of what you need is binary trees, linked lists and hashes" when it came to data structures/algorithms? (Probably add CSP to that list, given his prior work) So I'd dare to say he's a bit of a minimalist. Fanboys and fellow travelers aside, one of the few persons with a similar point of view that I can think of would be Niklaus Wirth. But that's the luxury that research and academia offer to you, if your problems can be approach from a tabula rasa view, you can tailor your tools to be similary pure (his "pure" obviously not being of the mathematical/functional persuasion). I mean, there's no solid reason why we have several different kinds of screwdrivers. But that doesn't help you when you have to assemble your IKEA Wöbsörwös furniture to earn your living.
- fosforsvenne 12y ago
  The letter W isn't used in Swedish except in proper names and the occasional loanword.
  - mhd 12y ago
    And just when I felt so smart about using "ö" and not "ø" for fake furniture…
    - fosforsvenne 12y ago
      Actually, I both noticed and appreciated that :). And a lot of IKEA product names are just first names anyway, so it _was_ overly pedantic.
adrusi 12y ago
Because for loops are special forms to the compiler and don't require general parametric polymorphism to be implemented in the language. Parametric polymorphism adds a lot of complexity to the language, so I guess the go team has decided to just not have it in the language. I'm not sure I agree with that decision, but it is true that parametric polymorphism isn't all that useful for the types of programs go is usually used for. You don't want to use this implementation because it uses reflection, which makes it slow and means the compiler can't catch your errors.
NateDad 12y ago
var admins = users.Where(user => user.isAdmin()).Select(user => user.Name) vs var admins []string for _, user := range users { if user.IsAdmin() { admins = append(admins, user.Name) } } So, at the end of the day, these two pieces of code do the exact same thing, and have mostly the exact same meaning. I read them almost identically... Make a list of admins. iterate through the users, if the user is an admin, append its name to the list of admins. The unrolled version is a lot easier to modify later. What if you decide that each of these admin users should have a star next to their name? For the go code, you just add one line: var admins []string for _, user := range users { if user.IsAdmin() { admins = append(admins, user.Name) user.Name = user.Name + "*" } } for the C# code, you either have to write a second LINQ query (and now you're iterating the list twice unnecessarily), or you have to unroll it into a normal loop. So now you've added complexity to the language that is only really good for saving carriage returns, and it makes your code harder to maintain. Sure, it's pretty and interesting from a programming theory POV... but at the end of the day, I don't get paid to write pretty or interesting code. I do get paid to write maintainable code.
- inglor 12y ago
  I humbly disagree - in the C# code I can pass an IQueryable and reuse the same code to make an SQL query, or a RavenDB query, or an XML query and it works just the same - there is a unified way to access data in the language. As for your query: ``` var admins = users.Where(u => u.IsAdmin()).Select(u => u.Name); var stars = admins.Select(name => name + "*"); ``` Like you said, you don't get paid per line feed - it's ok to have multiple lines and you can keep the nice syntax. It's ok to not like complicated code - no one likes complicated code - which is why I don't like for loops - I don't _care_ how something is iterated I just want to map every object.
  - NateDad 12y ago
    Well, so, that's actually not doing the right thing. You just changed the names in the list of strings that holds admin names, but not the names on the user objects themselves. Even if the first half were returning the users themselves to the admins list, you're still iterating over the list twice. This is yet another reason why the for loop is good - it makes it more clear what is actually going on. This is your code in for loops: var admins []string for _, u := range users { if u.IsAdmin() { admins = append(admins, u.Name) } } for i := range admins { admins[i] = admins[i] + "*" } The problem with the fancy syntax is that it makes it too easy to write suboptimal code that you'd never write if you were writing plain old for loops, like the above.

by inglor 12y ago

bunderbunder 12y ago

In most languages, the for loop ends up being faster. Sometimes noticeably so. Me, I generally start with declarative syntax, but the profiler frequently tells me to go back and change it. Being a systems programmer, wonder if it's easier for him to just use the for loop as a default. Performance demands are always high in systems programming (because there'll be a whole stack of additional software standing on top of your code and depending on it for performance), so one might end up needing to use for loops often enough that it's easier to just use them all the time for consistency's sake.

drcode 12y ago
That's the essence of Clojure's transducers http://clojure.org/transducers http://clojure.org/transducers ...they give you the declarative syntax of filter/reduce/etc but can have the same evaluation strategy as for loops, with comparable performance.
- adrusi 12y ago
  That's not why clojure has transducers. You can use a mapping transducer to express map, but it's actually going to have slightly worse performance than regular map. There's nothing inherently slow about higher order collection functions, rust has them and they compile to the same machine code as the equivalent loop construct. Its just that most languages implement them on the wrong data structures. Functional languages implement them on lazy lists, which is good, but has overhead. A lot of languages implement them on arrays, which is bad because then it needs to process the whole array at once, and allocate all the memory, even though the next transformation in the pipeline doesn't need all that memory. Rust implements them on iterators, which have all the sane benefits as lazy lists, but fit better into a performance-focused imperative language.
  - bunderbunder 12y ago
    .NET implements them on iterators, too, but that's not an inherently fast approach. .NET iterators are objects that implement a specific interface, and iterating over them involves two function calls per element - one to move to the next item, and one to retrieve the current item. Those two function calls introduce an overhead that may be significant in a "tight loop", and that may not exist in a hand-coded for loop. Perhaps .NET's optimizer is smart enough to get things down to a for loop equivalent in some cases, but in general I haven't seen much evidence of that happening.
    - pcwalton 12y ago
      > .NET iterators are objects that implement a specific interface, and iterating over them involves two function calls per element - one to move to the next item, and one to retrieve the current item. Any optimizing compiler will inline those functions. "Avoid function calls for performance" hasn't been relevant optimization advice for at least a decade.
      - inglor 12y ago
        In particular - the Microsoft .NET JIT will gladly make them as fast as loops in most cases - there might be overhead involving closures but that's mostly taken care of.
      - bunderbunder 12y ago
        A good optimizer can inline those functions, but that doesn't necessarily mean it will. .NET JIT is conservative in what it decides to inline. There's even a special hint you can use to ask it to be more aggressive about inlining a method. You can't necessarily apply that hint to every function, though. Anonymous ones, for example.
        pcwalton 12y ago
        > A good optimizer can inline those functions, but that doesn't necessarily mean it will. If the optimizer isn't choosing to inline where it is a performance win, it's not a good optimizer. Especially for trivial functions like Current() and MoveNext(), which are pretty much the textbook example of functions where inlining really helps (not so much because of the function call overhead, but because it enables so many intraprocedural loop optimizations).
deleted 12y ago
[deleted]
bjz_ 12y ago
I'm not sure what languages you are referring to, but Rust and D's iterator methods compile to the same code that you would write if you were using raw loops.
pcwalton 12y ago
> In most languages, the for loop ends up being faster. Sometimes noticeably so. Me, I generally start with declarative syntax, but the profiler frequently tells me to go back and change it. The optimization techniques to make higher-order functions compile down into the same code as a for loop are well-known. All you have to do is inline, constant propagate, and maybe SROA. Every optimizing compiler I know of, even JavaScript, has no problem doing this.
- Veedrac 12y ago
  FWIW, PyPy has a hard time with this since Python's iterators are really heavyweight.

svisser 12y ago

It's easy to dismiss an opinion but that doesn't mean it's wrong. For example, if you like filter/reduce you may criticize languages that don't encourage it (Go, Python, etc). But often this leads to copying ideas from one language resulting in code that's hard to maintain in another language which encourages different ways of expressing the same logic.

metaphorm 12y ago
filter and reduce are both builtins in Python and I see nothing discouraging anyone from taking full advantage of them.
- svisser 12y ago
  In Python 3 reduce was removed and you'd have to use functools.reduce. Similarly lambda functions are a bit of a mistake, in Guido's opinion, as well as his preference to use list comprehensions over filter(). I don't expect to see more functional programming constructs anytime soon in Python. That being said, list comprehensions work quite well in Haskell, Python, etc.
- deleted 12y ago
  [deleted]
- kasabali 12y ago
  GvR discourages them if that's a strong enough reason for you :)
  - metaphorm 12y ago
    its not, actually. he seems to discourage them on the grounds that a list comprehension would be better. however, that's just alternate syntax for a logically equivalent operation. personally, I use whichever one is more clear to me based on context. I appreciate having the option. Maybe you say that's un-Pythonic because "there should be just one way to do things", but I'm not particularly dogmatic and not persuaded by that.

seanmcdirmid 12y ago

Debugging. Try debugging a Select/Where stream in C#; it is a PITA. I often find myself unwinding my list comprehensions into for loops because I need to debug. There is a reason Haskell's type system has to be so strong :)

eropple 12y ago
I was pretty confused by this post for a minute until I remembered I've been on VS2015 since the first preview release. This problem is greatly reduced there.
- seanmcdirmid 12y ago
  I saw that in June, and I'm interested in how effective it is, and what the usability characteristics are (it looked complex at the time, but it was an early demo). But going from 2010 to 2013 was a painful, and I don't want to throw this in a VM, so I'll just wait for RTM.
  - eropple 12y ago
    2013 to 2015 is a much smaller jump. Unless you are doing weirder things than me (and I mean, that's kind of hard!), I'd bet you'd be party. YMMV, of course.
    - seanmcdirmid 12y ago
      I use a plugin that makes curly braces very small, so that my C# code looks more like Python. it seems to break on every version upgrade, and I can't stand normal sized C# curly braces anymore.
      - eropple 12y ago
        I'd honestly be surprised if that broke this time around. Every extension I've tried in VS2015 worked fine. Some I had to fight with (VSIX manifests with too-restrictive versions), but it's mostly just drop-in-and-go. They do run side-by-side just fine, too, FWIW.

mhd 12y ago

Wasn't one of Pike's earliest published whitepapers basically stating that "most of what you need is binary trees, linked lists and hashes" when it came to data structures/algorithms? (Probably add CSP to that list, given his prior work) So I'd dare to say he's a bit of a minimalist. Fanboys and fellow travelers aside, one of the few persons with a similar point of view that I can think of would be Niklaus Wirth. But that's the luxury that research and academia offer to you, if your problems can be approach from a tabula rasa view, you can tailor your tools to be similary pure (his "pure" obviously not being of the mathematical/functional persuasion). I mean, there's no solid reason why we have several different kinds of screwdrivers. But that doesn't help you when you have to assemble your IKEA Wöbsörwös furniture to earn your living.

fosforsvenne 12y ago
The letter W isn't used in Swedish except in proper names and the occasional loanword.
- mhd 12y ago
  And just when I felt so smart about using "ö" and not "ø" for fake furniture…
  - fosforsvenne 12y ago
    Actually, I both noticed and appreciated that :). And a lot of IKEA product names are just first names anyway, so it _was_ overly pedantic.

adrusi 12y ago

Because for loops are special forms to the compiler and don't require general parametric polymorphism to be implemented in the language. Parametric polymorphism adds a lot of complexity to the language, so I guess the go team has decided to just not have it in the language. I'm not sure I agree with that decision, but it is true that parametric polymorphism isn't all that useful for the types of programs go is usually used for. You don't want to use this implementation because it uses reflection, which makes it slow and means the compiler can't catch your errors.

NateDad 12y ago

var admins = users.Where(user => user.isAdmin()).Select(user => user.Name) vs var admins []string for _, user := range users { if user.IsAdmin() { admins = append(admins, user.Name) } } So, at the end of the day, these two pieces of code do the exact same thing, and have mostly the exact same meaning. I read them almost identically... Make a list of admins. iterate through the users, if the user is an admin, append its name to the list of admins. The unrolled version is a lot easier to modify later. What if you decide that each of these admin users should have a star next to their name? For the go code, you just add one line: var admins []string for _, user := range users { if user.IsAdmin() { admins = append(admins, user.Name) user.Name = user.Name + "*" } } for the C# code, you either have to write a second LINQ query (and now you're iterating the list twice unnecessarily), or you have to unroll it into a normal loop. So now you've added complexity to the language that is only really good for saving carriage returns, and it makes your code harder to maintain. Sure, it's pretty and interesting from a programming theory POV... but at the end of the day, I don't get paid to write pretty or interesting code. I do get paid to write maintainable code.

inglor 12y ago
I humbly disagree - in the C# code I can pass an IQueryable and reuse the same code to make an SQL query, or a RavenDB query, or an XML query and it works just the same - there is a unified way to access data in the language. As for your query: ``` var admins = users.Where(u => u.IsAdmin()).Select(u => u.Name); var stars = admins.Select(name => name + "*"); ``` Like you said, you don't get paid per line feed - it's ok to have multiple lines and you can keep the nice syntax. It's ok to not like complicated code - no one likes complicated code - which is why I don't like for loops - I don't _care_ how something is iterated I just want to map every object.
- NateDad 12y ago
  Well, so, that's actually not doing the right thing. You just changed the names in the list of strings that holds admin names, but not the names on the user objects themselves. Even if the first half were returning the users themselves to the admins list, you're still iterating over the list twice. This is yet another reason why the for loop is good - it makes it more clear what is actually going on. This is your code in for loops: var admins []string for _, u := range users { if u.IsAdmin() { admins = append(admins, u.Name) } } for i := range admins { admins[i] = admins[i] + "*" } The problem with the fancy syntax is that it makes it too easy to write suboptimal code that you'd never write if you were writing plain old for loops, like the above.