4 ms·

You have literally just described the set of objects asked for: the unique authors of the books with more than 1,000 pages. I don't understand how you expect to

You have literally just described the set of objects asked for: the unique authors of the books with more than 1,000 pages. I don't understand how you expect to get any simpler than that. The functional style isn't even requiring you to describe how to accomplish it, it almost verbatim simply describes the answer you're trying to get. If your entire objection is that you might want intermediate-named variables… you can just do that? var longBooks = books.filter(book => book.pageCount > 1000) var authors = longBooks.map(book => book.author) var distinctAuthors = authors.distinct() For short chains (95%+ of cases), this is far more mental overhead. For the remaining cases, you can just name the parts? I'm just completely failing to see your problem here.

jltsiren 2y ago
The problem is that it's easy to overdo it. When you are writing the code, you already know what it's supposed to do, and adding a few more things to the chain is convenient and attractive. But when you are reading unfamiliar code, you often wish that the author was more explicit with their code. Not just with what the code is actually doing, but what it's trying to do and what are the key waypoints to get there. With procedural code, it's widely accepted that you should not do too many things in a single statement. But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code.
- stouset 2y ago
  > The problem is that it's easy to overdo it. Welcome to all features of every programming language? Sacrificing readability, optimization, and simplicity for the 95% case because some un-principled developers might overdo it in the 5% case (when the cost of fixing it is trivially just inserting variable assignments) is… not a good trade-off.
  - jltsiren 2y ago
    5% is common enough that you'll encounter it almost every time you read code. And fixing it is not easy, because you first need to understand the code before you can add useful variable names. Besides, programming language evolution is mostly driven by the fact that everyone is lazy and unprincipled at least occasionally. If you need to be disciplined to avoid footguns, you'll trigger them sooner or later.
    - stouset 2y ago
      The cost of this "footgun" is basically zero. Every step in a functional pipeline is isolated and wholly independent. If you want to split such a pipeline in two, doing so is trivial.
    - whstl 2y ago
      5% is also low enough that you can just use another technique for the exceptions.
    - vendiddy 2y ago
      It's not clear what point you are trying to make because so far you are describing problems common to all programming languages.
- whstl 2y ago
  > But in functional code, the entire chain is a single statement Not necessarily. You can use intermediate variables when necessary.
- syklemil 2y ago
  > But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code. How are we deciding what's "functional code", here? Because functional languages also provide means like `let` and `where` bindings to break up statements. The example might in pseudo-Haskell be broken up like distinctAuthors = distinct authors where authors = map (\book -> book.author) longBooks longBooks = filter (\book -> book.pageCount > 1000) books IMO the code here is also simple enough that I don't see it needing much in the way of comments, but it is also possible and common to intersperse comments in the dot style, e.g. distinctAuthors = books // TODO: Where does this collection come from anyway? // books are officially considered long if they're over 1000 pages, c.f. the Council of Chalcedon (451) .filter(book => book.pageCount > 1000) // All books have exactly one author for some reason. Why? Shouldn't this be a flatmap or something? .map(book => book.author) // We obviously actually want a set[author] here, rather than a pruned list[author], // but in this imaginary DinkyLang we'd have to implement that as map[author, null] // and that's just too annoying to deal with .distinct()
- bonoboTP 2y ago
  If you're used to it, then it doesn't read like a single statement, even though technically it is. You put each call of the chain on its own line and it feels like reading the lines of regular imperative code. Except better because I can be sure that each line strictly only uses the result of the previous line, not two or three lines before so the logic flows nicely linearly.
__mharrison__ 2y ago
It's also harder to write and debug with the intermediate steps.
- nomel 2y ago
  How so? The states of the intermediate steps are logically and easily exposed in a debugger. You can also easily set conditional breakpoints relative to the intermediate states. I know that intermediate states are generally easier to comprehend, because I never have to explain them in code reviews. To avoid having to explain chains to others, I end up having to add descriptive comments to the intermediate steps, far exceeding the number of characters the descriptive intermediate variables would take. That's why I avoid them, or break them up: time spent in code reviews has proven to me that people have trouble with chains.
  - __mharrison__ 2y ago
    Build up and debug the chain as you work in an environment like Jupyter. No need to create variables. Just run the code and verify that the current step works. Then, proceed to the next. Then, put the chain in a function. If you want to be nice, put a .loc as the first step to explicitly list all of the input columns. Drop another .loc as the last step to validate the output columns. (This also serves as a test and documentation to future you about what needs to come in and out.) Create a simple unit test with a sample of the data if you desire. I've found that the constraint of thinking in chains forces me to think of the recipe that I need for my data. Of course, not everything can be done in a stepwise manner (.pipe helps with that), but often, this constraint forces you to think about what you are doing. Every good Pandas user I know uses it this way. I've taught hundreds more. Generally, it feels weird at first (kind of like whitespace in Python), but after a day, you get used to it. Do you store intermediate results of SQL?
    - nomel 2y ago
      > Build up and debug the chain as you work in an environment like Jupyter. > Just run the code and verify that the current step works. Then, proceed to the next. Yes, it's not debuggable/"viewable" without cut/paste/commenting out lines, once it's constructed.
galaxyLogic 2y ago
Your example shows that it is possible to give NAMES to the intermediate results of a long chain. Giving names to things makes it easier to understand the intention of the programmer. And that also allows you to create a TREE of dataflow-code not just a CHAIN. For instance 'longBooks' could be used as the starting point of multiple different chains. It gets complicated at some point but I think other approaches result in code that is even harder to understand.

jltsiren 2y ago

The problem is that it's easy to overdo it. When you are writing the code, you already know what it's supposed to do, and adding a few more things to the chain is convenient and attractive. But when you are reading unfamiliar code, you often wish that the author was more explicit with their code. Not just with what the code is actually doing, but what it's trying to do and what are the key waypoints to get there. With procedural code, it's widely accepted that you should not do too many things in a single statement. But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code.

stouset 2y ago
> The problem is that it's easy to overdo it. Welcome to all features of every programming language? Sacrificing readability, optimization, and simplicity for the 95% case because some un-principled developers might overdo it in the 5% case (when the cost of fixing it is trivially just inserting variable assignments) is… not a good trade-off.
- jltsiren 2y ago
  5% is common enough that you'll encounter it almost every time you read code. And fixing it is not easy, because you first need to understand the code before you can add useful variable names. Besides, programming language evolution is mostly driven by the fact that everyone is lazy and unprincipled at least occasionally. If you need to be disciplined to avoid footguns, you'll trigger them sooner or later.
  - stouset 2y ago
    The cost of this "footgun" is basically zero. Every step in a functional pipeline is isolated and wholly independent. If you want to split such a pipeline in two, doing so is trivial.
  - whstl 2y ago
    5% is also low enough that you can just use another technique for the exceptions.
  - vendiddy 2y ago
    It's not clear what point you are trying to make because so far you are describing problems common to all programming languages.
whstl 2y ago
> But in functional code, the entire chain is a single statement Not necessarily. You can use intermediate variables when necessary.
syklemil 2y ago
> But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code. How are we deciding what's "functional code", here? Because functional languages also provide means like `let` and `where` bindings to break up statements. The example might in pseudo-Haskell be broken up like distinctAuthors = distinct authors where authors = map (\book -> book.author) longBooks longBooks = filter (\book -> book.pageCount > 1000) books IMO the code here is also simple enough that I don't see it needing much in the way of comments, but it is also possible and common to intersperse comments in the dot style, e.g. distinctAuthors = books // TODO: Where does this collection come from anyway? // books are officially considered long if they're over 1000 pages, c.f. the Council of Chalcedon (451) .filter(book => book.pageCount > 1000) // All books have exactly one author for some reason. Why? Shouldn't this be a flatmap or something? .map(book => book.author) // We obviously actually want a set[author] here, rather than a pruned list[author], // but in this imaginary DinkyLang we'd have to implement that as map[author, null] // and that's just too annoying to deal with .distinct()
bonoboTP 2y ago
If you're used to it, then it doesn't read like a single statement, even though technically it is. You put each call of the chain on its own line and it feels like reading the lines of regular imperative code. Except better because I can be sure that each line strictly only uses the result of the previous line, not two or three lines before so the logic flows nicely linearly.

__mharrison__ 2y ago

It's also harder to write and debug with the intermediate steps.

nomel 2y ago
How so? The states of the intermediate steps are logically and easily exposed in a debugger. You can also easily set conditional breakpoints relative to the intermediate states. I know that intermediate states are generally easier to comprehend, because I never have to explain them in code reviews. To avoid having to explain chains to others, I end up having to add descriptive comments to the intermediate steps, far exceeding the number of characters the descriptive intermediate variables would take. That's why I avoid them, or break them up: time spent in code reviews has proven to me that people have trouble with chains.
- __mharrison__ 2y ago
  Build up and debug the chain as you work in an environment like Jupyter. No need to create variables. Just run the code and verify that the current step works. Then, proceed to the next. Then, put the chain in a function. If you want to be nice, put a .loc as the first step to explicitly list all of the input columns. Drop another .loc as the last step to validate the output columns. (This also serves as a test and documentation to future you about what needs to come in and out.) Create a simple unit test with a sample of the data if you desire. I've found that the constraint of thinking in chains forces me to think of the recipe that I need for my data. Of course, not everything can be done in a stepwise manner (.pipe helps with that), but often, this constraint forces you to think about what you are doing. Every good Pandas user I know uses it this way. I've taught hundreds more. Generally, it feels weird at first (kind of like whitespace in Python), but after a day, you get used to it. Do you store intermediate results of SQL?
  - nomel 2y ago
    > Build up and debug the chain as you work in an environment like Jupyter. > Just run the code and verify that the current step works. Then, proceed to the next. Yes, it's not debuggable/"viewable" without cut/paste/commenting out lines, once it's constructed.

galaxyLogic 2y ago

Your example shows that it is possible to give NAMES to the intermediate results of a long chain. Giving names to things makes it easier to understand the intention of the programmer. And that also allows you to create a TREE of dataflow-code not just a CHAIN. For instance 'longBooks' could be used as the starting point of multiple different chains. It gets complicated at some point but I think other approaches result in code that is even harder to understand.