11 ms·

> probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files I'm only halfway through John Ousterhout's

> probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files I'm only halfway through John Ousterhout's book Philosophy of Software Design but I think it agrees with you on this -- that smallness-of-file or smallness-of-function is not a target to shoot for because it prevents the things you build from being deep. That you should strive to build modules which have deep functionality and small interfaces and should contain their complexity within them so the users don't have to know that complexity.

Klathmon 8y ago
Now i'm squarely in frontend web-app development right now which definitely changes things (mainly the complexity is centered around enabling fast changes/additions to the codebase, and not the actual business logic for the most part), but while "deep functionality and small interfaces" sounds good on paper, most of the time giant files with a few functions exported aren't a good way to manage that. Sure, it solves the problem when viewed from the outside. "users" of the software (users being other devs in this case) get a nice small interface and docs that explain how to use it, but internally it's much harder to work with. Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it. In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. And it means that any devs that truly do understand it top-to-bottom would just waste a lot of time switching around files if it were split up. Putting it all in the same file here nudges you to really make sure you understand it top to bottom before making any big changes. It's intimidating and scary for a reason, because at its core it is a complex piece of code, and dressing it up in "simple code's clothing" won't help.
- TeMPOraL 8y ago
  > Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it. There's a balance to be struck here; you want to minimize the size of the code a developer has to understand to work on (or with) a given abstraction, but you don't want to split beyond that point, as it only makes the developer jump around files. In my experience (with Java in particular), current development trends involve splitting the code too much.
  - usrusr 8y ago
    Java (like many other languages of that generation) suffers from a lack of idiomatic 1:1 visibility. Whenever you split something up in Java it litters a namespace that is much bigger than necessary. Even private is too big when the class is full of tiny methods most of which most will never be meaningful to any of their peers except for that one call site. Sure, you can create inner function objects and with 8+ it's not even completely weird anymore, but that's still a far cry from the nested functions goodness of Pascal style "structured programming".
    - TeMPOraL 8y ago
      Agreed. I don't remember much from my brief exposure to Pascal years ago, but spending a good chunk of my programming years in Common Lisp has spoiled me; the simple ability to nest functions is something I sorely missed when working on Java codebases.
    - deleted 8y ago
      [deleted]
    - mrgriffin 8y ago
      One of my favorite features of Haskell is that I can use where to write the auxiliary function definitions after the code that use them, but keep them scoped inside of a single function. A relatively bad example here: https://wiki.haskell.org/Worker_wrapper#Hiding_the_worker https://wiki.haskell.org/Worker_wrapper#Hiding_the_worker
      - yawaramin 8y ago
        Bindings in the `where` clause can still access names in the main binding: https://wiki.haskell.org/Let_vs._Where#Advantages_of_where https://wiki.haskell.org/Let_vs._Where#Advantages_of_where This runs the risk of strongly coupling them to the main binding and making them less reusable. IMHO strong coupling where two bindings are aware of each others' internals should be avoided.
        8y ago
- taneq 8y ago
  > In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. I think you're on the money here. This was done intentionally, to ensure that devs understand the whole thing before making changes, because the functionality is so critical and so easy to mess up. Also, breaking single logical things up into smaller files (rather than into a composition of smaller logical things) just leads to unneccessary file hopping and wrecks locality of reference for the dev.
- tobr 8y ago
  Breaking things down into smaller pieces is good when it’s good... but there are drawbacks that I feel are often ignored. For example, as you say, a small piece of stand alone code implies that someone can do meaningful work on it without understanding the entire context around it. It also implies that it is suitable for reuse. But if it’s both reused liberally and encourages you to keep working on it, it will almost certainly mean that your changes will have unanticipated consequences. So you still can’t get away with being unaware of the context. When you break things apart you also fossilize that way of approaching the problem, which often makes it more difficult to see orthogonal approaches and refactor towards them later. Instead you keep working within the structure that’s already there, which often leads to concerns being spread out across different modules. Too often people decide on the structure before they even understand the problem. I think it has similar problems as religiously following the DRY principle. There are so many situations where your code will be much, much worse if you insist on always sticking to DRY.
  - andyidsinga 8y ago
    this. > Too often people decide on the structure before they even understand the problem.
  - TimTheTinker 8y ago
    > When you break things apart you also fossilize that way of approaching the problem, which often makes it more difficult to see orthogonal approaches and refactor towards them later. This is particularly true on teams where a significant proportion of the developers is reticent to refactor code as they go. It seems that given a developer with a sub-80th (or so)-percentile propensity to refactor, the more broken up the solution is (into smaller functions, methods, classes, modules, etc.), the less likely that developer will be to refactor the solution when an obviously better approach exists.
TeMPOraL 8y ago
I just finished his book yesterday; he has a lot to say about size and comments. For size, your summary is spot-on. I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs. As for comments, this file is essentially Ousterhout taken to the extreme. Still, I think he would have like it, given how critical this file is. In the book, he encourages writing more comments than the current trends would suggest, pointing out that you can't fully express abstractions in code, so all the things the code doesn't contain - the high-level concepts, the rationale, the caveats - should be documented in comments in appropriate places. Overall, I'm extremely impressed by the book, and its focus on reducing and mitigating complexity.
- sitkack 8y ago
  There is a review and discussion of that book here, https://news.ycombinator.com/item?id=18331219 https://news.ycombinator.com/item?id=18331219
  - TeMPOraL 8y ago
    That is the exact thread that prompted me to buy the book.
- thaumaturgy 8y ago
  > I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs. Another way of describing this, that I ran across recently, is that this increases the cognitive load for developers working on the code, and cognitive load is one metric by which code can be measured as "good" or "bad". Things like excessive scrolling and switching between files increases cognitive load. So heavily-commented code can decrease cognitive load if the programmer can read the comments and the related code block at the same time and if the comments help to explain why the code exists the way it does. Or, heavily-commented code can increase cognitive load if the comments don't accurately describe the code, or if they're so verbose that the programmer has to scroll up and down to digest both the comments and the code together.
partycoder 8y ago
> that smallness-of-file or smallness-of-function is not a target to shoot for I disagree. Unless you are methodical and know what you are doing (like NASA or the authors of Kubernetes), it is hard to create large functions and files by keeping levels of detail consistent and not repeating code. How do you test a function that has 100 unique outcomes? How do you safely maintain it to ensure it won't break? How do you even know it's working?
- sdenton4 8y ago
  Write 100 tests?
- AstralStorm 8y ago
  Write automated proofs? Probably best but tooling is junk - it's rarely supported. Constraint or property based tests as opposed to value tests? (That depends on how unique.) Usually the only thing that does unique things is a direct mapping, everything else is either an equation or has other preconditions, postconditions and internal or external properties. It also makes the logic more directly visible and refactoring easier presuming tests are reasonably written. Ensure coverage with a tool to be sure.
iamrohitbanga 8y ago
Haven't read the book. Goes to my Wish List. I completely agree about encapsulation of complexity inside the module. Similar to UX design principles. Just like google or other successful sites present a very simple interface (in case of google the home page is quite clean) but behind it lies very complex code. IMO, encapsulating complexity and handling all edge cases also lends to total functional programming [1] even without using pure functional languages. [1] https://softwareengineering.stackexchange.com/a/334876/38285 https://softwareengineering.stackexchange.com/a/334876/38285
loneranger_11x 8y ago
> That you should strive to build modules which have deep functionality and small interfaces... this. interfaces are not the most important thing in a software but definitely one of the most. you can live with a crappy implementation but your interfaces must cater to the usecase and should only hcange when core premises change.

Klathmon 8y ago

Now i'm squarely in frontend web-app development right now which definitely changes things (mainly the complexity is centered around enabling fast changes/additions to the codebase, and not the actual business logic for the most part), but while "deep functionality and small interfaces" sounds good on paper, most of the time giant files with a few functions exported aren't a good way to manage that. Sure, it solves the problem when viewed from the outside. "users" of the software (users being other devs in this case) get a nice small interface and docs that explain how to use it, but internally it's much harder to work with. Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it. In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. And it means that any devs that truly do understand it top-to-bottom would just waste a lot of time switching around files if it were split up. Putting it all in the same file here nudges you to really make sure you understand it top to bottom before making any big changes. It's intimidating and scary for a reason, because at its core it is a complex piece of code, and dressing it up in "simple code's clothing" won't help.

TeMPOraL 8y ago
> Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it. There's a balance to be struck here; you want to minimize the size of the code a developer has to understand to work on (or with) a given abstraction, but you don't want to split beyond that point, as it only makes the developer jump around files. In my experience (with Java in particular), current development trends involve splitting the code too much.
- usrusr 8y ago
  Java (like many other languages of that generation) suffers from a lack of idiomatic 1:1 visibility. Whenever you split something up in Java it litters a namespace that is much bigger than necessary. Even private is too big when the class is full of tiny methods most of which most will never be meaningful to any of their peers except for that one call site. Sure, you can create inner function objects and with 8+ it's not even completely weird anymore, but that's still a far cry from the nested functions goodness of Pascal style "structured programming".
  - TeMPOraL 8y ago
    Agreed. I don't remember much from my brief exposure to Pascal years ago, but spending a good chunk of my programming years in Common Lisp has spoiled me; the simple ability to nest functions is something I sorely missed when working on Java codebases.
  - deleted 8y ago
    [deleted]
  - mrgriffin 8y ago
    One of my favorite features of Haskell is that I can use where to write the auxiliary function definitions after the code that use them, but keep them scoped inside of a single function. A relatively bad example here: https://wiki.haskell.org/Worker_wrapper#Hiding_the_worker https://wiki.haskell.org/Worker_wrapper#Hiding_the_worker
    - yawaramin 8y ago
      Bindings in the `where` clause can still access names in the main binding: https://wiki.haskell.org/Let_vs._Where#Advantages_of_where https://wiki.haskell.org/Let_vs._Where#Advantages_of_where This runs the risk of strongly coupling them to the main binding and making them less reusable. IMHO strong coupling where two bindings are aware of each others' internals should be avoided.
      - 8y ago
taneq 8y ago
> In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. I think you're on the money here. This was done intentionally, to ensure that devs understand the whole thing before making changes, because the functionality is so critical and so easy to mess up. Also, breaking single logical things up into smaller files (rather than into a composition of smaller logical things) just leads to unneccessary file hopping and wrecks locality of reference for the dev.
tobr 8y ago
Breaking things down into smaller pieces is good when it’s good... but there are drawbacks that I feel are often ignored. For example, as you say, a small piece of stand alone code implies that someone can do meaningful work on it without understanding the entire context around it. It also implies that it is suitable for reuse. But if it’s both reused liberally and encourages you to keep working on it, it will almost certainly mean that your changes will have unanticipated consequences. So you still can’t get away with being unaware of the context. When you break things apart you also fossilize that way of approaching the problem, which often makes it more difficult to see orthogonal approaches and refactor towards them later. Instead you keep working within the structure that’s already there, which often leads to concerns being spread out across different modules. Too often people decide on the structure before they even understand the problem. I think it has similar problems as religiously following the DRY principle. There are so many situations where your code will be much, much worse if you insist on always sticking to DRY.
- andyidsinga 8y ago
  this. > Too often people decide on the structure before they even understand the problem.
- TimTheTinker 8y ago
  > When you break things apart you also fossilize that way of approaching the problem, which often makes it more difficult to see orthogonal approaches and refactor towards them later. This is particularly true on teams where a significant proportion of the developers is reticent to refactor code as they go. It seems that given a developer with a sub-80th (or so)-percentile propensity to refactor, the more broken up the solution is (into smaller functions, methods, classes, modules, etc.), the less likely that developer will be to refactor the solution when an obviously better approach exists.

TeMPOraL 8y ago

I just finished his book yesterday; he has a lot to say about size and comments. For size, your summary is spot-on. I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs. As for comments, this file is essentially Ousterhout taken to the extreme. Still, I think he would have like it, given how critical this file is. In the book, he encourages writing more comments than the current trends would suggest, pointing out that you can't fully express abstractions in code, so all the things the code doesn't contain - the high-level concepts, the rationale, the caveats - should be documented in comments in appropriate places. Overall, I'm extremely impressed by the book, and its focus on reducing and mitigating complexity.

sitkack 8y ago
There is a review and discussion of that book here, https://news.ycombinator.com/item?id=18331219 https://news.ycombinator.com/item?id=18331219
- TeMPOraL 8y ago
  That is the exact thread that prompted me to buy the book.
thaumaturgy 8y ago
> I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs. Another way of describing this, that I ran across recently, is that this increases the cognitive load for developers working on the code, and cognitive load is one metric by which code can be measured as "good" or "bad". Things like excessive scrolling and switching between files increases cognitive load. So heavily-commented code can decrease cognitive load if the programmer can read the comments and the related code block at the same time and if the comments help to explain why the code exists the way it does. Or, heavily-commented code can increase cognitive load if the comments don't accurately describe the code, or if they're so verbose that the programmer has to scroll up and down to digest both the comments and the code together.

partycoder 8y ago

> that smallness-of-file or smallness-of-function is not a target to shoot for I disagree. Unless you are methodical and know what you are doing (like NASA or the authors of Kubernetes), it is hard to create large functions and files by keeping levels of detail consistent and not repeating code. How do you test a function that has 100 unique outcomes? How do you safely maintain it to ensure it won't break? How do you even know it's working?

sdenton4 8y ago
Write 100 tests?
AstralStorm 8y ago
Write automated proofs? Probably best but tooling is junk - it's rarely supported. Constraint or property based tests as opposed to value tests? (That depends on how unique.) Usually the only thing that does unique things is a direct mapping, everything else is either an equation or has other preconditions, postconditions and internal or external properties. It also makes the logic more directly visible and refactoring easier presuming tests are reasonably written. Ensure coverage with a tool to be sure.

iamrohitbanga 8y ago

Haven't read the book. Goes to my Wish List. I completely agree about encapsulation of complexity inside the module. Similar to UX design principles. Just like google or other successful sites present a very simple interface (in case of google the home page is quite clean) but behind it lies very complex code. IMO, encapsulating complexity and handling all edge cases also lends to total functional programming [1] even without using pure functional languages. [1] https://softwareengineering.stackexchange.com/a/334876/38285 https://softwareengineering.stackexchange.com/a/334876/38285

loneranger_11x 8y ago

> That you should strive to build modules which have deep functionality and small interfaces... this. interfaces are not the most important thing in a software but definitely one of the most. you can live with a crappy implementation but your interfaces must cater to the usecase and should only hcange when core premises change.