3 ms·

> Modularity, indirection and abstraction are not panaceas for this problem. In most cases they reduce local complexity at the cost of global complexity. This i

by yoklov 11y ago

> Modularity, indirection and abstraction are not panaceas for this problem. In most cases they reduce local complexity at the cost of global complexity. This is a decision that should be consciously weighed in each case rather than assumed to be an unquestionable win. This is a very important point, and one that a lot of developers miss, or even believe the opposite of. It's common to approach problems by 1. identifying the abstractions that you'd want to have to solve the problem, 2. implement those abstractions, and 3. use the abstractions to solve the problem. This is how (by and large) we're taught to program, and it basically attacks the problem in a backwards way. Start specific, implement the whole thing with fairly minimal abstraction, then if the local complexity warrants abstraction, move to something more abstract. Why? So that you can put off "the point past which you can no longer collectively understand the system". Local complexity is better than global complexity, and defaulting to a globally complex system is a terrible choice.

TeMPOraL 11y ago
> 1. identifying the abstractions that you'd want to have to solve the problem, 2. implement those abstractions, and 3. use the abstractions to solve the problem. Also the reason to do it bottom-up is because you never understand the problem perfectly, especially if it has anything to do with real-world things. This means that whatever global domain model you created is wrong, and by the time you discover where exactly, you'll be so invested that it's too late to change it. It's easier to adjust the model when going bottom-up - you can change or discard just the top layers, as soon as you realize your new abstraction level doesn't match the problem domain properly.
- lumpypua 11y ago
  I watched the Structure and Interpretation of Computer Programs lecture recordings, and one lesson I got out of them very strongly: create a toolkit to solve problems in the space of your initial conception of the problem. Your initial instinct of what the problem is, is typically wrong, and the demands on your program typically evolve anyway. I hesitate to use the phrase "domain specific language" because it brings baggage, but that's the platonic ideal of a toolkit. The core skill of building toolkits is picking flexible ways to decompose problems. Top-down planning and decomposition has a way of biting you in the ass at the worst times, when you're almost done except for one corner case. Bottom-up construction tends to be more flexible if dirtier.
  - TeMPOraL 11y ago
    I actually liked the way they referred to those toolkits as "languages". Maybe let's not call them Domain Specific Languages because of the baggage, but it's a good way of thinking to see layers of abstraction as defining programming languages for layers above them. Familiarity with Lisp helps understand this concept, because creating a new "DSL" there is so easy that you treat it just like creating a new class or interface, and not as some kind of great endeavour that most people mean when thinking about DSLs today.
jamii 11y ago
I like Casey Muratori's idea of removing the word 'abstraction' from your vocabulary in favor of the word 'compression'. You start by writing code absolutely directly and then if the same patterns keep cropping up, you compress them into a named pattern. It then becomes clearly a bad idea to try to 'compress' something that you only use once in an attempt to make it simpler. http://mollyrocket.com/casey/stream_0019.html http://mollyrocket.com/casey/stream_0019.html
- eru 11y ago
  As a sort-of counterexample, it can still be useful to put code that you only use once in its own function: functions have very definite entry and exit points, and a defined interface for communicating with the rest of the code (via arguments, captured variables and return values).
  - yoklov 11y ago
    There are several tradeoffs here. Yes, this has a readability benefit, and helps ease developers into the module, especially if they're not familiar with it to start with. A good argument in favor of not breaking them into separate functions is available here[0], basically boiling down to the fact that splitting something into a separate function has the undesirable side effect of hiding the full scope (not in the programming sense) of what it does. If this function is pure, this isn't a problem, but if it manipulates -- or even just accesses -- global state, you might call it when it's in a state that it doesn't expect, leading to subtle bugs. You also might move the call to it to a new location, not realizing the full scope of what it does (this is doubly true for removing calls). [0] https://web.archive.org/web/20150926045718/http://number-none.com/blow/john_carmack_on_inlined_code.html https://web.archive.org/web/20150926045718/http://number-non... (sad to see this site slip away ...)
    - eru 11y ago
      I agree, all bets are off when manipulating global state. (Unless your type system tracks such manipulations in detail.)

by yoklov 11y ago

TeMPOraL 11y ago

> 1. identifying the abstractions that you'd want to have to solve the problem, 2. implement those abstractions, and 3. use the abstractions to solve the problem. Also the reason to do it bottom-up is because you never understand the problem perfectly, especially if it has anything to do with real-world things. This means that whatever global domain model you created is wrong, and by the time you discover where exactly, you'll be so invested that it's too late to change it. It's easier to adjust the model when going bottom-up - you can change or discard just the top layers, as soon as you realize your new abstraction level doesn't match the problem domain properly.

lumpypua 11y ago
I watched the Structure and Interpretation of Computer Programs lecture recordings, and one lesson I got out of them very strongly: create a toolkit to solve problems in the space of your initial conception of the problem. Your initial instinct of what the problem is, is typically wrong, and the demands on your program typically evolve anyway. I hesitate to use the phrase "domain specific language" because it brings baggage, but that's the platonic ideal of a toolkit. The core skill of building toolkits is picking flexible ways to decompose problems. Top-down planning and decomposition has a way of biting you in the ass at the worst times, when you're almost done except for one corner case. Bottom-up construction tends to be more flexible if dirtier.
- TeMPOraL 11y ago
  I actually liked the way they referred to those toolkits as "languages". Maybe let's not call them Domain Specific Languages because of the baggage, but it's a good way of thinking to see layers of abstraction as defining programming languages for layers above them. Familiarity with Lisp helps understand this concept, because creating a new "DSL" there is so easy that you treat it just like creating a new class or interface, and not as some kind of great endeavour that most people mean when thinking about DSLs today.

jamii 11y ago

I like Casey Muratori's idea of removing the word 'abstraction' from your vocabulary in favor of the word 'compression'. You start by writing code absolutely directly and then if the same patterns keep cropping up, you compress them into a named pattern. It then becomes clearly a bad idea to try to 'compress' something that you only use once in an attempt to make it simpler. http://mollyrocket.com/casey/stream_0019.html http://mollyrocket.com/casey/stream_0019.html

eru 11y ago
As a sort-of counterexample, it can still be useful to put code that you only use once in its own function: functions have very definite entry and exit points, and a defined interface for communicating with the rest of the code (via arguments, captured variables and return values).
- yoklov 11y ago
  There are several tradeoffs here. Yes, this has a readability benefit, and helps ease developers into the module, especially if they're not familiar with it to start with. A good argument in favor of not breaking them into separate functions is available here[0], basically boiling down to the fact that splitting something into a separate function has the undesirable side effect of hiding the full scope (not in the programming sense) of what it does. If this function is pure, this isn't a problem, but if it manipulates -- or even just accesses -- global state, you might call it when it's in a state that it doesn't expect, leading to subtle bugs. You also might move the call to it to a new location, not realizing the full scope of what it does (this is doubly true for removing calls). [0] https://web.archive.org/web/20150926045718/http://number-none.com/blow/john_carmack_on_inlined_code.html https://web.archive.org/web/20150926045718/http://number-non... (sad to see this site slip away ...)
  - eru 11y ago
    I agree, all bets are off when manipulating global state. (Unless your type system tracks such manipulations in detail.)