7 ms·

Warning: controversy. The text misses the point in that having to hold a program in one's mind is not required to begin with. The real problem is complexity.

Warning: controversy. The text misses the point in that having to hold a program in one's mind is not required to begin with. The real problem is complexity. There's just so much going on that it's difficult to remember. And complexity is solved through abstraction - separating the problem out into manageable components. That way the programmer works at a system level, at which components are composed into a solution, or at the component level, where one specific aspect is dealt with. That minimizes the amount of stuff the programmer needs to hold in his head. If you find yourself oscillating between higher and lower levels of abstraction you should probably be revisiting your architecture. It's called separation of concerns. No tool improvement is going to solve that. Neither will a rethink of ways of working.

batgaijin 14y ago
What about Haskell? The type system tells you exactly what's going on.
- Spearchucker 14y ago
  Some tools are better than others. I'm not familiar with Haskell but can tell you that in the .NET world LINQ is bad, because it forces the melding of process components with data access logic.
  - batgaijin 14y ago
    Haskell has it's own warts of course, but the functional straightjacket you're forced into makes it very easy to understand the scope of a computation.
chubot 14y ago
That's begging the question. To come up with the abstractions in the first place requires understanding the whole problem. There's no getting around this. I can't tell you how many programs I've seen where the wrong abstraction is never perturbed, and as a result all feature development pays a huge cost in reliability and functionality. To create or change an abstraction, you have to get your mind around all sides of it. That requires A LOT of context. You should always be "revisiting your architecture". It's basically never true that the first architecture works, unless you've written an extremely similar program before. The worst programs result when people are afraid to shift boundaries. That happens a lot in big company development that PG talks about -- each team just accumulates crud on their side of the abstraction boundary, but they never talk to each other so they can globally simplify. It's true that abstraction reduces complexity, and thus is the only way you can build big programs, but it takes a lot of hard work to get there.
- Spearchucker 14y ago
  The "whole problem" is simply a process. Processes in the real world are composed of 1-* sub-processes, and ultimately distill down into transformations (input -> transformation -> output). You don't need to understand every low-level process to make a start. You raise two interesting points though. Yes, revisiting an architecture is required when you've not solved a similar problem before. Consider though that in this community 99.9% of the time others have solved your problem before. You could reinvent the wheel (which is arrogant), or you could do some research to see which architecture has been successful for others. Sure, that means doing some work before writing code which seems decidedly unpopular these days. It's a free world. You also say that it's a lot of hard work. Yes. Nothing worth doing is easy. [Edit] There's a really good book (there always is) that describes complexity and problem solving better than any other I've found. http://www.amazon.com/dp/0787967653 http://www.amazon.com/dp/0787967653
  - ambrop7 14y ago
    > Consider though that in this community 99.9% of the time others have solved your problem before. You could reinvent the wheel (which is arrogant), or you could do some research to see which architecture has been successful for others. The best way to learn something is to come up with it yourself. If you base your architecture on what you read in books or elsewhere, you probably only have a limited understanding of it. And that's especially bad when you're talking about the architecture of a program. So maybe a better idea in terms of the end result is to first come up with your own solution, implement at least a functioning prototype, and then do some research, to improve upon your idea based on what you have discovered (or throw it away, in the worst case, but of course keep what you've learned doing it).
    - Spearchucker 14y ago
      You're describing semantics of moving from a solution concept to an implementation. How you get there is entirely up to you. What I was suggesting is that a design that separates concerns will obviate the need to keep the entire thing in your head. This is problem solving through the composition of discrete components. AKA managing complexity.
w_t_payne 14y ago
Nice idea, but, sorry, it does not work as well in real life as it does in theory. :-( In any case, everybody is talking as if holding a program in your head is a huge problem. It really isn't. All it means is that particular individual understands the problem (and the solution) deeply and comprehensively. That sounds like a good thing to me. It is true that communicating that level of understanding can be challenging, which is true of most domains. What is special about software development is the degree to which we attempt to communicate that understanding. Well, why do we do that? Do we really need to communicate that understanding when it is sitting happily in that developer's head? Instead of moving the mental model from one developer to another, trying to squeeze that rich, internal representation through the band-limited drinking-straw that is the human sensory suite, can we just move the developer himself, mental model and all? The problem really arises because of all those preconceptions that we have about how team-working and large organizations should function, about how people should be replaceable cogs: jelly-beans that can be discarded at will. Do you like being a cog? Do you like working for organizations that treat people that way? I do not. Instead of trying to dehumanize our profession, can we not re-humanize it? Instead of talking about re-using code, why do we not talk about re-using developers? Recognizing that developing code is as much about developing knowledge and expertise in the head of the developer as it is about herding electrons through tiny pieces of semiconductor. We should value that learning, that knowledge and expertise, and seek to maximise the return from it, rather than hanging on to the (inappropriate, perverse, and just plain wrong) idea that humans are substitutable parts in a machine. Write code, yes. Write documentation, yes, but recognize and acknowledge the unavoidable truth that a significant part of your intellectual capital lives, not on pieces of paper, nor on magnetized platters, but inside somebody's head. Consequently, when you seek to maximize the return on your investment, seek to build on your developer's expertise by redeploying him or her to use the same expertise to solve as many related problems as possible.
- Spearchucker 14y ago
  Interesting. I've been writing code for 26 years. For the last 12 of those I've been an architect where my job is to identify, design and often implement components of a solution, their attributes, their relationships , and attributes of those relationships. Projects I've worked on range from Nokia Maps for Windows Phone to a GB£12M healthcare system. I'd say I have a fairly excellent understanding of what works in theory, and what works in reality. There is no interest or intention to de-humanise anything. In fact I swing pretty hard the other way. Consider a presentation I gave on the human aspects of architecture - http://www.wittenburg.co.uk/Entry.aspx?id=d5002929-97b2-4902-9eb0-71befdf961de http://www.wittenburg.co.uk/Entry.aspx?id=d5002929-97b2-4902....

batgaijin 14y ago

What about Haskell? The type system tells you exactly what's going on.

Spearchucker 14y ago
Some tools are better than others. I'm not familiar with Haskell but can tell you that in the .NET world LINQ is bad, because it forces the melding of process components with data access logic.
- batgaijin 14y ago
  Haskell has it's own warts of course, but the functional straightjacket you're forced into makes it very easy to understand the scope of a computation.

chubot 14y ago

That's begging the question. To come up with the abstractions in the first place requires understanding the whole problem. There's no getting around this. I can't tell you how many programs I've seen where the wrong abstraction is never perturbed, and as a result all feature development pays a huge cost in reliability and functionality. To create or change an abstraction, you have to get your mind around all sides of it. That requires A LOT of context. You should always be "revisiting your architecture". It's basically never true that the first architecture works, unless you've written an extremely similar program before. The worst programs result when people are afraid to shift boundaries. That happens a lot in big company development that PG talks about -- each team just accumulates crud on their side of the abstraction boundary, but they never talk to each other so they can globally simplify. It's true that abstraction reduces complexity, and thus is the only way you can build big programs, but it takes a lot of hard work to get there.

Spearchucker 14y ago
The "whole problem" is simply a process. Processes in the real world are composed of 1-* sub-processes, and ultimately distill down into transformations (input -> transformation -> output). You don't need to understand every low-level process to make a start. You raise two interesting points though. Yes, revisiting an architecture is required when you've not solved a similar problem before. Consider though that in this community 99.9% of the time others have solved your problem before. You could reinvent the wheel (which is arrogant), or you could do some research to see which architecture has been successful for others. Sure, that means doing some work before writing code which seems decidedly unpopular these days. It's a free world. You also say that it's a lot of hard work. Yes. Nothing worth doing is easy. [Edit] There's a really good book (there always is) that describes complexity and problem solving better than any other I've found. http://www.amazon.com/dp/0787967653 http://www.amazon.com/dp/0787967653
- ambrop7 14y ago
  > Consider though that in this community 99.9% of the time others have solved your problem before. You could reinvent the wheel (which is arrogant), or you could do some research to see which architecture has been successful for others. The best way to learn something is to come up with it yourself. If you base your architecture on what you read in books or elsewhere, you probably only have a limited understanding of it. And that's especially bad when you're talking about the architecture of a program. So maybe a better idea in terms of the end result is to first come up with your own solution, implement at least a functioning prototype, and then do some research, to improve upon your idea based on what you have discovered (or throw it away, in the worst case, but of course keep what you've learned doing it).
  - Spearchucker 14y ago
    You're describing semantics of moving from a solution concept to an implementation. How you get there is entirely up to you. What I was suggesting is that a design that separates concerns will obviate the need to keep the entire thing in your head. This is problem solving through the composition of discrete components. AKA managing complexity.

w_t_payne 14y ago

Nice idea, but, sorry, it does not work as well in real life as it does in theory. :-( In any case, everybody is talking as if holding a program in your head is a huge problem. It really isn't. All it means is that particular individual understands the problem (and the solution) deeply and comprehensively. That sounds like a good thing to me. It is true that communicating that level of understanding can be challenging, which is true of most domains. What is special about software development is the degree to which we attempt to communicate that understanding. Well, why do we do that? Do we really need to communicate that understanding when it is sitting happily in that developer's head? Instead of moving the mental model from one developer to another, trying to squeeze that rich, internal representation through the band-limited drinking-straw that is the human sensory suite, can we just move the developer himself, mental model and all? The problem really arises because of all those preconceptions that we have about how team-working and large organizations should function, about how people should be replaceable cogs: jelly-beans that can be discarded at will. Do you like being a cog? Do you like working for organizations that treat people that way? I do not. Instead of trying to dehumanize our profession, can we not re-humanize it? Instead of talking about re-using code, why do we not talk about re-using developers? Recognizing that developing code is as much about developing knowledge and expertise in the head of the developer as it is about herding electrons through tiny pieces of semiconductor. We should value that learning, that knowledge and expertise, and seek to maximise the return from it, rather than hanging on to the (inappropriate, perverse, and just plain wrong) idea that humans are substitutable parts in a machine. Write code, yes. Write documentation, yes, but recognize and acknowledge the unavoidable truth that a significant part of your intellectual capital lives, not on pieces of paper, nor on magnetized platters, but inside somebody's head. Consequently, when you seek to maximize the return on your investment, seek to build on your developer's expertise by redeploying him or her to use the same expertise to solve as many related problems as possible.

Spearchucker 14y ago
Interesting. I've been writing code for 26 years. For the last 12 of those I've been an architect where my job is to identify, design and often implement components of a solution, their attributes, their relationships , and attributes of those relationships. Projects I've worked on range from Nokia Maps for Windows Phone to a GB£12M healthcare system. I'd say I have a fairly excellent understanding of what works in theory, and what works in reality. There is no interest or intention to de-humanise anything. In fact I swing pretty hard the other way. Consider a presentation I gave on the human aspects of architecture - http://www.wittenburg.co.uk/Entry.aspx?id=d5002929-97b2-4902-9eb0-71befdf961de http://www.wittenburg.co.uk/Entry.aspx?id=d5002929-97b2-4902....