9 ms·

My perspective, as someone who came relatively later to programming as a career, is that the skill of "thinking like a programmer" isn't too hard to acquire onc

My perspective, as someone who came relatively later to programming as a career, is that the skill of "thinking like a programmer" isn't too hard to acquire once you've taken a few college CS classes and started to simply program a lot. Rather, what I'm constantly amazed at (and maybe its because I'm working in stereotypical enterprisey environments?) is how complex, convoluted, long and hard to follow a lot of code is. So much code goes into what to a non programmer or novice would seem like a fairly simple task. And so much code is really horribly written and designed and implemented. Its very much true what your professors say about how you'll spend more time reading code than writing it. Anyways, I'm sure my perspective as both a junior dev and a late career switcher is somewhat biased, but there you have it.

wtetzner 10y ago
Well, that's the thing. Simple code is hard to write. It's easy to end up with a convoluted mess, because often the first version is just the programmer implementing their thought process as they think through the problem. It's rare that people get the chance to go back and fix up the code once they have a better understanding of the problem.
twa927 10y ago
Why do you hold the assumption that code can be made much simpler than it is? I've done many years of programming and wrote some bigger programs from scratch by myself. Almost always the actual code is far more complicated than one would assume it should be. And it's not because it's of low quality. It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. If you were in school recently, maybe you were lead to think that programming is an extension of math, where everything is pure, simple and provable. But it isn't - programming is much messier.
- Chris_Newton 10y ago
  It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. I think it helps to separate essential complexity, due to the nature of the problem you’re trying to solve, from accidental complexity, which comes from other sources. In an ideal world, you’d represent the essential complexity as clearly and flexibly as possible, and minimise the amount of accidental complexity you put on top. That accidental complexity can come from lots of different sources: tools that aren’t a perfect fit for what you’re trying to do, a design that isn’t as clean as it could be, an unfortunate choice of data structures of algorithms… I suspect it probably is fair to say that a lot of accidental complexity that goes into real world programs could have been avoided if, for example, more appropriate tools had been used or better design decisions had been made during development.
  - twa927 10y ago
    I claim that the essential complexity of real world software is inherently high. Yes, the accidental complexity can balloon to infinity (FizzBuzzEnterpriseEdition), but even the simplest possible software solving a given real world problem will be always highly complex. This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. No particular step of a program is complex, but it combines a lot of these simple steps.
    - Chris_Newton 10y ago
      I claim that the essential complexity of real world software is inherently high. Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. See the CompCert C compiler[1] for an example of using formal proofs in real world programming at a much larger scale. No particular step of a program is complex, but it combines a lot of these simple steps. And this leads to the “secret” of being able to construct proofs for realistic programs in useful amounts of time, in my experience: you have to be able to decompose the fundamental problem into manageable parts, prove some properties of interest for those parts, and then be able to compose the things you’ve proved separately in order to prove useful things about the overall system. We do this all the time with some properties in strong, static type systems, for example. [1] http://compcert.inria.fr/compcert-C.html http://compcert.inria.fr/compcert-C.html
      - twa927 10y ago
        > Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? Even the simplest "real world" programs do things like: communication with a database, string manipulation, calling library functions which can fail. > As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. I wrote a master's thesis that touched the subject and it's what I put in an introduction as an illustration of the field's difficulty (I believe it was not taken from air ;). I don't know the field now but ca. 10 years ago program's proofs were mainly refinements of high level axioms to executable code so there was no separate "proof" step of a finished program, but still the added complexity of proving the refinement steps was huge.
- gambler 10y ago
  I'm not OP, but I can answer the question as if it was addressed to me. I believe that a lot of "real life" code can be simplified because I've had way too many cases where I open some program and remove 60 to 80 percent of its code without affecting the overall logic. (This usually includes obvious duplication, needless abstraction layers and things that reimplement functionality of standard libraries.) The difficult part usually isn't the restructuring itself, but understanding what the program is supposed to be doing in the first place. Fortunately, deleting code is one of the best ways to learn about what it's doing. You need some good tooling to do this safely, though.
  - cesarbs 10y ago
    > needless abstraction layers In my experience this is one of the most offending things in large code bases. I've worked on code that does what it's supposed to do, but you can't see that it's doing it because the solution to the initial problem statement is completely diluted in a mess of factories, abstract classes, gigantic class hierarchies, delegates, and so many other abused patterns. In some cases the extreme level of abstraction can be justified by the need to have a system that's extensible in many different places. But more often that's not it, it's really just patterns being overly used and abused. You can rewrite the code to be much clearer with less than half the original size and have no negative impact from it.
    - Sacho 10y ago
      And after you've "fixed" this "problem" for the current state of the app, you happily leave, and the next person is tasked with adding new features, and suddenly they find the app incredibly rigid and impossible to modify, so they add some abstractions like factories, delegates, etc... I mean, anecdotes. I find it funny that in a profession allegedly heavily influenced by science, we keep trading anecdotes(in my experience, "more often than not", etc) instead of having any hard data, tests and experiments to compare.
      - cesarbs 10y ago
        I wasn't advocating writing extremely rigid, non-extensible code. I was alluding to unnecessarily abstract code bases where e.g. there's a class hierarchy with 7 classes when in fact 3 would properly describe the problem domain. Some people get really carried away coming up with abstractions and in the end they just write a lot of meaningless or purposeless code.
ArkyBeagle 10y ago
It could be that the reality reflected by the code base is simply complicated. It could be that technical debt isn't measured properly or at all. It could be a "that's your problem, buddy". That's more likely.
jsolson 10y ago
There are two common ways in which I've seen this happen. In one, the code started simple and evolved in place to accommodate bugs or additional use cases. Some refactoring has happened in the past, and that added some complexity to the overall project, but still not all of the abstractions were right. Further refactoring may have been attempted, but proved too involved given the value of the task at hand and the complexity the code had already acquired. The code is now more complex further raising the bar for refactoring when the next change comes along. In the other, substantial effort went into designing the code to be flexible. It was recognized that not all use cases could be understood at the outside, and that refactoring things mid project often fails due to time constraints. A design was arrived at best on the most complete understanding of the requirements at the time, and that naturally design had a certain amount of intrinsic complexity to it as the requirements were complex. As development progressed, it was discovered that the design was not ideal for the actual requirements of the project. Plumbing was added and layers were tacked on to work around the issue. What I conclude from this that you're damned if you do and you're damned if you don't. Practically, I generally prefer working with code that was arrived at by the first approach. It often has a few good abstractions in along with a bunch of methods or classes that just combine too much stuff in one place. Perhaps some things are plumbed into places they shouldn't be. This can, with time and patience, be peeled apart into something a bit more manageable. By contrast, my experience with the second strategy is that it leads to a much less manageable mess where most time is spent finding the code that actual does anything. There is much more plumbing to rip out and seeming innocuous changes have far-reaching consequences.
wlievens 10y ago
The mismatch between the high-level notion of programming (solving interesting problems) and the daily routine is there because in practice only a relatively small part if code deals with domain knowledge, and the rest deals with mundane technicalities (persistence, validation, security, logging, diagnostics, user interface technicalities, command routing, data translation ...). The latter part certainly has its share of interesting challenges, but most of it is boring cruft.
- twa927 10y ago
  Mathematical proofs of correctness of even simple programs WITHOUT the mentioned technicalities are very complex (multiple times longer than the actual programs). This suggests that there is a lot of inherent complexity in the core software.
wrp 10y ago
I have a similar story. After taking a not-unusual academic path from Math through Linguistics to Cognitive Science, I eventually got started in programming to support my research. I had previously avoided programming largely out of fear. When I thought of the effort it took me to produce a half page of correct mathematics, I blanched at the thought of trying to write thousands of lines of working code. When I finally delved into Perl, Python, C, etc. and reading open source code (mostly C, mostly greenfield projects), I was flabbergasted to discover how much coding is by guess and by golly. I am genuinely unable to grasp how a person with the intelligence to master the technical details that programming requires could be so lacking in ability to elegantly conceptualize a problem domain and produce suitably organized code. I have come to wonder if it isn't a psychological inhibition. Like some people keep their desk obsessively neat but others work amid a filthy messs, not because they're too lazy to clean but because they're more comfortable that way. Once in a while, I see a programmer demonstrating the clarity of analysis I'm accustomed to seeing in science (and even the humanities). This is the kind of programmer who will produce a shorter and clearer solution in C than the average programmer will produce in Python. It's rare, though, so I'm constantly amused/irritated by the frequently stated assumption on HN and elsewhere that programmers have superior analytical skills.
- pointernil 10y ago
  For any non-trivial (code size, complexity etc) programming-problem there are programmers with little analytical skills but given enough time they will end up with code doing mostly the right things... unfortunately from the out side there is no way to tell to what degree the code evolved from try and error or from true analysis... Then there are skillful and analytical programmers who over the course of the dev cycle for a problem burn out their "analytical mana" due to simple exhaustion, external pressure or both or simply because the problem grows conceptually over their 'head-room' during the work on it... backtracking and reworking already written down code is still cumbersome, always a risk and unpleasant ('trashing something already done') Then there often is time eroding the most clean, analytical code with every little update into a mass, unless ofc energy is spent to re-analyse and rewrite. Then there are ofc some differences between areas in which programming is used. 'how much coding is by guess and by golly' I hear this quite often from ppl with math, physics background etc. who are often accustomed to read expositions of ideas in math papers: presenting the final versions of the formulas. Code quite often is more equivalent to the notes a mathematician would make and use before summarizing them into the published paper. Those Math-Whitepapers are executed in the heads of the humans so they tend to try to keep it clean and minimal and analytical sound. Those Code-Artefacts are executed by machines which don't care at all about such things, thus as long as the humans are satisfied with the end-results the 'inner-structure' tends to erode. Code is read more often than written but it is often only read later and by other people, so ... first things first: it has to work, right? ;) But yea, I mostly agree that many programmers slide down into a habit of very special kind of "reactive programming" ;) where they try to hit the target by incrementally zeroing down on a solution reacting only on the results of the previous version of the code. Due to the action - reaction structure this tends to be quite flow-inducing and satisfactory... but the results are seldom analytically pleasing or sound.

wtetzner 10y ago

Well, that's the thing. Simple code is hard to write. It's easy to end up with a convoluted mess, because often the first version is just the programmer implementing their thought process as they think through the problem. It's rare that people get the chance to go back and fix up the code once they have a better understanding of the problem.

twa927 10y ago

Why do you hold the assumption that code can be made much simpler than it is? I've done many years of programming and wrote some bigger programs from scratch by myself. Almost always the actual code is far more complicated than one would assume it should be. And it's not because it's of low quality. It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. If you were in school recently, maybe you were lead to think that programming is an extension of math, where everything is pure, simple and provable. But it isn't - programming is much messier.

Chris_Newton 10y ago
It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. I think it helps to separate essential complexity, due to the nature of the problem you’re trying to solve, from accidental complexity, which comes from other sources. In an ideal world, you’d represent the essential complexity as clearly and flexibly as possible, and minimise the amount of accidental complexity you put on top. That accidental complexity can come from lots of different sources: tools that aren’t a perfect fit for what you’re trying to do, a design that isn’t as clean as it could be, an unfortunate choice of data structures of algorithms… I suspect it probably is fair to say that a lot of accidental complexity that goes into real world programs could have been avoided if, for example, more appropriate tools had been used or better design decisions had been made during development.
- twa927 10y ago
  I claim that the essential complexity of real world software is inherently high. Yes, the accidental complexity can balloon to infinity (FizzBuzzEnterpriseEdition), but even the simplest possible software solving a given real world problem will be always highly complex. This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. No particular step of a program is complex, but it combines a lot of these simple steps.
  - Chris_Newton 10y ago
    I claim that the essential complexity of real world software is inherently high. Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. See the CompCert C compiler[1] for an example of using formal proofs in real world programming at a much larger scale. No particular step of a program is complex, but it combines a lot of these simple steps. And this leads to the “secret” of being able to construct proofs for realistic programs in useful amounts of time, in my experience: you have to be able to decompose the fundamental problem into manageable parts, prove some properties of interest for those parts, and then be able to compose the things you’ve proved separately in order to prove useful things about the overall system. We do this all the time with some properties in strong, static type systems, for example. [1] http://compcert.inria.fr/compcert-C.html http://compcert.inria.fr/compcert-C.html
    - twa927 10y ago
      > Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? Even the simplest "real world" programs do things like: communication with a database, string manipulation, calling library functions which can fail. > As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. I wrote a master's thesis that touched the subject and it's what I put in an introduction as an illustration of the field's difficulty (I believe it was not taken from air ;). I don't know the field now but ca. 10 years ago program's proofs were mainly refinements of high level axioms to executable code so there was no separate "proof" step of a finished program, but still the added complexity of proving the refinement steps was huge.
gambler 10y ago
I'm not OP, but I can answer the question as if it was addressed to me. I believe that a lot of "real life" code can be simplified because I've had way too many cases where I open some program and remove 60 to 80 percent of its code without affecting the overall logic. (This usually includes obvious duplication, needless abstraction layers and things that reimplement functionality of standard libraries.) The difficult part usually isn't the restructuring itself, but understanding what the program is supposed to be doing in the first place. Fortunately, deleting code is one of the best ways to learn about what it's doing. You need some good tooling to do this safely, though.
- cesarbs 10y ago
  > needless abstraction layers In my experience this is one of the most offending things in large code bases. I've worked on code that does what it's supposed to do, but you can't see that it's doing it because the solution to the initial problem statement is completely diluted in a mess of factories, abstract classes, gigantic class hierarchies, delegates, and so many other abused patterns. In some cases the extreme level of abstraction can be justified by the need to have a system that's extensible in many different places. But more often that's not it, it's really just patterns being overly used and abused. You can rewrite the code to be much clearer with less than half the original size and have no negative impact from it.
  - Sacho 10y ago
    And after you've "fixed" this "problem" for the current state of the app, you happily leave, and the next person is tasked with adding new features, and suddenly they find the app incredibly rigid and impossible to modify, so they add some abstractions like factories, delegates, etc... I mean, anecdotes. I find it funny that in a profession allegedly heavily influenced by science, we keep trading anecdotes(in my experience, "more often than not", etc) instead of having any hard data, tests and experiments to compare.
    - cesarbs 10y ago
      I wasn't advocating writing extremely rigid, non-extensible code. I was alluding to unnecessarily abstract code bases where e.g. there's a class hierarchy with 7 classes when in fact 3 would properly describe the problem domain. Some people get really carried away coming up with abstractions and in the end they just write a lot of meaningless or purposeless code.

ArkyBeagle 10y ago

It could be that the reality reflected by the code base is simply complicated. It could be that technical debt isn't measured properly or at all. It could be a "that's your problem, buddy". That's more likely.

jsolson 10y ago

There are two common ways in which I've seen this happen. In one, the code started simple and evolved in place to accommodate bugs or additional use cases. Some refactoring has happened in the past, and that added some complexity to the overall project, but still not all of the abstractions were right. Further refactoring may have been attempted, but proved too involved given the value of the task at hand and the complexity the code had already acquired. The code is now more complex further raising the bar for refactoring when the next change comes along. In the other, substantial effort went into designing the code to be flexible. It was recognized that not all use cases could be understood at the outside, and that refactoring things mid project often fails due to time constraints. A design was arrived at best on the most complete understanding of the requirements at the time, and that naturally design had a certain amount of intrinsic complexity to it as the requirements were complex. As development progressed, it was discovered that the design was not ideal for the actual requirements of the project. Plumbing was added and layers were tacked on to work around the issue. What I conclude from this that you're damned if you do and you're damned if you don't. Practically, I generally prefer working with code that was arrived at by the first approach. It often has a few good abstractions in along with a bunch of methods or classes that just combine too much stuff in one place. Perhaps some things are plumbed into places they shouldn't be. This can, with time and patience, be peeled apart into something a bit more manageable. By contrast, my experience with the second strategy is that it leads to a much less manageable mess where most time is spent finding the code that actual does anything. There is much more plumbing to rip out and seeming innocuous changes have far-reaching consequences.

wlievens 10y ago

The mismatch between the high-level notion of programming (solving interesting problems) and the daily routine is there because in practice only a relatively small part if code deals with domain knowledge, and the rest deals with mundane technicalities (persistence, validation, security, logging, diagnostics, user interface technicalities, command routing, data translation ...). The latter part certainly has its share of interesting challenges, but most of it is boring cruft.

twa927 10y ago
Mathematical proofs of correctness of even simple programs WITHOUT the mentioned technicalities are very complex (multiple times longer than the actual programs). This suggests that there is a lot of inherent complexity in the core software.

wrp 10y ago

I have a similar story. After taking a not-unusual academic path from Math through Linguistics to Cognitive Science, I eventually got started in programming to support my research. I had previously avoided programming largely out of fear. When I thought of the effort it took me to produce a half page of correct mathematics, I blanched at the thought of trying to write thousands of lines of working code. When I finally delved into Perl, Python, C, etc. and reading open source code (mostly C, mostly greenfield projects), I was flabbergasted to discover how much coding is by guess and by golly. I am genuinely unable to grasp how a person with the intelligence to master the technical details that programming requires could be so lacking in ability to elegantly conceptualize a problem domain and produce suitably organized code. I have come to wonder if it isn't a psychological inhibition. Like some people keep their desk obsessively neat but others work amid a filthy messs, not because they're too lazy to clean but because they're more comfortable that way. Once in a while, I see a programmer demonstrating the clarity of analysis I'm accustomed to seeing in science (and even the humanities). This is the kind of programmer who will produce a shorter and clearer solution in C than the average programmer will produce in Python. It's rare, though, so I'm constantly amused/irritated by the frequently stated assumption on HN and elsewhere that programmers have superior analytical skills.

pointernil 10y ago
For any non-trivial (code size, complexity etc) programming-problem there are programmers with little analytical skills but given enough time they will end up with code doing mostly the right things... unfortunately from the out side there is no way to tell to what degree the code evolved from try and error or from true analysis... Then there are skillful and analytical programmers who over the course of the dev cycle for a problem burn out their "analytical mana" due to simple exhaustion, external pressure or both or simply because the problem grows conceptually over their 'head-room' during the work on it... backtracking and reworking already written down code is still cumbersome, always a risk and unpleasant ('trashing something already done') Then there often is time eroding the most clean, analytical code with every little update into a mass, unless ofc energy is spent to re-analyse and rewrite. Then there are ofc some differences between areas in which programming is used. 'how much coding is by guess and by golly' I hear this quite often from ppl with math, physics background etc. who are often accustomed to read expositions of ideas in math papers: presenting the final versions of the formulas. Code quite often is more equivalent to the notes a mathematician would make and use before summarizing them into the published paper. Those Math-Whitepapers are executed in the heads of the humans so they tend to try to keep it clean and minimal and analytical sound. Those Code-Artefacts are executed by machines which don't care at all about such things, thus as long as the humans are satisfied with the end-results the 'inner-structure' tends to erode. Code is read more often than written but it is often only read later and by other people, so ... first things first: it has to work, right? ;) But yea, I mostly agree that many programmers slide down into a habit of very special kind of "reactive programming" ;) where they try to hit the target by incrementally zeroing down on a solution reacting only on the results of the previous version of the code. Due to the action - reaction structure this tends to be quite flow-inducing and satisfactory... but the results are seldom analytically pleasing or sound.