4 ms·

Why do you hold the assumption that code can be made much simpler than it is? I've done many years of programming and wrote some bigger programs from scratch b

by twa927 10y ago

Why do you hold the assumption that code can be made much simpler than it is? I've done many years of programming and wrote some bigger programs from scratch by myself. Almost always the actual code is far more complicated than one would assume it should be. And it's not because it's of low quality. It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. If you were in school recently, maybe you were lead to think that programming is an extension of math, where everything is pure, simple and provable. But it isn't - programming is much messier.

Chris_Newton 10y ago
It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. I think it helps to separate essential complexity, due to the nature of the problem you’re trying to solve, from accidental complexity, which comes from other sources. In an ideal world, you’d represent the essential complexity as clearly and flexibly as possible, and minimise the amount of accidental complexity you put on top. That accidental complexity can come from lots of different sources: tools that aren’t a perfect fit for what you’re trying to do, a design that isn’t as clean as it could be, an unfortunate choice of data structures of algorithms… I suspect it probably is fair to say that a lot of accidental complexity that goes into real world programs could have been avoided if, for example, more appropriate tools had been used or better design decisions had been made during development.
- twa927 10y ago
  I claim that the essential complexity of real world software is inherently high. Yes, the accidental complexity can balloon to infinity (FizzBuzzEnterpriseEdition), but even the simplest possible software solving a given real world problem will be always highly complex. This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. No particular step of a program is complex, but it combines a lot of these simple steps.
  - Chris_Newton 10y ago
    I claim that the essential complexity of real world software is inherently high. Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. See the CompCert C compiler[1] for an example of using formal proofs in real world programming at a much larger scale. No particular step of a program is complex, but it combines a lot of these simple steps. And this leads to the “secret” of being able to construct proofs for realistic programs in useful amounts of time, in my experience: you have to be able to decompose the fundamental problem into manageable parts, prove some properties of interest for those parts, and then be able to compose the things you’ve proved separately in order to prove useful things about the overall system. We do this all the time with some properties in strong, static type systems, for example. [1] http://compcert.inria.fr/compcert-C.html http://compcert.inria.fr/compcert-C.html
    - twa927 10y ago
      > Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? Even the simplest "real world" programs do things like: communication with a database, string manipulation, calling library functions which can fail. > As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. I wrote a master's thesis that touched the subject and it's what I put in an introduction as an illustration of the field's difficulty (I believe it was not taken from air ;). I don't know the field now but ca. 10 years ago program's proofs were mainly refinements of high level axioms to executable code so there was no separate "proof" step of a finished program, but still the added complexity of proving the refinement steps was huge.
      - Chris_Newton 10y ago
        Even the simplest "real world" programs do things like: communication with a database, string manipulation, calling library functions which can fail. Sure, but these things are exactly what I’m talking about with accidental complexity. You’re talking here about databases and strings and library functions, not about whatever real world problem you’re ultimately trying to solve. Complexity that comes from the tools you use or how you use them is exactly what I’m arguing we should ideally minimise when programming, but we often don’t for various reasons.
- Avshalom 10y ago
  The problem is that when people get told to cut a board to 1 meter and we get handed a .5 meter board we say "fuck you" the program on the other hand needs to be explicitly told how/when to say fuck you, and it needs that for basically every single possible source of error. That doesn't fit into either of your categories really but it's where so much of the actual complexity comes from.
  - raould42 10y ago
    I for one applaud your use of the vulgar and am doing my best to upvote you mightily.
gambler 10y ago
I'm not OP, but I can answer the question as if it was addressed to me. I believe that a lot of "real life" code can be simplified because I've had way too many cases where I open some program and remove 60 to 80 percent of its code without affecting the overall logic. (This usually includes obvious duplication, needless abstraction layers and things that reimplement functionality of standard libraries.) The difficult part usually isn't the restructuring itself, but understanding what the program is supposed to be doing in the first place. Fortunately, deleting code is one of the best ways to learn about what it's doing. You need some good tooling to do this safely, though.
- cesarbs 10y ago
  > needless abstraction layers In my experience this is one of the most offending things in large code bases. I've worked on code that does what it's supposed to do, but you can't see that it's doing it because the solution to the initial problem statement is completely diluted in a mess of factories, abstract classes, gigantic class hierarchies, delegates, and so many other abused patterns. In some cases the extreme level of abstraction can be justified by the need to have a system that's extensible in many different places. But more often that's not it, it's really just patterns being overly used and abused. You can rewrite the code to be much clearer with less than half the original size and have no negative impact from it.
  - Sacho 10y ago
    And after you've "fixed" this "problem" for the current state of the app, you happily leave, and the next person is tasked with adding new features, and suddenly they find the app incredibly rigid and impossible to modify, so they add some abstractions like factories, delegates, etc... I mean, anecdotes. I find it funny that in a profession allegedly heavily influenced by science, we keep trading anecdotes(in my experience, "more often than not", etc) instead of having any hard data, tests and experiments to compare.
    - cesarbs 10y ago
      I wasn't advocating writing extremely rigid, non-extensible code. I was alluding to unnecessarily abstract code bases where e.g. there's a class hierarchy with 7 classes when in fact 3 would properly describe the problem domain. Some people get really carried away coming up with abstractions and in the end they just write a lot of meaningless or purposeless code.

by twa927 10y ago

Chris_Newton 10y ago

It happens that writing something nontrivial that works in the real world requires all the details, checks and abstractions it contains. I think it helps to separate essential complexity, due to the nature of the problem you’re trying to solve, from accidental complexity, which comes from other sources. In an ideal world, you’d represent the essential complexity as clearly and flexibly as possible, and minimise the amount of accidental complexity you put on top. That accidental complexity can come from lots of different sources: tools that aren’t a perfect fit for what you’re trying to do, a design that isn’t as clean as it could be, an unfortunate choice of data structures of algorithms… I suspect it probably is fair to say that a lot of accidental complexity that goes into real world programs could have been avoided if, for example, more appropriate tools had been used or better design decisions had been made during development.

twa927 10y ago
I claim that the essential complexity of real world software is inherently high. Yes, the accidental complexity can balloon to infinity (FizzBuzzEnterpriseEdition), but even the simplest possible software solving a given real world problem will be always highly complex. This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. No particular step of a program is complex, but it combines a lot of these simple steps.
- Chris_Newton 10y ago
  I claim that the essential complexity of real world software is inherently high. Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? This is why mathematical proofs of software correctness are impossible in practice. A small 100-200 LOC program takes a qualified mathematician weeks to create a proof of correctness. As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. See the CompCert C compiler[1] for an example of using formal proofs in real world programming at a much larger scale. No particular step of a program is complex, but it combines a lot of these simple steps. And this leads to the “secret” of being able to construct proofs for realistic programs in useful amounts of time, in my experience: you have to be able to decompose the fundamental problem into manageable parts, prove some properties of interest for those parts, and then be able to compose the things you’ve proved separately in order to prove useful things about the overall system. We do this all the time with some properties in strong, static type systems, for example. [1] http://compcert.inria.fr/compcert-C.html http://compcert.inria.fr/compcert-C.html
  - twa927 10y ago
    > Sometimes it certainly is high, but why inherently? Surely it depends on the particular problem you are interested in solving? Even the simplest "real world" programs do things like: communication with a database, string manipulation, calling library functions which can fail. > As someone who actually does formally prove things about algorithms significantly larger than that from to time, I believe you’re exaggerating here. I wrote a master's thesis that touched the subject and it's what I put in an introduction as an illustration of the field's difficulty (I believe it was not taken from air ;). I don't know the field now but ca. 10 years ago program's proofs were mainly refinements of high level axioms to executable code so there was no separate "proof" step of a finished program, but still the added complexity of proving the refinement steps was huge.
    - Chris_Newton 10y ago
      Even the simplest "real world" programs do things like: communication with a database, string manipulation, calling library functions which can fail. Sure, but these things are exactly what I’m talking about with accidental complexity. You’re talking here about databases and strings and library functions, not about whatever real world problem you’re ultimately trying to solve. Complexity that comes from the tools you use or how you use them is exactly what I’m arguing we should ideally minimise when programming, but we often don’t for various reasons.
Avshalom 10y ago
The problem is that when people get told to cut a board to 1 meter and we get handed a .5 meter board we say "fuck you" the program on the other hand needs to be explicitly told how/when to say fuck you, and it needs that for basically every single possible source of error. That doesn't fit into either of your categories really but it's where so much of the actual complexity comes from.
- raould42 10y ago
  I for one applaud your use of the vulgar and am doing my best to upvote you mightily.

gambler 10y ago

I'm not OP, but I can answer the question as if it was addressed to me. I believe that a lot of "real life" code can be simplified because I've had way too many cases where I open some program and remove 60 to 80 percent of its code without affecting the overall logic. (This usually includes obvious duplication, needless abstraction layers and things that reimplement functionality of standard libraries.) The difficult part usually isn't the restructuring itself, but understanding what the program is supposed to be doing in the first place. Fortunately, deleting code is one of the best ways to learn about what it's doing. You need some good tooling to do this safely, though.

cesarbs 10y ago
> needless abstraction layers In my experience this is one of the most offending things in large code bases. I've worked on code that does what it's supposed to do, but you can't see that it's doing it because the solution to the initial problem statement is completely diluted in a mess of factories, abstract classes, gigantic class hierarchies, delegates, and so many other abused patterns. In some cases the extreme level of abstraction can be justified by the need to have a system that's extensible in many different places. But more often that's not it, it's really just patterns being overly used and abused. You can rewrite the code to be much clearer with less than half the original size and have no negative impact from it.
- Sacho 10y ago
  And after you've "fixed" this "problem" for the current state of the app, you happily leave, and the next person is tasked with adding new features, and suddenly they find the app incredibly rigid and impossible to modify, so they add some abstractions like factories, delegates, etc... I mean, anecdotes. I find it funny that in a profession allegedly heavily influenced by science, we keep trading anecdotes(in my experience, "more often than not", etc) instead of having any hard data, tests and experiments to compare.
  - cesarbs 10y ago
    I wasn't advocating writing extremely rigid, non-extensible code. I was alluding to unnecessarily abstract code bases where e.g. there's a class hierarchy with 7 classes when in fact 3 would properly describe the problem domain. Some people get really carried away coming up with abstractions and in the end they just write a lot of meaningless or purposeless code.