5 ms·

There's a lot more that can be said about how things change as the scale of a software system grows. Every order of magnitude increase requires a new level of

There's a lot more that can be said about how things change as the scale of a software system grows. Every order of magnitude increase requires a new level of discipline. At 10^3 lines, you can do whatever you want -- all your function and variable names can be one or two letters, you don't need comments (or indeed any documentation), your functions don't need well-defined contracts because they only need to work in a few cases, etc. etc. At 10^4 lines, if you're smart you can still get away with making a mess, but it starts to become helpful to name things carefully, to add a few comments, to clear away dead code, to fuss a little over style and readability. At 10^5 lines, those things are not just helpful but necessary, and new things start to matter. It helps to think about your module boundaries and contracts more carefully. You need to minimize the preconditions of your contracts as much as practical -- meaning, make your functions handle all the corner cases you can -- because you can no longer mentally track all the restrictions and special cases. By 10^6 lines, architecture has become more important than coding. Clean interfaces are essential. Minimizing coupling is a major concern. It's easier to work on 10 10^5-line programs than one 10^6-line program, so the goal is to make the system behave, as much as possible, like a weakly interacting collection of subsystems. There's probably a book that explains all this much better than I can here, but perhaps this conveys the general idea.

Waterluvian 10y ago
The way you describe relevance at each order of magnitude is eye opening to me. It's exactly what we have been doing, but it was never explained so eloquently why we succeed because of that. Did you get this from somewhere or is it your wording?
- ScottBurson 10y ago
  This is some combination of something I read once somewhere (Fred Brooks?? possibly) with my own experience over the years. (I didn't go past 10^6 lines because I've never worked on anything larger.)
  - Bombthecat 10y ago
    Anecdote time, once when I was younger I was a programmer. I should work on a java program. ( was a piece of shit and mess) refactoring will never happen ( costs money ) so i switched programming to baring a sysadmin. Valuable lesson I learned: code will be unlikely rewritten or recaptured. It costs money. But at least you get the chance to upgrade hardware and software once in a while, because it wants out of support or breaks down. I net this java program is still a mess and runs somewhere. Thanks to this code I realized programming isn't something I want to do 24/7 :)
wtbob 10y ago
Very good point — that's why good abstractions are such a win. With functions, I don't need to think about all the code needed to read the body of an HTTP request; I just read it. That's also why I think that macros are so valuable: a single macro can abstract away an extremely complex and tricky piece of code into a single line, or two. With the right abstractions, we can turn 10^6 projects back into 10^3 projects.
- ludamad 10y ago
  This alludes to another issue: you can pack a lot into one line, but that doesn't necessarily mean it'll be a tractable line of code lest something goes wrong
  - wtbob 10y ago
    True enough, and it applies to functions as well as macros. A function named getThingList which also sets the user's password and send a rude email to the Premier of Elbonia is just as bad as an unhygienic macro!
- deleted 10y ago
  [deleted]
jmorrison 10y ago
I was going to reply with a link to Manny Lehman's "FEAST" publication page, which (tragically - and I am not being cavalier) has seemed to disappear following his passing. I highly recommend chasing links starting from his Wikipedia page: https://en.wikipedia.org/wiki/Manny_Lehman_(computer_scientist) https://en.wikipedia.org/wiki/Manny_Lehman_(computer_scienti... It is challenging to summarize in a brief reply, and I would not do justice in any event - characterizing the relationships between the evolutionary pressures on software, the limitations of the people and organizations who produce it to handle the attendant complexity of any such system, and the feedback loops which drive the process, etc., is not a 1 paragraph post. See http://users.ece.utexas.edu/~perry/work/papers/feast1.pdf http://users.ece.utexas.edu/~perry/work/papers/feast1.pdf for instance My crotchety old geezer, "get off my lawn" take is that minimizing LOC count is vastly unappreciated (as FEAST underscores how increasing LOC count decreases the ability to evolve/change/modify software), and that DSLs are therefore far more promising than are "index cards" and "stories" and "methodologies" to decrease LOC count and thus increase agility
- ScottBurson 10y ago
  Anytime someone mentions LOC count, I have to reply with this great line from Dijkstra: [I]f we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger.
huherto 10y ago
> There's probably a book that explains all this much better than I can here, but perhaps this conveys the general idea. Why should I read a book when you have done a great job in one paragraph. Seriously, everyone who has built large systems should understand it immediately. This is a big part of the software engineering craft.

Waterluvian 10y ago

The way you describe relevance at each order of magnitude is eye opening to me. It's exactly what we have been doing, but it was never explained so eloquently why we succeed because of that. Did you get this from somewhere or is it your wording?

ScottBurson 10y ago
This is some combination of something I read once somewhere (Fred Brooks?? possibly) with my own experience over the years. (I didn't go past 10^6 lines because I've never worked on anything larger.)
- Bombthecat 10y ago
  Anecdote time, once when I was younger I was a programmer. I should work on a java program. ( was a piece of shit and mess) refactoring will never happen ( costs money ) so i switched programming to baring a sysadmin. Valuable lesson I learned: code will be unlikely rewritten or recaptured. It costs money. But at least you get the chance to upgrade hardware and software once in a while, because it wants out of support or breaks down. I net this java program is still a mess and runs somewhere. Thanks to this code I realized programming isn't something I want to do 24/7 :)

wtbob 10y ago

Very good point — that's why good abstractions are such a win. With functions, I don't need to think about all the code needed to read the body of an HTTP request; I just read it. That's also why I think that macros are so valuable: a single macro can abstract away an extremely complex and tricky piece of code into a single line, or two. With the right abstractions, we can turn 10^6 projects back into 10^3 projects.

ludamad 10y ago
This alludes to another issue: you can pack a lot into one line, but that doesn't necessarily mean it'll be a tractable line of code lest something goes wrong
- wtbob 10y ago
  True enough, and it applies to functions as well as macros. A function named getThingList which also sets the user's password and send a rude email to the Premier of Elbonia is just as bad as an unhygienic macro!
deleted 10y ago
[deleted]

jmorrison 10y ago

I was going to reply with a link to Manny Lehman's "FEAST" publication page, which (tragically - and I am not being cavalier) has seemed to disappear following his passing. I highly recommend chasing links starting from his Wikipedia page: https://en.wikipedia.org/wiki/Manny_Lehman_(computer_scientist) https://en.wikipedia.org/wiki/Manny_Lehman_(computer_scienti... It is challenging to summarize in a brief reply, and I would not do justice in any event - characterizing the relationships between the evolutionary pressures on software, the limitations of the people and organizations who produce it to handle the attendant complexity of any such system, and the feedback loops which drive the process, etc., is not a 1 paragraph post. See http://users.ece.utexas.edu/~perry/work/papers/feast1.pdf http://users.ece.utexas.edu/~perry/work/papers/feast1.pdf for instance My crotchety old geezer, "get off my lawn" take is that minimizing LOC count is vastly unappreciated (as FEAST underscores how increasing LOC count decreases the ability to evolve/change/modify software), and that DSLs are therefore far more promising than are "index cards" and "stories" and "methodologies" to decrease LOC count and thus increase agility

ScottBurson 10y ago
Anytime someone mentions LOC count, I have to reply with this great line from Dijkstra: [I]f we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger.

huherto 10y ago

> There's probably a book that explains all this much better than I can here, but perhaps this conveys the general idea. Why should I read a book when you have done a great job in one paragraph. Seriously, everyone who has built large systems should understand it immediately. This is a big part of the software engineering craft.