4 ms·

I think projects displaying the pattern often have a root problem that the writeup didn't discuss: An initial application with requirements created by non-techn

I think projects displaying the pattern often have a root problem that the writeup didn't discuss: An initial application with requirements created by non-technical folks who had it implemented by whatever the equivalent is of outsourced workers/students/interns/guy they hired without knowing how to hire a developer. The initial developer doesn't understand, know, or follow basic CS knowledge or design trade trade offs and constraints, but will try to create a reasonable system using whatever has press at the time - Rails, Php + Zend, etc. Unfortunately they will follow weird aspects of tutorials they googled for studiously while missing out on the design implication of their choices leading to monstrosities like a product display page that requires hundreds of recursively made SQL queries, an entire application where 90% of the code is in one file, database tables that are composed in the most awful ways, and more. It's this first layer that causes everything that comes after. The root of the problem isn't that trying something new is right or wrong. It's that the original was done so badly nobody can reasonably continue down the same path and everyone who comes after can't figure out how to cope. Of course it's typically exacerbated by deadlines. When you can add a new dependency and meet your deadline or try to do things the inscrutably bad way they were done previously and miss it (let alone have any ability to test it), it's hardly surprising that people choose to add what they know. When confronted with something intractable of course you throw your favorite spaghetti at the wall. And of course there's not any sort of order to what sticks. So I think the lesson is more about the criticalness of doing things remotely right to begin with.

debaserab2 12y ago
Or the requirements massively shift over time. Not mutually exclusive with what you mentioned, but can exacerbate the situation or leave a codebase with even an adept programmer a mess.
mreiland 12y ago
stop blaming non-developers for the problems created by developers. It's the result of developer who don't have the maturity to consider the long term risk of their decisions. That has nothing to do with the business leaders. I freelance which means I see a lot of shit. In any given week I can touch/work with 10+ tech stacks across both Windows and *nix. When you do that on a regular basis it gives you a different perspective on things. In particular, you write/design code in the same vein as the code around it. It doesn't matter what your personal values as a developer are. Consistency is more important than that. I have been known to pull people to the side and ask them "I'm about to do X, how do you normally do X in codebase Y? I want to do it exactly the way you do". If you're going to change the way something like a DAL accesses its data you better be able to either do all of it, or you do none of it for exactly the reasons the article stated. That is purely a technical decision, and the blame for that lies with every single developer who chose to introduce a new tech into the stack for no other reason than sensibilities.
- oldmanjay 12y ago
  Silly question - if we aren't to blame those poor put-upon business folks who hired the incapable, immature developer, then why is the developer accountable?
- yason 12y ago
  Honestly, we can't really blame the developers either. If you inherit a compilation of different techniques, there isn't much you could do to it. There isn't necessarily a code base that you could build on as the same thing is already been done in a number of different ways. So any way you're going to make changes there isn't going to make the code much better and, thus, the least worst choice would be to start gradually injecting some consistency into the codebase even if that's the fifth layer on top of the previous ones. Maybe, just possibly, the second developer could at least theoretically continue exactly from where the first developer left off (a single person or team == a single codebase that's at least somewhat consistent) but it could often make the mess even worse if the first developer really didn't have much of a clue of what he was doing. The managers are basically the customer who ordered the software and they can't be trusted to know anything. It's a very common case that the customer doesn't know what they need and it's the job of the developers to first figure out the customer's real problem before beginning prototyping.
  - chris_wot 12y ago
    Thats fine for the initial stab at the problem. But then someone who does know what he is doing come along and recommends a module be rewritten and management says no, because "it's working". At this point, it's management's fault.
- jghn 12y ago
  I've used this same argument for why "show me your github" isn't necessarily the best way to judge a developer. My group uses github for all of our code and while it's private that might change in the future. If that happens, there are a lot of places where the answer to "why did you do X this way?" is "that's the way they do it" and not that I agreed with it. That's potentially more harm than good if someone was looking at it.
guelo 12y ago
It's not "CS knowledge" that causes tech fashion to change over time. CS knowledge hasn't actually changed very much in the last 30 years. The people that designed SQL knew all about the advantages and tradeoffs of "NoSQL" key-value stores. The impedance mismatch of ORM Object-SQL systems has been studied for decades. If anything changes it's the engineering "best practices", like unit testing methodologies. But mostly it's just fashion, and who has the biggest platforms and marketing budgets for pushing their framework. Software development as a field sucks, no one really knows what they're doing. We're all just figuring it out as we go along. As he says in the article: "We think that the current shiny best way is the end of history; that it will never be superseded or seen to be suspect with hindsight."
- guard-of-terra 12y ago
  "The people that designed SQL knew all about the advantages and tradeoffs of "NoSQL" key-value stores" I think they could not possibly imagine fitting the whole working set into server's RAM. Which changes quite a lot.
  - jghn 12y ago
    That doesn't mean that the knowledge wasn't there, just that there were different tradeoffs being made back then. I disagree w/ the GP that it's purely about fashion, but certainly there distinct pendulums which swing back and forth in our field. IMO that pendulum isn't purely driven by fancy alone (i.e. fashion) but also changes in computing power. For instance the ever driving swing of the local driven/server driven architecture. These days it's called "cloud" but it's gone by other names in the past. At some point the demands of the applications demand too much for the network traffic/network storage/etc and an increasing amount of work is offloaded to the local client. And then, at some point even further in the future there's an advance on the capabilities of remote computing (e.g. network bandwith) and the cycle repeats itself.
nnq 12y ago
Agree! I think the real advice should be: Throw away the first prototype version of a software and rewire v1.0 from scratch that you can use as a solid base for future development (for any application that is expected to have a long life - think years, not months) even if it's awesome for the users and works business-wise. And always write a prototype/MVP first, and plan to throw it away, even if you end up using it in production and it ends up crucial for you business. Yes, you should try and build the MVP in weeks, not months, and you should already have its successor done when you're out of beta... but regardless of this, the pancakes rule will apply: the first one is always awful, and if try to stack the next ones on top of it, the whole pancakes tower will crumble. So throw away the first pancake and start from the second!
Terr_ 12y ago
I think that's just one kind of initial nucleation point for the future crystals of failure. Looking at the 10-year-old PHP codebase we have at work, the real cause is the organizational fluid around the project, a lack of accountability/ownership. Nobody on the business-die can confidently say what ought to be current practice, so the system becomes--via circular logic--its own reference-implementation Since no end-user can independently state "How We Do Things Here" except by saying "Whatever The System Already Does", that channels the software's future evolutionary paths: You cannot make "common sense" fixes, because the bug might be a feature.

debaserab2 12y ago

Or the requirements massively shift over time. Not mutually exclusive with what you mentioned, but can exacerbate the situation or leave a codebase with even an adept programmer a mess.

mreiland 12y ago

stop blaming non-developers for the problems created by developers. It's the result of developer who don't have the maturity to consider the long term risk of their decisions. That has nothing to do with the business leaders. I freelance which means I see a lot of shit. In any given week I can touch/work with 10+ tech stacks across both Windows and *nix. When you do that on a regular basis it gives you a different perspective on things. In particular, you write/design code in the same vein as the code around it. It doesn't matter what your personal values as a developer are. Consistency is more important than that. I have been known to pull people to the side and ask them "I'm about to do X, how do you normally do X in codebase Y? I want to do it exactly the way you do". If you're going to change the way something like a DAL accesses its data you better be able to either do all of it, or you do none of it for exactly the reasons the article stated. That is purely a technical decision, and the blame for that lies with every single developer who chose to introduce a new tech into the stack for no other reason than sensibilities.

oldmanjay 12y ago
Silly question - if we aren't to blame those poor put-upon business folks who hired the incapable, immature developer, then why is the developer accountable?
yason 12y ago
Honestly, we can't really blame the developers either. If you inherit a compilation of different techniques, there isn't much you could do to it. There isn't necessarily a code base that you could build on as the same thing is already been done in a number of different ways. So any way you're going to make changes there isn't going to make the code much better and, thus, the least worst choice would be to start gradually injecting some consistency into the codebase even if that's the fifth layer on top of the previous ones. Maybe, just possibly, the second developer could at least theoretically continue exactly from where the first developer left off (a single person or team == a single codebase that's at least somewhat consistent) but it could often make the mess even worse if the first developer really didn't have much of a clue of what he was doing. The managers are basically the customer who ordered the software and they can't be trusted to know anything. It's a very common case that the customer doesn't know what they need and it's the job of the developers to first figure out the customer's real problem before beginning prototyping.
- chris_wot 12y ago
  Thats fine for the initial stab at the problem. But then someone who does know what he is doing come along and recommends a module be rewritten and management says no, because "it's working". At this point, it's management's fault.
jghn 12y ago
I've used this same argument for why "show me your github" isn't necessarily the best way to judge a developer. My group uses github for all of our code and while it's private that might change in the future. If that happens, there are a lot of places where the answer to "why did you do X this way?" is "that's the way they do it" and not that I agreed with it. That's potentially more harm than good if someone was looking at it.

guelo 12y ago

It's not "CS knowledge" that causes tech fashion to change over time. CS knowledge hasn't actually changed very much in the last 30 years. The people that designed SQL knew all about the advantages and tradeoffs of "NoSQL" key-value stores. The impedance mismatch of ORM Object-SQL systems has been studied for decades. If anything changes it's the engineering "best practices", like unit testing methodologies. But mostly it's just fashion, and who has the biggest platforms and marketing budgets for pushing their framework. Software development as a field sucks, no one really knows what they're doing. We're all just figuring it out as we go along. As he says in the article: "We think that the current shiny best way is the end of history; that it will never be superseded or seen to be suspect with hindsight."

guard-of-terra 12y ago
"The people that designed SQL knew all about the advantages and tradeoffs of "NoSQL" key-value stores" I think they could not possibly imagine fitting the whole working set into server's RAM. Which changes quite a lot.
- jghn 12y ago
  That doesn't mean that the knowledge wasn't there, just that there were different tradeoffs being made back then. I disagree w/ the GP that it's purely about fashion, but certainly there distinct pendulums which swing back and forth in our field. IMO that pendulum isn't purely driven by fancy alone (i.e. fashion) but also changes in computing power. For instance the ever driving swing of the local driven/server driven architecture. These days it's called "cloud" but it's gone by other names in the past. At some point the demands of the applications demand too much for the network traffic/network storage/etc and an increasing amount of work is offloaded to the local client. And then, at some point even further in the future there's an advance on the capabilities of remote computing (e.g. network bandwith) and the cycle repeats itself.

nnq 12y ago

Agree! I think the real advice should be: Throw away the first prototype version of a software and rewire v1.0 from scratch that you can use as a solid base for future development (for any application that is expected to have a long life - think years, not months) even if it's awesome for the users and works business-wise. And always write a prototype/MVP first, and plan to throw it away, even if you end up using it in production and it ends up crucial for you business. Yes, you should try and build the MVP in weeks, not months, and you should already have its successor done when you're out of beta... but regardless of this, the pancakes rule will apply: the first one is always awful, and if try to stack the next ones on top of it, the whole pancakes tower will crumble. So throw away the first pancake and start from the second!

Terr_ 12y ago

I think that's just one kind of initial nucleation point for the future crystals of failure. Looking at the 10-year-old PHP codebase we have at work, the real cause is the organizational fluid around the project, a lack of accountability/ownership. Nobody on the business-die can confidently say what ought to be current practice, so the system becomes--via circular logic--its own reference-implementation Since no end-user can independently state "How We Do Things Here" except by saying "Whatever The System Already Does", that channels the software's future evolutionary paths: You cannot make "common sense" fixes, because the bug might be a feature.