9 ms·

Consultants Ate My Unit Tests

128 points by dragthor 11y ago

taylodl 11y ago
I've seen this with my own two eyes. When asked why they removed the unit test they said after they made some rather extensive code changes the tests were failing so they deleted them. Nor was there any trace of irony in their response. When asked didn't they realize that was the whole point of having unit tests they replied that they knew, but they had deadlines to meet and there wasn't enough time. Since controls had been put in place to not ship with failing unit tests but not to check for test coverage they took the easy way out and deleted the tests. I informed management what they had done and thankfully they were relieved of their contract and their consulting company was informed what they had done. I don't know what happened to them after that. Thing is, once the PM was informed what they had done we did a couple of sprints where we restored the unit tests (which we still had in our history) and got them passing. They had no idea the consultants were doing such shoddy work in order to meet deadlines. The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted - and that's for all his projects not just that one particular projects. Now that the story is part of our IT lore the new consultants get the message loud and clear we're not afraid to send them packing. In the end this turned out to be a good story for us - but wow.
- Spooky23 11y ago
  Always assume that consultants will never perform any task that isn't required for milestone sign off.
  - daigoba66 11y ago
    I'm a consultant; this is good advise. It's not laziness or apathy. It's just that we're only paid to do a task.
    - rukuu001 11y ago
      > It's not laziness or apathy. It's just that we're only paid to do a task. That's contractor-talk. If you're a consultant, consult. Educate your client, make sure they understand the consequences of what they're not paying you for.
      - tootie 11y ago
        Sometimes it's a tough sell. I work for a consulting company that I think would rate as very high quality. We try to bake it into contracts and sales pitches that we work a bit slower because we don't cut corners. This isn't always appreciated by a client who's rushing to market. If you think about the old project management triangle, it's scope, budget quality and you can only have 2 out of 3. If you have a contract with fixed scope and budget, it's quality that goes. Getting a client to sign a contract with flexible scope is not easy unless you have a ton of credibility.
        jonesb6 11y ago
        "Rushing to market" Reminds me of the kid going "I want to play, I want to play".. sorry George you suck at basketball, go shoot some free throws. "You won't believe one startup's new growth hack, their teammates hate them!"
    - catmanjan 11y ago
      Good luck explaining the point of unit tests to technology illiterate managers - they don't understand and really they shouldn't have to.
      - myko 11y ago
        > they don't understand and really they shouldn't have to. They shouldn't have to because they'll listen to us and allow us to write and maintain tests as we develop, right?
        rhizome31 11y ago
  - megablast 11y ago
    Well of course. If unit tests aren't in the requirements, and it sounds like they are not, why do them? You have a dealine, people want you out of there asap, you don't start doing extra work. That is going to piss off everyone. Most companies don't value tests, and it sounds like these guys didn't either.
- Nutmog 11y ago
  Regarding the manager's response, are unit tests any use against someone who's trying to circumvent them? If a test passes a couple of values into a function and checks the result, wouldn't the "attacker"/programmer just hardcode those expected results into the function for those input values? Perhaps because that kind of code would look obviously suspicious?
  - Huggernaut 11y ago
    My experience is that if you can cheat the tests in this way then they are either too narrow in scope, or badly written tests. This is something that TDD tries to expose when being adhered to strictly, that is, do the simplest thing to make this test pass - if you `assertTrue(isEmpty())` then I can simply `return true` to make you improve your tests.
    - chris_wot 11y ago
      If a consultant writes assert(isEmpty()) then the first thing I'm going to do is fire up version control, find the consultant and have them fired.
- Natsu 11y ago
  > The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted I just hope that story doesn't continue in the future with something like: @Test public void testFoo(){ /* Original testFoo() code here. */ assertTrue(true); // Unnecessary bloat. }
  - chopin 11y ago
    Been there, seen that.
- benashford 11y ago
  That story had an unexpectedly happy ending. Several times in the past when I've needed to complain about external consultants cutting corners, I've found the consultants have got in first and told my own management about their unique red-tape cutting powers, and how things like continuous integration (yes, the first thing they did was to switch off the CI server so no-one knew what was happening) were a time-consuming luxury.
jasonkester 11y ago
Ah, the sadness of seeing your code again after somebody else has had it for a couple years. I feel the author's pain. I had a company license the source code from one of my old projects to use as the basis for a new site they were building. I built out the bits they needed on a consulting basis and got them launched. Then they started thinking about how expensive I was, and how much they would save if they moved the development overseas. Fair enough. My philosophy on such matters is that it's best to let a client learn this lesson through painful experience. All the best, guys. They made it about a year before the site stopped working completely. Reading this article reminded me of the first time I opened the source that came back. You just wouldn't believe that you could do that much damage in such a short time. I quoted them double my old bill rate and got them back up and running, with all the new functionality they had been wanting, and re-launched. They brought in a new guy to run things, and he started making comments about expenses, and how surely there must be a way to do this cheaper. He even found a local kid who knew some C# and would be willing to help out for a lot less than I was billing. No problem. All the best. I took a quick look at the codebase when they got back in touch a few months later and politely declined to come back on board.
- sriram_sun 11y ago
  Wasn't worth doubling rates again? :)
  - ams6110 11y ago
    Some things just aren't worth any amount of money, and all the moreso if you have other options.
- chrisabrams 11y ago
  It's sad how often this scenario comes up :/ get what you pay for, plus another, larger bill later. The technical debt collector will find you.
  - jxramos 11y ago
    yes he will find you and he holds a mighty ransom!
x3n0ph3n3 11y ago
The format of this article was downright annoying.
- chillaxtian 11y ago
  yes. i don't understand the trend of duplicating quotes from the article inline in a larger font.
  - to3m 11y ago
    It used to be common in magazines (a magazine is like a Moleskine with a bit of blog post on each page - maybe ask your great-grandparents).
    - jacalata 11y ago
      My great-grandparents are dead, you insensitive clod.
  - bunderbunder 11y ago
    It's mimicking something many print magazines do. But without any apparent understanding of why or how print magazines do it.
    - delecti 11y ago
      > But without any apparent understanding of why or how print magazines do it I have honestly never had any idea why print magazines do that. Maybe I've never seen a print magazine do it "right", but I think it seems just as pointless and annoying in print or online.
      - newjersey 11y ago
        Well, the problem is I think they are not for you. They are for lazy people like me who don't always read the whole articles. It is a sort of a few sprinkled tl;dr who just want to drive by. The Verge does it as well and I think it is OK.
knappe 11y ago
I've been on the other side of this as a consultant. I once was consulting for a place where I asked if I could run the existing tests to confirm I hadn't broken anything when adding in some new functionality. The owner of the project yelled (actually, really yelled) at me to never run the tests, ever. Apparently they deleted production data. I didn't spend much time consulting for them.
- AnimalMuppet 11y ago
  I would have given them a bid for fixing that problem...
  - troels 11y ago
    The yelling problem?
    - AnimalMuppet 11y ago
      The "can't run tests without deleting production data" problem. The yelling problem, I'd be tempted to fix for free...
      - joshmn 11y ago
        > The yelling problem, I'd be tempted to fix for free... So, run the tests?
        AnimalMuppet 11y ago
        My temptation was more toward a knuckle sandwich. Not a productive response, and probably death for a consultant's business, but I would be tempted...
- pavel_lishin 11y ago
  > Apparently they deleted production data. I would love to see the downhill-rolling trainwreck that eventually produced this sort of inferno.
joeld42 11y ago
I don't agree with the implicit premise that if the cheaper consultants had been required to maintain the tests and provide coverage for what they wrote, the code quality would have stayed high. In my experience, it would have turned into just as much of a mess, except would have taken three times as much work to maintain all the tests in the process. Maybe the tests would have had some benefits, but would they have enough to justify all the extra time to write/maintain them? Certainly not. In my experience, good programmers write good code, tests or not, and bad programmers write bad code, tests or not. I'm not against testing altogether, it makes sense in some contexts (like very widely used libraries). It's also useful for beginners who aren't sure "where to start" with a project. But even in cases where people swear that their tests are invaluable because it's tricky code that breaks often, that's a sign to me that that code needs to be made less fragile, not more well tested. TDD is a cargo cult.
- joesmo 11y ago
  I couldn't agree more. Almost every other form of testing, automated or manual, is more effective at catching bugs than unit testing in my experience. I rarely find unit tests catch any actual bugs (two I can remember in two years in two major codebases) and they are actually a liability rather than an asset because of that in many cases due to the time it takes to create and maintain them. Often, tests are so far removed from their actual run context with mocks and stubs that they're only testing the functionality of the mock/stub framework itself. OOP code is hardly ever unit test friendly by definition and a lot of bugs are at integration points and due to app state that won't show up in unit tests. There are few places, in other words, where they are useful outside known algorithms with expected outputs in OOP. For one bug prevented per application per year, it's easy to see why spending weeks a year on preventing said bug might not be worth it. I'd rather use them sparingly and test in other ways that are more effective at catching bugs.
  - twistedpair 11y ago
    > I rarely find unit tests catch any actual bugs What kind of code are you writing? Sure, a Spring service that just integrates a bunch of other services is hard to test due to mocks galore, but any dense logic blocks should be moved to standalone, testable classes. For my math libraries, Fibonacci compression algos, etc, I find the hundreds of unit tests invaluable. Add to that, in tools like SBT, "sbt ~testOnly com.MyTest" will rerun tests on every change, giving me feedback in under a second, rather than the whole dev/test cycle of save/restart_server/click_on_UI. Write unit tests for the tricky bits that will benefit from them.
revelation 11y ago
Of course, then when the OP was asked back for one particular thing, he didn't go and setup the unit tests again, either. He did the same exact thing as the consultants.
- tedmiston 11y ago
  He was contracted to fix one thing, not to rewrite the unit tests. At the end of the day, he can't change what the company does or doesn't value.
  - revelation 11y ago
    Yes, as are the consultants. Why put them down when they are doing exactly what was asked of them? Of course they don't bother with unit tests when asked to change thing A that breaks a bunch of them.
bunderbunder 11y ago
I don't think this should be surprising to anyone. Outside developers don't have any real ownership of their work product. Even if they know better in principle (and many do), they don't have any incentive to practice good stewardship. And bargain-basement outside developers aren't magical productivity pixies; the reason they are able to charge lower prices because they don't need to charge you for all the things they aren't taking the time to do.
deleted 11y ago
[deleted]
ErikAugust 11y ago
Moral of the story: Great developers are profit centers, bad developers are cost centers. Unfortunately, many execs without development experience neglect to see it this way. They see "expensive cost center" that can easily replaced with cheaper labor. Cool thing is, as others (like the author) have said, they tend to get burned and end up re-hiring you on even better-than-original terms. Kudos to the author for doing good work!
astockwell 11y ago
I have this same underlying feeling every place I work. Maybe it's because I'm not in the Valley, but in the last several jobs I've had, I was the only person advocating for things like unit tests, release pipelines, or even VCS(!). And each time I get a new job, team members and bosses from the old one inevitably call with similar self-inflicted crisis like this.
- dietrichepp 11y ago
  I don't think it's because you're not in the valley. I think it's because only a few percent of developers in the world care enough to become leads and make this stuff happen across the team.
- zeisss 11y ago
  No VCS? Let me guess, they write their business logic as stored procedures for databases.
scotty79 11y ago
That's normal. Any non-essential complexity gets dropped. Accidental complexity accumulates. In one project next developer ate my build script. Why bother with compiling less to css if you can just edit css and forget the less files and build script. Same went for r.js bundler. In other project I participated angular got dropped because people couldn't be bothered with it.
rsyring 11y ago
The bulk of the first comment of the article was as good as the article IMO. Quoting here: >> But maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)? > This is implying maybe they intentionally disorganize the code so they'll get more billable hours? > I seriously seriously doubt it. > Rather, it's as simple as: If you want to add features or make changes paying the cheapest possible amount, you simply aren't paying for well-organized code. > It's not just less experienced programmers, it's less experienced programmers trying to get you the feature in the least possible hours. They aren't doing artificial things to bill you more hours, it's in fact quite the reverse -- they really are trying to get the feature done in as few hours as possible, to bill you as few hours as possible. > And when you do that, you simply don't have time to organize the code well, or keep the tests working. You're shoving it out the door as quick as you can. Of course it takes more time to keep the code well-organized than to just hack it until the feature works as quickly as possible. Of course it takes more time to maintain the tests -- over the short term, and all you ever have is the short term when you're paying and getting paid little by little feature by feature. > They have other customers, they don't need to bill you artificial hours. They really are billing you as few hours as they possibly can -- because that's how they get customers, being the cheapest. That's exactly what the client asked for, it to be done as cheaply as possible. > You get what you pay for. I have only recently started working for a consultant that bills hourly. It's an eye opener. We are a small shop of experienced developers, we work for people that do want us to produce quality, and we do, we do good work, and we do write tests and maintain our tests. But it is a still a constant struggle inside my head between doing it as right as I really want to, and not charging the client more hours than the feature seems 'worth'. > If you as a client prioritizes price even higher over quality (and you have no way to judge internal quality anyway, not being a coder, so why wouldn't you? It looks nice and works, what else can you judge?), if you're hiring less expensive developers to save money... they might be skilled devs in fact, but if you are trying to get it done as cheaply as possible, you are not going to get well-organized code. And you as a client don't care, what do you care about well-organized code? Until it reaches a breaking point where your technical debt is so high you can't get any more features at all, and you realize, oops... and probably still don't understand what you did wrong or could have done differently. > Good software is expensive. Too expensive. More expensive than most people who need good software can afford. Which is why we have so much shitty software.
cognivore 11y ago
I'm going to be a contrarian here and say that if unit tests, in most cases, made economic sense they'd be a given in almost all software development. But they're not. Do we assume business owners are dumb? Do we assume managers are dumb? That's a popular position for technical people. "Oh, you just don't understand how valuable those unit tests are!" But the cost of maintaining unit test vs. the cost of fixing bugs later might favor just fixing the bugs later. Maybe it doesn't sometimes - but one is an upfront cost you know about, and one is a nebulous cost that you might or might not have to pay later. Given that, I know what most people paying the bills are going to choose.
- Ace17 11y ago
  Business owners are not dumb, they're ignorant ... like the rest of us. Software engineering is an immature field ; we still have doubts about what practises are beneficial. It comes as no surprise that our managers might not always know what will benefit their software in the long run. The whole notion of technical debt is just starting to make its way into their heads (and into ours). How many times was your manager astonished by the time estimate you just gave her while she was "only asking you to add a checkbox"?
- golergka 11y ago
  They make sense dependig on where your business objectives are on quality-speed-cost triangle.
- bo1024 11y ago
  It's a question of incentive alignment. Managers are not dumb, their incentives are aligned to short-term success rather than long term. Consider the difference in scenarios between producing ok code very quickly and good code more slowly. The manager will immediately feel the negative consequences of taking the second decision, if their incentives are badly designed. Meanwhile they may never feel the negative consequences of the first decision. When bad code breaks down, how likely is the manager of three years prior the one to take the blame or heat for it, let alone face real consequences? Often they won't.
chvid 11y ago
Some of the reasons why I delete unit tests from time to time: - Often the quality of unit test code is a lot worse than the production code. - Some times people test too much in a single test (ie. more than one unit) making it very hard to change the design of the application. - Some times a unit test misunderstands how the application is supposed to work or the requirements change. - Some times a unit test tests something too little or too trivial. - Some other test is already covering the case. I personally don't weight 100% test coverage as critical. Often other aspects of QA in application development are more important (i.e. integration test or documentation).
- msie 11y ago
  I don't know why you were down-voted. TDD fanatics?
Gratsby 11y ago
I had a different experience with tests. 18 years ago at one company I laid out a test plan. Some of it was automated, the majority of it was manual. It applied specifically to a large custom built application. 2 years ago, I was at an entirely different company. They handed me their test plan documentation. Lo and behold, it was my document, now laden with another consulting company's logo. It was missing a few tests, but there was nothing new. I was angry for a solid 4 hours. Then I had to laugh about it. I would be flattered that my 18 year old work was still being used, but it was quite simply dated. If a big company was being cheap and getting things built by offshore resources who had no better way to accomplish it than by stealing my work, then they got what they deserved.
stcredzero 11y ago
maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)? I know of one consulting shop that had a dependable stream of income from a major European bank, because that bank had a project that didn't use source control, so they got to be the periodic hero and get paid for it.
alexc05 11y ago
I, very recently, began writing a suite of tests to go along-side a complex mobile app. During a stand-up meeting I (somewhat) proudly announced that for a particularly onerous piece of business logic, I had 100% coverage with tests that run in less than a second. The project manager in the room was worried about the time it was going to take. All the developers in the room piped up to say "oh, no, this actually saves time" The next day, I found myself in a private meeting room with the project manager and my line manager. Both were trying to tell me that unit tests are not part of the scope of the project, not were they approved and, that I was to stop writing them. Manual QA was the way to go at the end of the project. Yesterday, I was in the office from 10am to 2am on a national holiday addressing issues arising from QA. It was unproductive to spend the day wondering how many of those defects I'd have been able to catch months ago if I'd been writing tests all along. When you're the only one on your team who respects the tests, the world feels like a very bleak place.
- douche 11y ago
  It does suck how much of the time you wind up in the situation that, to make any progress, you have to work in secret behind some smokescreen of other less useful work, to keep the micromanaging and the hemming and hawing from consuming any improvements. It's not uncommon that orders of magnitude more time wasted in debate and FUD'ing round on things than it takes to actually do them.
- chris_wot 11y ago
  Skunk works the tests. Maintain a private branch that management never sees and run them anyway. When they fail, fix the issue. You'll possibly seen as a genius, but you certainly won't be working on public holidays.
raulk 11y ago
Cheap labor != consultant. In fact, real consultants tend to be expensive. A real consultant is an expert in a field, not an asterisk or wildcard for anything.
vinceguidry 11y ago
Man, this makes me sad. Not about the atrophied codebase, but about the inevitably extreme difficulty of keeping one properly maintained. Expertise in software maintenance isn't common and it isn't cheap. Every greenfield dev is going to eventually see his project handed off to people who won't understand or respect his conventions and methods. That mostly destroys it, but business requirements can put it on chemo and a food tube and keep it alive against its will for a long, long time, and it's not a pretty sight. It's practically impossible to. I've been maintaining an ancient Rails codebase for the last two and a half years. I'm perhaps the fourth or fifth such guy to do so since it was built. The project was never updated past a certain point release and it's more work than it would be economical to do to do that now. I'm experienced enough now to take it on, but both me and the company have better things to do with our time and resources. It is so ridiculously easy to make a bad decision that seems like a good decision that instead ultimately dooms the project. Software is like this weird life-form from another part of the galaxy whose motives are so alien to us that we can't even begin to come to grips with it. We know it's not actively plotting against us, well, pretty sure, but more than that? All you can do is shrug your shoulders and hope it doesn't consider you a threat today.
daxfohl 11y ago
FWIW I've been eating my own unit tests these days. I find they slow me down and distract me from what I'm doing. You've got the "large scope" unit tests that test functions that take lots of dependencies, via mocking. These often end up being so mocked, that you're testing the mocks more than the function itself. You feel smart when you write it, but a week afterwards you can't even remember what it's testing. You've got the "small scope" tests that test small pure logic functions. Great, but these hardly ever change. It doesn't seem worth the overhead to maintain a test for unchanging code. There's a "fuzzy middle" that could be reasonably worth unit testing, but these are the functions that are most likely to get completely ripped up during refactoring and new feature dev. To me, a sound architecture (no global variables, a minimum of mutation[1], etc) and a sound type system means things generally work the first time anyway, without the need to maintain a big honking test suite and everything required to run it. I'm finding it's much quicker to get things out the door without this overhead, the architecture ends up being simpler (far simpler in some cases) because I don't have to put test hooks in for everything, I can refactor without the worry of how many unit tests will break, and I've experienced no downturn in quality. Of course there are cases (public core libraries, large distributed dev teams, really complex pure logic) where unit tests are absolutely necessary, but for most of my projects these days I'm finding them to be redundant. [1] In fact, one could argue that the greatest benefit from the TDD phase was that TDD is harder with globals and mutation, so it reduced incidences thereof.

128 points by dragthor 11y ago

I've seen this with my own two eyes. When asked why they removed the unit test they said after they made some rather extensive code changes the tests were failing so they deleted them. Nor was there any trace of irony in their response. When asked didn't they realize that was the whole point of having unit tests they replied that they knew, but they had deadlines to meet and there wasn't enough time. Since controls had been put in place to not ship with failing unit tests but not to check for test coverage they took the easy way out and deleted the tests. I informed management what they had done and thankfully they were relieved of their contract and their consulting company was informed what they had done. I don't know what happened to them after that. Thing is, once the PM was informed what they had done we did a couple of sprints where we restored the unit tests (which we still had in our history) and got them passing. They had no idea the consultants were doing such shoddy work in order to meet deadlines. The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted - and that's for all his projects not just that one particular projects. Now that the story is part of our IT lore the new consultants get the message loud and clear we're not afraid to send them packing. In the end this turned out to be a good story for us - but wow.

Spooky23 11y ago
Always assume that consultants will never perform any task that isn't required for milestone sign off.
- daigoba66 11y ago
  I'm a consultant; this is good advise. It's not laziness or apathy. It's just that we're only paid to do a task.
  - rukuu001 11y ago
    > It's not laziness or apathy. It's just that we're only paid to do a task. That's contractor-talk. If you're a consultant, consult. Educate your client, make sure they understand the consequences of what they're not paying you for.
    - tootie 11y ago
      Sometimes it's a tough sell. I work for a consulting company that I think would rate as very high quality. We try to bake it into contracts and sales pitches that we work a bit slower because we don't cut corners. This isn't always appreciated by a client who's rushing to market. If you think about the old project management triangle, it's scope, budget quality and you can only have 2 out of 3. If you have a contract with fixed scope and budget, it's quality that goes. Getting a client to sign a contract with flexible scope is not easy unless you have a ton of credibility.
      - jonesb6 11y ago
        "Rushing to market" Reminds me of the kid going "I want to play, I want to play".. sorry George you suck at basketball, go shoot some free throws. "You won't believe one startup's new growth hack, their teammates hate them!"
  - catmanjan 11y ago
    Good luck explaining the point of unit tests to technology illiterate managers - they don't understand and really they shouldn't have to.
    - myko 11y ago
      > they don't understand and really they shouldn't have to. They shouldn't have to because they'll listen to us and allow us to write and maintain tests as we develop, right?
      - rhizome31 11y ago
- megablast 11y ago
  Well of course. If unit tests aren't in the requirements, and it sounds like they are not, why do them? You have a dealine, people want you out of there asap, you don't start doing extra work. That is going to piss off everyone. Most companies don't value tests, and it sounds like these guys didn't either.
Nutmog 11y ago
Regarding the manager's response, are unit tests any use against someone who's trying to circumvent them? If a test passes a couple of values into a function and checks the result, wouldn't the "attacker"/programmer just hardcode those expected results into the function for those input values? Perhaps because that kind of code would look obviously suspicious?
- Huggernaut 11y ago
  My experience is that if you can cheat the tests in this way then they are either too narrow in scope, or badly written tests. This is something that TDD tries to expose when being adhered to strictly, that is, do the simplest thing to make this test pass - if you `assertTrue(isEmpty())` then I can simply `return true` to make you improve your tests.
  - chris_wot 11y ago
    If a consultant writes assert(isEmpty()) then the first thing I'm going to do is fire up version control, find the consultant and have them fired.
Natsu 11y ago
> The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted I just hope that story doesn't continue in the future with something like: @Test public void testFoo(){ /* Original testFoo() code here. */ assertTrue(true); // Unnecessary bloat. }
- chopin 11y ago
  Been there, seen that.
benashford 11y ago
That story had an unexpectedly happy ending. Several times in the past when I've needed to complain about external consultants cutting corners, I've found the consultants have got in first and told my own management about their unique red-tape cutting powers, and how things like continuous integration (yes, the first thing they did was to switch off the CI server so no-one knew what was happening) were a time-consuming luxury.

jasonkester 11y ago

Ah, the sadness of seeing your code again after somebody else has had it for a couple years. I feel the author's pain. I had a company license the source code from one of my old projects to use as the basis for a new site they were building. I built out the bits they needed on a consulting basis and got them launched. Then they started thinking about how expensive I was, and how much they would save if they moved the development overseas. Fair enough. My philosophy on such matters is that it's best to let a client learn this lesson through painful experience. All the best, guys. They made it about a year before the site stopped working completely. Reading this article reminded me of the first time I opened the source that came back. You just wouldn't believe that you could do that much damage in such a short time. I quoted them double my old bill rate and got them back up and running, with all the new functionality they had been wanting, and re-launched. They brought in a new guy to run things, and he started making comments about expenses, and how surely there must be a way to do this cheaper. He even found a local kid who knew some C# and would be willing to help out for a lot less than I was billing. No problem. All the best. I took a quick look at the codebase when they got back in touch a few months later and politely declined to come back on board.

sriram_sun 11y ago
Wasn't worth doubling rates again? :)
- ams6110 11y ago
  Some things just aren't worth any amount of money, and all the moreso if you have other options.
chrisabrams 11y ago
It's sad how often this scenario comes up :/ get what you pay for, plus another, larger bill later. The technical debt collector will find you.
- jxramos 11y ago
  yes he will find you and he holds a mighty ransom!

x3n0ph3n3 11y ago

The format of this article was downright annoying.

chillaxtian 11y ago
yes. i don't understand the trend of duplicating quotes from the article inline in a larger font.
- to3m 11y ago
  It used to be common in magazines (a magazine is like a Moleskine with a bit of blog post on each page - maybe ask your great-grandparents).
  - jacalata 11y ago
    My great-grandparents are dead, you insensitive clod.
- bunderbunder 11y ago
  It's mimicking something many print magazines do. But without any apparent understanding of why or how print magazines do it.
  - delecti 11y ago
    > But without any apparent understanding of why or how print magazines do it I have honestly never had any idea why print magazines do that. Maybe I've never seen a print magazine do it "right", but I think it seems just as pointless and annoying in print or online.
    - newjersey 11y ago
      Well, the problem is I think they are not for you. They are for lazy people like me who don't always read the whole articles. It is a sort of a few sprinkled tl;dr who just want to drive by. The Verge does it as well and I think it is OK.

knappe 11y ago

I've been on the other side of this as a consultant. I once was consulting for a place where I asked if I could run the existing tests to confirm I hadn't broken anything when adding in some new functionality. The owner of the project yelled (actually, really yelled) at me to never run the tests, ever. Apparently they deleted production data. I didn't spend much time consulting for them.

AnimalMuppet 11y ago
I would have given them a bid for fixing that problem...
- troels 11y ago
  The yelling problem?
  - AnimalMuppet 11y ago
    The "can't run tests without deleting production data" problem. The yelling problem, I'd be tempted to fix for free...
    - joshmn 11y ago
      > The yelling problem, I'd be tempted to fix for free... So, run the tests?
      - AnimalMuppet 11y ago
        My temptation was more toward a knuckle sandwich. Not a productive response, and probably death for a consultant's business, but I would be tempted...
pavel_lishin 11y ago
> Apparently they deleted production data. I would love to see the downhill-rolling trainwreck that eventually produced this sort of inferno.

joeld42 11y ago

I don't agree with the implicit premise that if the cheaper consultants had been required to maintain the tests and provide coverage for what they wrote, the code quality would have stayed high. In my experience, it would have turned into just as much of a mess, except would have taken three times as much work to maintain all the tests in the process. Maybe the tests would have had some benefits, but would they have enough to justify all the extra time to write/maintain them? Certainly not. In my experience, good programmers write good code, tests or not, and bad programmers write bad code, tests or not. I'm not against testing altogether, it makes sense in some contexts (like very widely used libraries). It's also useful for beginners who aren't sure "where to start" with a project. But even in cases where people swear that their tests are invaluable because it's tricky code that breaks often, that's a sign to me that that code needs to be made less fragile, not more well tested. TDD is a cargo cult.

joesmo 11y ago
I couldn't agree more. Almost every other form of testing, automated or manual, is more effective at catching bugs than unit testing in my experience. I rarely find unit tests catch any actual bugs (two I can remember in two years in two major codebases) and they are actually a liability rather than an asset because of that in many cases due to the time it takes to create and maintain them. Often, tests are so far removed from their actual run context with mocks and stubs that they're only testing the functionality of the mock/stub framework itself. OOP code is hardly ever unit test friendly by definition and a lot of bugs are at integration points and due to app state that won't show up in unit tests. There are few places, in other words, where they are useful outside known algorithms with expected outputs in OOP. For one bug prevented per application per year, it's easy to see why spending weeks a year on preventing said bug might not be worth it. I'd rather use them sparingly and test in other ways that are more effective at catching bugs.
- twistedpair 11y ago
  > I rarely find unit tests catch any actual bugs What kind of code are you writing? Sure, a Spring service that just integrates a bunch of other services is hard to test due to mocks galore, but any dense logic blocks should be moved to standalone, testable classes. For my math libraries, Fibonacci compression algos, etc, I find the hundreds of unit tests invaluable. Add to that, in tools like SBT, "sbt ~testOnly com.MyTest" will rerun tests on every change, giving me feedback in under a second, rather than the whole dev/test cycle of save/restart_server/click_on_UI. Write unit tests for the tricky bits that will benefit from them.

revelation 11y ago

Of course, then when the OP was asked back for one particular thing, he didn't go and setup the unit tests again, either. He did the same exact thing as the consultants.

tedmiston 11y ago
He was contracted to fix one thing, not to rewrite the unit tests. At the end of the day, he can't change what the company does or doesn't value.
- revelation 11y ago
  Yes, as are the consultants. Why put them down when they are doing exactly what was asked of them? Of course they don't bother with unit tests when asked to change thing A that breaks a bunch of them.

bunderbunder 11y ago

I don't think this should be surprising to anyone. Outside developers don't have any real ownership of their work product. Even if they know better in principle (and many do), they don't have any incentive to practice good stewardship. And bargain-basement outside developers aren't magical productivity pixies; the reason they are able to charge lower prices because they don't need to charge you for all the things they aren't taking the time to do.

deleted 11y ago

[deleted]

ErikAugust 11y ago

Moral of the story: Great developers are profit centers, bad developers are cost centers. Unfortunately, many execs without development experience neglect to see it this way. They see "expensive cost center" that can easily replaced with cheaper labor. Cool thing is, as others (like the author) have said, they tend to get burned and end up re-hiring you on even better-than-original terms. Kudos to the author for doing good work!

astockwell 11y ago

I have this same underlying feeling every place I work. Maybe it's because I'm not in the Valley, but in the last several jobs I've had, I was the only person advocating for things like unit tests, release pipelines, or even VCS(!). And each time I get a new job, team members and bosses from the old one inevitably call with similar self-inflicted crisis like this.

dietrichepp 11y ago
I don't think it's because you're not in the valley. I think it's because only a few percent of developers in the world care enough to become leads and make this stuff happen across the team.
zeisss 11y ago
No VCS? Let me guess, they write their business logic as stored procedures for databases.

scotty79 11y ago

That's normal. Any non-essential complexity gets dropped. Accidental complexity accumulates. In one project next developer ate my build script. Why bother with compiling less to css if you can just edit css and forget the less files and build script. Same went for r.js bundler. In other project I participated angular got dropped because people couldn't be bothered with it.

rsyring 11y ago

The bulk of the first comment of the article was as good as the article IMO. Quoting here: >> But maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)? > This is implying maybe they intentionally disorganize the code so they'll get more billable hours? > I seriously seriously doubt it. > Rather, it's as simple as: If you want to add features or make changes paying the cheapest possible amount, you simply aren't paying for well-organized code. > It's not just less experienced programmers, it's less experienced programmers trying to get you the feature in the least possible hours. They aren't doing artificial things to bill you more hours, it's in fact quite the reverse -- they really are trying to get the feature done in as few hours as possible, to bill you as few hours as possible. > And when you do that, you simply don't have time to organize the code well, or keep the tests working. You're shoving it out the door as quick as you can. Of course it takes more time to keep the code well-organized than to just hack it until the feature works as quickly as possible. Of course it takes more time to maintain the tests -- over the short term, and all you ever have is the short term when you're paying and getting paid little by little feature by feature. > They have other customers, they don't need to bill you artificial hours. They really are billing you as few hours as they possibly can -- because that's how they get customers, being the cheapest. That's exactly what the client asked for, it to be done as cheaply as possible. > You get what you pay for. I have only recently started working for a consultant that bills hourly. It's an eye opener. We are a small shop of experienced developers, we work for people that do want us to produce quality, and we do, we do good work, and we do write tests and maintain our tests. But it is a still a constant struggle inside my head between doing it as right as I really want to, and not charging the client more hours than the feature seems 'worth'. > If you as a client prioritizes price even higher over quality (and you have no way to judge internal quality anyway, not being a coder, so why wouldn't you? It looks nice and works, what else can you judge?), if you're hiring less expensive developers to save money... they might be skilled devs in fact, but if you are trying to get it done as cheaply as possible, you are not going to get well-organized code. And you as a client don't care, what do you care about well-organized code? Until it reaches a breaking point where your technical debt is so high you can't get any more features at all, and you realize, oops... and probably still don't understand what you did wrong or could have done differently. > Good software is expensive. Too expensive. More expensive than most people who need good software can afford. Which is why we have so much shitty software.

cognivore 11y ago

I'm going to be a contrarian here and say that if unit tests, in most cases, made economic sense they'd be a given in almost all software development. But they're not. Do we assume business owners are dumb? Do we assume managers are dumb? That's a popular position for technical people. "Oh, you just don't understand how valuable those unit tests are!" But the cost of maintaining unit test vs. the cost of fixing bugs later might favor just fixing the bugs later. Maybe it doesn't sometimes - but one is an upfront cost you know about, and one is a nebulous cost that you might or might not have to pay later. Given that, I know what most people paying the bills are going to choose.

Ace17 11y ago
Business owners are not dumb, they're ignorant ... like the rest of us. Software engineering is an immature field ; we still have doubts about what practises are beneficial. It comes as no surprise that our managers might not always know what will benefit their software in the long run. The whole notion of technical debt is just starting to make its way into their heads (and into ours). How many times was your manager astonished by the time estimate you just gave her while she was "only asking you to add a checkbox"?
golergka 11y ago
They make sense dependig on where your business objectives are on quality-speed-cost triangle.
bo1024 11y ago
It's a question of incentive alignment. Managers are not dumb, their incentives are aligned to short-term success rather than long term. Consider the difference in scenarios between producing ok code very quickly and good code more slowly. The manager will immediately feel the negative consequences of taking the second decision, if their incentives are badly designed. Meanwhile they may never feel the negative consequences of the first decision. When bad code breaks down, how likely is the manager of three years prior the one to take the blame or heat for it, let alone face real consequences? Often they won't.

chvid 11y ago

Some of the reasons why I delete unit tests from time to time: - Often the quality of unit test code is a lot worse than the production code. - Some times people test too much in a single test (ie. more than one unit) making it very hard to change the design of the application. - Some times a unit test misunderstands how the application is supposed to work or the requirements change. - Some times a unit test tests something too little or too trivial. - Some other test is already covering the case. I personally don't weight 100% test coverage as critical. Often other aspects of QA in application development are more important (i.e. integration test or documentation).

msie 11y ago
I don't know why you were down-voted. TDD fanatics?

Gratsby 11y ago

I had a different experience with tests. 18 years ago at one company I laid out a test plan. Some of it was automated, the majority of it was manual. It applied specifically to a large custom built application. 2 years ago, I was at an entirely different company. They handed me their test plan documentation. Lo and behold, it was my document, now laden with another consulting company's logo. It was missing a few tests, but there was nothing new. I was angry for a solid 4 hours. Then I had to laugh about it. I would be flattered that my 18 year old work was still being used, but it was quite simply dated. If a big company was being cheap and getting things built by offshore resources who had no better way to accomplish it than by stealing my work, then they got what they deserved.

stcredzero 11y ago

maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)? I know of one consulting shop that had a dependable stream of income from a major European bank, because that bank had a project that didn't use source control, so they got to be the periodic hero and get paid for it.

alexc05 11y ago

I, very recently, began writing a suite of tests to go along-side a complex mobile app. During a stand-up meeting I (somewhat) proudly announced that for a particularly onerous piece of business logic, I had 100% coverage with tests that run in less than a second. The project manager in the room was worried about the time it was going to take. All the developers in the room piped up to say "oh, no, this actually saves time" The next day, I found myself in a private meeting room with the project manager and my line manager. Both were trying to tell me that unit tests are not part of the scope of the project, not were they approved and, that I was to stop writing them. Manual QA was the way to go at the end of the project. Yesterday, I was in the office from 10am to 2am on a national holiday addressing issues arising from QA. It was unproductive to spend the day wondering how many of those defects I'd have been able to catch months ago if I'd been writing tests all along. When you're the only one on your team who respects the tests, the world feels like a very bleak place.

douche 11y ago
It does suck how much of the time you wind up in the situation that, to make any progress, you have to work in secret behind some smokescreen of other less useful work, to keep the micromanaging and the hemming and hawing from consuming any improvements. It's not uncommon that orders of magnitude more time wasted in debate and FUD'ing round on things than it takes to actually do them.
chris_wot 11y ago
Skunk works the tests. Maintain a private branch that management never sees and run them anyway. When they fail, fix the issue. You'll possibly seen as a genius, but you certainly won't be working on public holidays.

raulk 11y ago

Cheap labor != consultant. In fact, real consultants tend to be expensive. A real consultant is an expert in a field, not an asterisk or wildcard for anything.

vinceguidry 11y ago

Man, this makes me sad. Not about the atrophied codebase, but about the inevitably extreme difficulty of keeping one properly maintained. Expertise in software maintenance isn't common and it isn't cheap. Every greenfield dev is going to eventually see his project handed off to people who won't understand or respect his conventions and methods. That mostly destroys it, but business requirements can put it on chemo and a food tube and keep it alive against its will for a long, long time, and it's not a pretty sight. It's practically impossible to. I've been maintaining an ancient Rails codebase for the last two and a half years. I'm perhaps the fourth or fifth such guy to do so since it was built. The project was never updated past a certain point release and it's more work than it would be economical to do to do that now. I'm experienced enough now to take it on, but both me and the company have better things to do with our time and resources. It is so ridiculously easy to make a bad decision that seems like a good decision that instead ultimately dooms the project. Software is like this weird life-form from another part of the galaxy whose motives are so alien to us that we can't even begin to come to grips with it. We know it's not actively plotting against us, well, pretty sure, but more than that? All you can do is shrug your shoulders and hope it doesn't consider you a threat today.

daxfohl 11y ago

FWIW I've been eating my own unit tests these days. I find they slow me down and distract me from what I'm doing. You've got the "large scope" unit tests that test functions that take lots of dependencies, via mocking. These often end up being so mocked, that you're testing the mocks more than the function itself. You feel smart when you write it, but a week afterwards you can't even remember what it's testing. You've got the "small scope" tests that test small pure logic functions. Great, but these hardly ever change. It doesn't seem worth the overhead to maintain a test for unchanging code. There's a "fuzzy middle" that could be reasonably worth unit testing, but these are the functions that are most likely to get completely ripped up during refactoring and new feature dev. To me, a sound architecture (no global variables, a minimum of mutation[1], etc) and a sound type system means things generally work the first time anyway, without the need to maintain a big honking test suite and everything required to run it. I'm finding it's much quicker to get things out the door without this overhead, the architecture ends up being simpler (far simpler in some cases) because I don't have to put test hooks in for everything, I can refactor without the worry of how many unit tests will break, and I've experienced no downturn in quality. Of course there are cases (public core libraries, large distributed dev teams, really complex pure logic) where unit tests are absolutely necessary, but for most of my projects these days I'm finding them to be redundant. [1] In fact, one could argue that the greatest benefit from the TDD phase was that TDD is harder with globals and mutation, so it reduced incidences thereof.