3 ms·

The economics of software are broken as the article somewhat suggests, but this is less simple than the article suggests. First of all most software developers

The economics of software are broken as the article somewhat suggests, but this is less simple than the article suggests. First of all most software developers have no idea what a quality software developer is until they see somebody who delivers 10x productivity and even then they still cannot typically extract what makes that one developer so much more productive. This blindness is often present in organizations that don't know how to hire or who value their subjective considerations over objective criteria. Secondly, there is little or no incentive to hire a good developer. A greater developer may deliver up to 4-10x productivity on only a 2-3x salary, but it also typically costs more to find good developers. This problem is deeply compounded by the previous point in that many organizations cannot identify what a good developer is. Third, most organizations don't want high quality software. They want software that is popular and appealing to entry level developers. As an example search just about any article for simple versus easy and then consider what you are reading against some of the processes, configurations, and piles of abstractions you have to go through at work to make things easier. Fourth, and perhaps most importantly, most organizations cannot account for economic considerations until they become huge, as in retaining hundreds or even thousands of developers. Economics, particular software economics, is not something vision that is well understood either academically or in practice. I mostly agree with the rest of article except for the extreme praise of code coverage. Artificially boosting code coverage with unnecessary tests is one of the greatest contributors to tech debt, as all tests are ultimately debt never seen or appreciated by the end user. Code coverage does not account for whether tests are positive versus negative tests, whether collisions of features uncover unexpected behavior, or test quality. It also, in many cases, does not even account for whether the code is working. It isn't supposed to. Code coverage simply lets you know what code is unnecessary to your automation. Instead think of covering code like this: * If there is critical code there should be some manner of test automation to account for it in the various ways it is executed by a user. If that critical code is removed existing tests should break, the code is actually not critical at all, or you have missing tests. Developers should be rewarded for removing unnecessary code. * Tests take time to write and execute. That wasted time is a form of debt. This wastes people time. You want to ensure the software works correctly and that features do not introduce untended defects or regression but you don't want to spend more developer time on test execution than writing software. I have seen this at a major .com. * Code coverage is useful to determine what code is executed during testing and what code isn't. That is all it is useful for. When tests are written to artificially boost code coverage analysis the analysis becomes a meaningless way for developers to justify increased effort without contributing value back to the application. Instead, use code coverage analysis to make decisions about what code to remove, refactor, or rearrange.

nfrbc 7y ago
I completely agree with the code coverage part. A previous codebase I worked in was terrible to refactor because of the insane amount of tautological unit tests and abuse of mocking. For new features we stopped writing tests that were too specific and started writing more integration tests. We achieved a leap in quality and ease of development.
- dwoot 7y ago
  This! I'm working in a codebase at the moment where unit tests do the following: 1. cross multiple boundaries (integration test) by asserting that some system two degrees away from itself is being run/invoked (not a unit test's duty) 2. test descriptions read along the lines of, "it should work for valid use cases". This one made me laugh when I first saw it. This made refactoring a HUGE undertaking as I had to comb through all of it to understand it 3. mocking systems that weren't created by "us" -- this resulted in blind spots in various places Test/code coverage is only one of many metrics to consider. I think it's useful, but its utility heavily relies on the quality of tests that are written. I'm beginning to think that a hallmark of a good developer is one that understand what tests need to be written and what assertions must be made. This shines when the tests are written such that the developer can see how API design affects testing. I've worked in both functional and imperative languages and FP testing is harder to screw up as it forces the use of dependency injection. The idea of DI and inserting role players in imperative langs makes testing easier and shows that someone has thought about a reliance on behavior rather than concretions. So when I interview people, I always ask about dependency injection and polymorphism. But working with seemingly more experienced (on paper) than me, I know that experience is also a terrible metric to rely on.
  - nfrbc 7y ago
    I had all three of the things you described. But let me add a new one to the list that was incredibly frustrating: 4. Instead of integration tests (or static typing), all systems made by a couple programmers only had unit tests. The call boundary was NEVER tested, and everything relied on RSpec's "allow(obj).to receive" mocking. So, not only it was very difficult to refactor (because I had to change two or more different sets of tests + mocks), the tests for dependencies didn't fail when I changed function signatures, or even if I removed chunks of code randomly. I had to find which test to change by myself. In the end we had tests and we had coverage, but they didn't test anything properly and made maintenance a major pain in the ass.

nfrbc 7y ago

I completely agree with the code coverage part. A previous codebase I worked in was terrible to refactor because of the insane amount of tautological unit tests and abuse of mocking. For new features we stopped writing tests that were too specific and started writing more integration tests. We achieved a leap in quality and ease of development.

dwoot 7y ago
This! I'm working in a codebase at the moment where unit tests do the following: 1. cross multiple boundaries (integration test) by asserting that some system two degrees away from itself is being run/invoked (not a unit test's duty) 2. test descriptions read along the lines of, "it should work for valid use cases". This one made me laugh when I first saw it. This made refactoring a HUGE undertaking as I had to comb through all of it to understand it 3. mocking systems that weren't created by "us" -- this resulted in blind spots in various places Test/code coverage is only one of many metrics to consider. I think it's useful, but its utility heavily relies on the quality of tests that are written. I'm beginning to think that a hallmark of a good developer is one that understand what tests need to be written and what assertions must be made. This shines when the tests are written such that the developer can see how API design affects testing. I've worked in both functional and imperative languages and FP testing is harder to screw up as it forces the use of dependency injection. The idea of DI and inserting role players in imperative langs makes testing easier and shows that someone has thought about a reliance on behavior rather than concretions. So when I interview people, I always ask about dependency injection and polymorphism. But working with seemingly more experienced (on paper) than me, I know that experience is also a terrible metric to rely on.
- nfrbc 7y ago
  I had all three of the things you described. But let me add a new one to the list that was incredibly frustrating: 4. Instead of integration tests (or static typing), all systems made by a couple programmers only had unit tests. The call boundary was NEVER tested, and everything relied on RSpec's "allow(obj).to receive" mocking. So, not only it was very difficult to refactor (because I had to change two or more different sets of tests + mocks), the tests for dependencies didn't fail when I changed function signatures, or even if I removed chunks of code randomly. I had to find which test to change by myself. In the end we had tests and we had coverage, but they didn't test anything properly and made maintenance a major pain in the ass.