6 ms·
This is a no-brainer. As a development manager for a quarter-century, and an active software developer for a lot longer than that, I can definitely say that ev
by ChrisMarshallNY 7y ago
This is a no-brainer.
As a development manager for a quarter-century, and an active software developer for a lot longer than that, I can definitely say that every place there's a "meeting of the minds" is a place for bugs.
In the software itself, the more complex the design, the more of these "trouble nodes" (what I call them) there are. Each interface, each class, each data interface, is a bug farm.
That's why I'm a skeptic of a number of modern development practices that deliberately increase the complexity of software. I won't name them, because I will then get a bunch of pithy responses.
These practices are often a response to the need to do Big Stuff.
In order to do Big Stuff, you need a Big Team.
In order to work with a Big Team, you need a Big Plan.
That Big Plan needs to have delegation to all the team members; usually by giving each one a specific domain, and specifying how they will interact with each other in a Big Integration Plan.
Problem is, you need this "Big" stuff. It's crazy to do without it.
The way that I have found works for me, is to have an aggregate of much more sequestered small parts, each treated as a separate full product. It's a lot more work, and takes a lot more time, with a lot more overhead, but it results in a really high-quality product, and also has a great deal of resiliency and flexibility.
There is no magic bullet.
Software development is hard.
- jve 7y ago> I'm a skeptic of a number of modern development practices that deliberately increase the complexity of software. I won't name them, because I will then get a bunch of pithy responses. Bring it on. Share your experience with youngsters. And let the elite confront with methodologies you maybe didn't have experience with.
- ChrisMarshallNY 7y agoNah, that's OK. Thanks. Just to clarify. I have been down this road. I am not interested in sacred cows or third rails. I'm trying to do all my writing and commenting, based only on my own experience and insight. I'm done with fighting on the Internet. I don't have the energy for it anymore.
- carlmr 7y agoI can respect that. Although I would have liked to hear your take on it, I know what you mean.
- ChrisMarshallNY 7y agoI'm always happy to chat (pontificate in an "OK Boomer" kind of way, I guess), but not in public forums. My screen name applies in a lot of places.
- bob33212 7y agoAgile and Scrum can add a lot of meetings and processes and metrics into a development team. For non-technical managers this is great because they can create nice charts with story points for upper management. The above comment can devolve into a flame war because non-technical managers see Agile and Scrum in a different light. They believe that without proper management developers will be unproductive.
- carlmr 7y agoI can't believe I forgot this one. Scrum is so terrible, I don't know why it's still being introduced.
- carlmr 7y agoI'd also be interested. Some drivers of complexity I would think of: * Using many of the GoF/OOP patterns, because you may need extensibility at some point. Basically YAGNI. * Complex, hard to mentally map, build systems (e.g. CMake). * Designing for purity over simplicity (I'm actually big on FP, here I'm thinking of the Haskell crowd which IMHO sometimes overdoes it). * Writing a complex architecture without prototyping. Often your prototype will tell you what you need. If you start architecting too much beforehand then you often waste time on some details that don't matter, and even worse, afterwards you try to force it into your architecture which doesn't actually fit the problem. The beauty of software is that it's easy to change things. Architecture on buildings is different because you need to make sure that you're not building the wrong thing. In software building the wrong thing can give you the right insights and still be faster than planning for every eventuality.
- heisenbit 7y ago* Splitting requirements documents into user stories and then evolving the document further. This practice can be improved upon by adding redundancy with attaching the document to user stories and enterprise wiki. Further optimization can get to the org to CMMI level 5 but requires investment in diversity of the most critical requirements in other locations and protection of these assets from being linked.
- doublement 7y ago> Writing a complex architecture without prototyping. Often your prototype will tell you what you need. One million times yes. In my experience using a prototype as an input to a specification works much better than the other way around. > Complex, hard to mentally map, build systems (e.g. CMake). Also yes. It's almost to the point where one has to understand every detail of how CMake works to get it to do one (1) specific thing you need in your build process.
- calineczka 7y ago> an aggregate of much more sequestered small parts, each treated as a separate full product What does it mean exactly? I feel you are trying to share a nice idea but I can't comprehend it. What are those small parts? Classes? Modules? Services? What does it mean to treat them as a separate full product within an organization?
- winrid 7y agoBased on my understanding a common name would probably be Slimlane, or Blade...
- carlmr 7y agoNot OP, but to me this is the Unix philosophy of having many small tools that work well and interact well. Even if your modules are very separated, if you can't individually use and play around with them they become a part of a big blob of software. Services may be products, but only if they're idependently usable. If you have a small product that's useful in and of itself (e.g. git) you can much more easily make it work well and then integrate with other good tools and replace those if necessary (e.g. if you have problems with Bitbucket/Jira/Confluence, you can switch them out for other solutions, e.g. Gitea). But if you have a huge clomplex product then at some point it becomes organizationally impossible to move away from it.
- ChrisMarshallNY 7y agoExactly.
- nradov 7y agoThe Unix philosophy is all well and good, but it's very limiting. There are major types of valuable software products that simply can't be built out of small tools or services.
- ChrisMarshallNY 7y agoExactly. It's just one way of doing things, and I definitely admit that some very good things have come from much more complex systems. However, bug control is absolutely vital with these systems, and you definitely need some kind of quality-assurance system, or you will be hatin' life.
- jiggawatts 7y agoSo, just a day ago, I got dragged into a meeting where many people were involved in a discussion about the new Cloud Enterprise Application Architecture Template. Or whatever. It had a 3-tier architecture. I asked: Why? And they answered: Why not? I answered: Because layers must only be introduced if needed. Is there a need? They answered: The standard design is the need. I clarified: Is there a technical requirement? Or perhaps an organisation one, such as disparate teams working on the two components? They answered: No! Of course not! It's a unified codebase for a single app written by a single person! But it is not Enterprise enough! It must be split into layers! And then, you see, it will will match our pattern and belong. I verified the insanity: Are you saying that this finished, working application isn't currently split into layers, but you want it split into layers simply so that it can have layers? They chorused: Yes.
- NicoJuicy 7y agoI actually like DDD, where code is executed per domain with a bounded context. Not for a single person though, but it will force kin developers to think more deeply. Otherwise it could easily result in a code-mesh/hell.
- Vinnl 7y ago> I answered: Because layers must only be introduced if needed. Is there a need? It might have gone better if you had also stated why that is the case, e.g. "every additional layer exponentially increases the likelihood of bugs being introduced, so their introduction must be worth that risk, or the higher cost of mitigating measures". Of course, the challenge will still be that "likelihood of bugs" is rather abstract, and often people believe they can be prevented just by paying more attention, and assume that that will happen of its own accord.
- organsnyder 7y agoI did a project like that at my old job. It was a single screen with a single purpose (login the user via oauth SSO, then collect a couple of pieces of input and submit them to a backend webservice), so it was assigned to me rather than creating a project team. To make it match the rest of our application architecture I created a REST layer in C# that communicated via SOAP to an ESB that forwarded the SOAP request to a Java webservice that finally forwarded it to the backend system. I did depart from our organizational norm by doing the frontend with plain HTML+JS (plus Boostrap, IIRC) rather than AngularJS. Yes, it was complicated, but I think there is a benefit: it's very clear where certain functionality should live. The C# REST layer was application-facing, so it took care of SSO and basic validation. The Java webservice contained the business logic to validate things from an broader enterprise perspective. The ESB was a piece of trash that did provide authnz so the Java webservice didn't have to. Was it worth the complexity? Probably not, in this case. But those sorts of applications tend to have long lifespans and evolving requirements, so the standardization can be helpful.
- andydavieswork 7y ago> In order to do Big Stuff, you need a Big Team. Depends on your definition of Big Stuff. If you mean send a rocket to Mars, then yes. But the vast majority of us are working on simple web apps that might call a few apis, yet these seem to require Big Teams. Compare that to what a single game developer might produce, and compare the complexity and performance of the product. I think we need Big Teams for Small Stuff precisely _because_ of these 'modern development practices' that you mention. Getting things done in these paradigms takes _forever_, so you need a Big Team.
- ChrisMarshallNY 7y agoThat's true. Like I said, I can only speak from my experience. I do think that we are in a sort of "dependency hell," that is sorting itself out. In the end, a few really good dependencies will still be standing in the blasted wasteland. Dependencies mean that a small team can do Big Stuff, but that relies on the dependency being good. "Good" means a lot of things. Low bug count is one metric, but so is clear documentation, community support, developer support, and even tribal knowledge. It doesn't necessarily mean "buzzword-compliant," but sometimes aligning to modern "buzz" means that it benefits from the tribal knowledge that exists for that term, and you can deprecate some things like documentation and training. People often think that I'm a dependency curmudgeon. I'm not. I am, however, a dependency skeptic. I will rely on operating system frameworks and utilities almost without question, but I won't just add any old data processor to my project because it's "cool." I need to be convinced that it has good quality, good support, and a high "bus coefficient," not to mention that it meets my needs, and doesn't introduce a lot of extra overhead. Nothing sucks more than building a system, based on a subsystem that disintegrates a year down the road. I suspect many folks that have built systems based on various Google tech, can relate. I have had that experience with Apple tech, over the years (Can you say "OpenDoc"? I knew you could!).
- chiefalchemist 7y ago> I think we need Big Teams for Small Stuff precisely _because_ of these 'modern development practices' that you mention. Perhaps. But what I've also seen is the head count of a given project is a direct reflection of the intra-org status of the person heading the project. There's a belief - that's a myth - that if 3 ppl is good the 6 is twice as good and time will be cut in half. I think we also know - with rare exception - that productivity slides as heads increase. That's a given. Then there's also a belief - again a myth - that some mod dev practices can fix the increased head count issue. It might mitigate it here and there. But MDP can only do so much to fix a dysfunctional org/group. Ultimately it's a leadership/management issue. Process and technology are too often lipstick on a pig.
- the_af 7y ago> That's why I'm a skeptic of a number of modern development practices that deliberately increase the complexity of software Maybe I'm nitpicking, but the article points out the number #1 predictor of software bugs is not the complexity of software but of the organization itself. A single person can make an hugely complex piece of software, and a relatively large team can make a conceptually simple system. As for software complexity itself, there's an interesting research result that the thing that matters the most is line count. Not cyclomatic complexity, not the type system, not modularity or test coverage, not the programming language -- looking at the line count alone trumped all the other metrics in predicting flaws. (I can't look for this paper now, but I'm sure with a bit of googling anyone can).
- ChrisMarshallNY 7y agoThat makes sense. Every line is a potential bug. But it's not the only factor, and, quite frequently, it's a matter of correlation, as opposed to causation. That kind of thing can be very tricky to determine. When I write software, my first stab at a function tends to be a fairly linear, high-LoC solution, which I then refactor in stages; reducing LoC each time, and ensuring that the quality level remains consistent, or improves. As far as quality goes, my first, naive stab, was just fine, and I have actually introduced bugs during my refactoring reduction.
- the_af 7y agoNote that I'm not arguing about underlying reasons, just saying what the empirical results show. That's reality. So if you try to predict software bugs using modularization (or lack thereof) or "if-then-else" branches, or whatever complexity metric you can think of, you'll get one result. If you try to predict them using simple line count, you'll get another result. The second one will have better precision & recall. So no metric so far has been shown to be better than simply counting lines. That's not an obvious result, but it's the truth. Sadly, you'll have to believe me because I cannot find the studies right now.
- shantly 7y ago
- pjc50 7y agoIn order to get Big VC Money, before doing a Big IPO, you need a Big Team doing a Big Plan. Nobody's going to give you a billion dollars for something that's simple enough for one person to do. You may say, "but this problem doesn't need a billion dollars!", to which I say "your corporate ownership structure isn't complicated enough, you need to make sure that as much of the billion dollars sticks to your hands as possible after you fail". WeWork passim.
- bob33212 7y agoI interviewed at a well funded company for engineering manager. The interview centered around how they were going to build a 100 person team so the job would include lots of interviewing and hiring. I assume that some investor was told that "We are going to have 100 developers while our competitor has only 20" and the investor bought into that plan.
- deleted 7y ago[deleted]
- blowski 7y agoPlus managers who define their worth by easily measurable values like how many 0s on their p+l statement, and lines to them on the org-chart. Success of the project is much more subjective.
- Meai 7y agoI've started to see it that way too. Every place where there is source code, the complexity grows instantly - it's like a virus. The only firewall you can have for this is if you have it all in different projects that need to justify themselves on their own on all merits, financially, technically, etc.
- afarrell 7y ago> This is a no-brainer. It is NOT obvious to someone who hasn't thought about it for a while. Suppose someone is trying to persuade another person and just assumes that they already realize the costs of organisational complexity. There's a good chance they'll run into a wall and not get the message across. If you think realizing it is a no-brainer, then your 25 years of experience is showing.
- streetcat1 7y agoYou might be ignoring the productivity gains in software dev over the last 10 years. With open-source, languages like go/rust, excellent IDEs and basically free compute, the amount that a single developer can produce is 10x/20x more.
- ChrisMarshallNY 7y agoTell me about it. I write in Swift. I love it. I started with Machine Code (not ASM -Machine Code). Also, all those lovely system frameworks are wonderful. I used to use MacApp (Google it), and PowerPlant (Same). AppKit and UIKit knock them into a cocked hat. SwiftUI shows promise, but it may be a year or two before it can really match the standards.
- abraxas 7y agoThis is pure hogwash. All that productivity and more has been available for two decades with Java and C#. It's just that hipsters rejected it wholesale because those are their parents' programming languages.
- 0x445442 7y agoI agree with everything you have said. But the trends in "The Enterprise" are to eschew these ideas in favor of overly complex serverless architectures with poorly specified interfaces where any pluggable developer/resource can perform hit and runs on any component. What I've seen this lead to is the most brittle, bug ridden, low quality software in my career. And the irony is it's led to none of the perceived benefits of serverless architectures but has enhanced all the drawbacks of monolith architectures. My best guess is to why things have become this way is that middle management in "The Enterprised" reckoned "Agile" as an opportunity to commoditize software development.
- downerending 7y ago> Problem is, you need this "Big" stuff. It's crazy to do without it. Sometimes you do. But many times big stuff gets written for reasons other than need. One of the best wins in our industry is to recognize that the big stuff isn't needed and to never start the project in the first place.