7 ms·
In my experience, the overwhelmingly most difficult thing about programming is writing code that makes sense, even after it has gone through a couple rounds of
by mbrock 11y ago
In my experience, the overwhelmingly most difficult thing about programming is writing code that makes sense, even after it has gone through a couple rounds of requirements changes and bug fixes.
Any concrete coding task can be dealt with straightforwardly enough, but projects as a whole tend to deteriorate by arbitrary fixes, maintenance patches, and failure to uphold a conceptual schema.
The idea of composability is a good approximation to what it means for a program to make sense. If any part of the program can be understood as a sensible composition of parts, then the program probably makes sense.
But so many parts of real world code bases seem to end up incoherent. You look at a function and instead of seeing code that obviously, say, maps the render function over the list of widgets, you see three obscure conditionals, an apologetic comment, a bug tracker issue number, two boolean parameters named "force" and "dontRenderLast" (for some reason), and so on.
This is why I think the ideas from "domain-driven design" are so important, including the notion of a ubiquitous domain vocabulary. Also the FP idea of denotational semantics. And that well-known benefit of unit test driven design.
There's not enough recognition and clear understanding of this problem, in my opinion. We all know the problem, but somehow we don't talk about it enough, and don't acknowledge it as an enormous problem for development speed, programmer happiness, agility, and so on.
- turaw 11y agoDo you think that sort of a problem (incremental changes eroding an initially sound structure) could be fixed with better refactoring tools? It seems like the issue you're describing is that logic is leaking from some parts of the codebase to others, which might be fixable with sufficiently powerful refactorings.
- Roboprog 11y agoYes. But it's still a lot of work to identify what to click on in an overly complicated function / class and then to select "Transmogrification # 17" on it/them. Getting to a deeper understanding of what the old code is doing takes a lot of work. When we get to that point, refactoring to make what we learned apparent is a good idea. Assuming there is a way to test the changes. (Now I get to spend 2 days making "mocks" in environment X -- learning the mockup tool, and determining the content appropriate for the use cases) It's a good point you bring up, but it's not a freebie. I am in favor of doing what you said, when you can.
- mbrock 11y agoYeah, I think the sheer manual friction of moving things around is a big impediment to change. But also, I think there has to be some conceptual basis for us to be able to think about factoring in a clear way, and that kind of intellectual labor is hard on another level. Eric Evans' book on domain-driven design has some examples of programming tasks that generate confusing and unclear code, until the coder together with the "domain expert" reaches some kind of epiphany—like "ohh, if we think of these three things as being parts of the same kind of thing, then this part of the program makes more sense." A program is a kind of philosophical theory about a certain topic, and if the theory isn't very clear, then you'll end up with a lot of special cases and obscureness even if you have unit tests and refactoring tools. They say naming is the hardest thing in programming and that becomes more true when you start to introduce any level of abstraction. "Design patterns" were an attempt to create more general small theoretical pieces, above the syntactic level. In the FP world, a similar vision is that algebraic and categorical abstractions can provide some of those pieces. We still haven't seen much of what algebraic abstractions can do for ordinary programming... But these abstract pieces, I think, are pretty crucial if we want programs to be understandable as anything but arbitrary collections of procedures and types. So yes to refactoring tools, and yes to more discussion about design patterns—it didn't end with the Gang of Four! read Christopher Alexander and Richard P. Gabriel!—and yes to more inspiration from mathematics and engineering and philosophy and all kinds of stuff, to provide inspiration for factoring—yes to study of logic and language and ... I'm getting carried away.
- amatic 11y agoVery smart comment. I've had a long brake from programming and all this functional and lisp stuff is new, and it is not always clear what is the benefit. But is great to see a diversity and experimentation.
- johnmaguire2013 11y agoLisp is actually the second oldest programming language (behind Fortran.) 1. https://en.wikipedia.org/wiki/Lisp_%28programming_language%29 https://en.wikipedia.org/wiki/Lisp_%28programming_language%2...
- perlgeek 11y ago> Do you think that sort of a problem (incremental changes eroding an initially sound structure) could be fixed with better refactoring tools? Only if those refactoring tools could refactor the domain itself (and the environment), not just the code :-) Often these special cases aren't intrinsic to the code, but caused by inconsistent business rules, or weird issues with the environment (some browsers not sending some events, workarounds for compiler bugs on ancient platforms that must still be supported, socket closing might or might not cause a flush depending on whether you're on Windows or *nix, ...)
- dasmoth 11y agoPossibly, but if so I suspect it'll be a big change from the tooling we've got at the moment. And there's always a risk that tooling which makes some changes (rename a method...) trivial will make substantive changes seem even harder (because, comparatively, they are). Somewhat the same arguments apply to approaches which put a lot of emphasis on small-scale/unit testing -- they offer reassurance when doing small-scale refactoring at the expense of making substantive refactoring harder.
- Consultant32452 11y agoIn my experience the only thing that prevents that phenomenon is unrelenting stubbornness from the development staff. It takes a tremendous amount of thought and care with each change to make sure it doesn't happen. Also in my experience, there is no "fixing" it once things have gone down hill. It's theoretically possible, but I've never personally seen it happen. Whereas I have seen fairly large code bases remain coherent after years of significant changes. That dev team was like a bunch of nazis as far as code quality was concerned. It was the best team I've ever worked with.
- allengeorge 11y agoOnly in the most trivial cases. Most systems I've worked on have some overriding architecture and assumptions. The worst quality erosions happen when changes are shoehorned into the code without a complete understanding of those factors: it's likely there'll be semantic mismatch between the new feature and existing components, unexpected bugs, and harder to modify code (because the overall architecture is no longer coherent). You can mitigate the disintegration by having a maintainer with a strong understanding of the architecture and the problem(s) being solved by the original system. Talking with them can give you a roadmap on how to modify it properly, and if it's even possible to do so while maintaining architectural and design coherence. Note: my comment is an opinion based on personal experience. I'm not trying to play it off as fact.
- heleph 11y agoI think it's difficult because no one particular change is a problem and in isolation they are mostly completely reasonable. It's the accumulation of these changes that cause the code to rot. There are a whole heap of "human nature" problems that are the same like getting into debt, being over weight, protecting the environment. The short term pay-off is fairly big, the contribution to the long term problem is very small. Also the person who is often responsible for making the decision may not be technical, so finds it difficult to imagine the eventual impact. Plus in the fast moving field of software development, they'll be unlikely to even be around when it happens.
- nickpeterson 11y agoOne thing I really liked about the DDD idea of ubiquitous language was that you reduce the amount of artificial abstractions between domain experts and the code, and it forces developers to be able to converse better with the business. I also think we should be doing a much better job documenting the intent of an entire system. New features get added by working them into the narrative of the 'Document of Intent', rather than becoming some set of requirements in an issue tracker. Otherwise we have difficulty understanding 'why' a system works the way it does, and possibly, if that way is actually incorrect. Many companies I've worked at, the system 'becomes' the rule. Even domain experts will ask what the system currently does when pressed for how something should happen. I think at a certain scale, it's just hard to keep business process in a single person's head.
- noir_lord 11y agoSomething I've considered on and off is that our current way of writing code is insufficient, I'd really like to have an editor/IDE that had "layers" so that I could attach code related comments in one layer and intent/documentation in another layer - throw in the ability to hide/show layers at will and you'd have a much more flexible way of documenting code than we currently have.
- sklogic 11y agoIt was possible do it with literate programming for ages. Combine it with emacs, mmm-mode and, say, auctex - and you'll get your IDE.
- rapind 11y agoI used litcoffee* for a while, and it really makes you think about and code for a future reader / maintainer, which is pretty great. It really slows you down though (good thing?) when trying to dump your ideas into code, especially if you end up writing exploratory code that you later delete (I do this a lot). I think literate code could be really great however for refactors. Once you've figured out the correct implementation and have the time & hopefully budget to revisit your fast code. Maybe it won't feel like the chore of straight up documentation, and more like the enjoyment of polishing your art. * http://coffeescript.org/#literate http://coffeescript.org/#literate
- davidjnelson 11y agoTotally agree. It seems like the driver is the tendency for businesses to think in the short term. It seems like building only the things that _really_ matter would be a solution. Then time would be more easily allocated to test driven design, composable architecture, state management, and data modeling. The question then becomes: "what truly matters to build". That's a whole other discussion.
- jacquesm 11y agoComplexity creep is something that should be fought tooth and nail where ever you find it. It's the biggest enemy of keeping your code maintainable. As soon as you feel that you're losing track of what is going on you need to step back and re-think your approach and re-factor. If you don't then in the long term you'll lose control completely. It's not a matter of 'if' but 'when' and by the time 'when' rolls around you will end up wishing you had taken care of the problem when it was still tractable. Short term thinking on long term projects is not an option.
- jcoffland 11y agoThis is absolutely correct. Maintaining control of your software means maintaining a good understanding of it which requires relentless refactoring. Refactoring without guilt. Too many programmers think they are wasting time when they refractor large parts of their program but this is always necessary.
- derefr 11y ago> Projects as a whole tend to deteriorate by arbitrary fixes, maintenance patches, and failure to uphold a conceptual schema. Personally, I think this isn't inherent, but is just an effect of how all the current programming paradigms (procedural, functional, even logic) tend to compose features together by simply sticking more code into each node of the function graph. So code that was originally a single stream of simple definitions becomes a complex mess where each line could have come from a consideration a week ago or a decade ago, and you can't "follow the train of thought" that led to the current state any given function is in. I think we need to think very differently. A truly aspect-oriented programming language, like Inform7 but not limited to the domain of Interactive Fiction, could allow for "literate programming" that treats a codebase as a linear narrative of feature additions, rather than a mutable graph representing current behavior. You'd actually be able to read a codebase from "beginning to end" and get a sense of what thoughts went into constructing it, in order. Just reading git commit history won't get you there. Commits aren't the right granularity, for one thing—a log of merged PRs might work better. More importantly, git commits to a codebase are a mix of feature-commits and bug-fixing commits, which horribly muddies your vision of how each feature is defined. What you really want is for features to each be files, and bug-fixes to be commits to those files, such that you can read "the up-to-date edition" of each of the features. Do note that we already have this in one particular place. This is how database migrations work. We just need code migrations! :)
- jschwartzi 11y agoDeveloping a linear narrative is a good part of why revision control systems and issue trackers exist. You use them to explain the evolution of the code base.
- mbrock 11y agoI'd love to see some non-game software written in Inform 7. Like, a web framework. Seriously!
- greenyoda 11y ago"that treats a codebase as a linear narrative of feature additions, rather than a mutable graph representing current behavior. You'd actually be able to read a codebase from "beginning to end" and get a sense of what thoughts went into constructing it, in order." That might be a good way of understanding the evolution of the system if you're already familiar with what the system does. But if you were a newly hired developer trying to understand the system from the ground up, wouldn't you want the features grouped together by logical function (systems and subsystems) rather than by their order of addition?
- JustSomeNobody 11y agoWhat I've seen kill code quality is a) management putting out fires and b) cocky programmers who boast they can do X in a few hours. Ok, sure, any of us can but why don't you take longer and properly refactor the code. Problem is, they've already opened their mouth so now management expects it in a few hours.
- rorykoehler 11y agoBusiness people care about instant revenue generation therefore the fastest programmer is the highest value one in their eyes. Maintainability and technical debt is an abstract and far off cost through this lens. Agile methodology was developed in response to this but in my experience the move fast and break things mindset it brings usually translates into move fast, break things and then put the broken things onto a ever growing bug database where only the most critical bugs ever get fixed.
- jcoffland 11y agoIf you're afraid you're going to break things then you need to refractor. Fear that changes will break things is a sign of code fragility.
- rorykoehler 11y agoI agree but there is little you can do about the code of your peers especially when they have seniority and you're not doing the code reviews. It's a cultural issue and I'm sure it's not like that everywhere but it is definitely prevalent. Everyone talks a good game about code quality but in practice moving fast is valued more by the money men.
- deleted 11y ago[deleted]
- golergka 11y agoYou either have a very bad luck with "business people" or talking about a stereotype that is about as close to the truth as similar stereotypes about programmers are. Business people are very familiar with terms like "infrastructure", "extensibility", "maintenance" and the like. They're not idiots. May be you should try explaining why refactoring and good architecture is good in business terms instead of programming?
- deleted 11y ago[deleted]
- invalidOrTaken 11y ago>structured programming visual, which is misguided. Can you expand on this? What is "structured programming"? Why is it misguided to make it visual? I am working on a tool that could be described as 2D functional programming, so these are not idle questions.
- erichocean 11y agoHistorically, structured programming was developed to eliminate the need for the GOTO statement and gave us the conventional control structures present in most PLs today: for-loops, while-loops, if-then-else statements, switch-case statements, etc. I think these structures are inappropriate in 2D because they assume a linear, 1D context for understanding.
- invalidOrTaken 11y agoThis seems worth remembering: >Structured programming gave us the conventional control structures present in most PLs: for-loops, while-loops, if-then-else statements, switch-case statements, etc. None of them are appropriate in 2D (but are very appropriate in a 1D program, usually written in text). Historically, structured programming was developed to enable the elimination of the GOTO statement. However I don't understand how conventional control structures are necessarily inappropriate for 2D. It seems to me that the 1D nature of text is used to map to execution point; even if that point jumps around a lot, it starts at the beginning, moves down by one line unless told otherwise, and ends at the bottom. In my tool, execution order is determined by dependencies, which is fine because there's no mutation (not by code anyway) so order doesn't really matter. So some constructs---`for`, `while` and `until` for instance---do become obsolete/meaningless. If and switch, however, remain very relevant. For that matter, switch draws its name from real-life switches in real-life machines, which run on ultra-concurrent physics. But we still find a use for them. Have I mis-stated or misunderstood you? As for the 2D similarity of 1D languages, that certainly sounds interesting. By "presenting them in 2D," do you mean something like Blockly? Blockly kind of leaves me cold because it seems like normal code, plus extreme syntax help. I mean, cool and stuff, but we can dream bigger, you know? Anyway, do you mean that imperative, functional, and concatenative all have the same basic made-from-text look when put into something like Blockly, or is there a deeper point I'm missing? Thanks a ton, btw. I'm always interested to hear what people think on this subject.
- hyperpallium 11y agoAlso, just normal efficiency and error handling tend to obscure the basic idea of code. Keeping a commented-out copy of the original can document the basic idea. Or, it's epicycles within epicycles until a paradigm shift to a simpler theory. Though most real-world programming goals are not intrinsically simple, but a mish-mash of mess.