11 ms·

Does the software industry learn?

rossdavidh 5y ago
I would also like to read a well-written analysis of COBOL, what it did right and what it did wrong. A problem the article doesn't mention, is that the percentage of still-programming developers who are young (<30 years) is higher than in other professions, for several reasons, so the lessons learned 15 years ago are often lost.
- agumonkey 5y ago
  The few I know is the language tried to be so easy to read by anybody that it made creating abstraction impossible. If the problem fits cobol, you get a nanoSQL like DSL to manage on disk records nicely. If it doesn't (controlling multiple sub systems, ad-hoc state, graphs) it becomes a monstrosity. Over the years IBM and others tried to fix that by making tooling or having nicer languages transpiling down to COBOL, which lead to ever bigger monstrosity. I was shown old code base blending manual code + generated code + manual patch of the two.. it was a kind of office prank to let new guys read that for a day.
  - SwoopsFromAbove 5y ago
    You see, this sounds really cool. So what kinds of problems fit COBOL? And why did "we" conclude from this that general purpose languages were the solution, rather than building many different languages for specific problems? Did "we" even conclude that, or is that just my impression, 50 years later?
    - agumonkey 5y ago
      IMO, society doesn't conclude, it reacts, mostly by imitation and deviation. Among long term cycles
    - marcus_holmes 5y ago
      From what I remember of the times, it was the PC that killed COBOL. COBOL was considered a "serious" language, for "serious" business problems running on "serious" hardware that lived in a server room. PC's were not "serious", and lived on people's desks, running Lotus123 and other such trivial tools. Then VB/Delphi/etc came along, and a generation of developers writing "little" applications directly for the desktop (of which I was one, and it was fun). You could consider these as specialist DSL's for user interactive applications. But that was not the general view. Writing an application in VB was several orders of magnitude cheaper than writing the equivalent in COBOL, and if you squinted hard enough from a non-technical point of view they did the same job. Non-COBOL developers started writing server applications that lived on a PC in the server room for a fraction of the cost of a COBOL application living on a mainframe. And that was that. Except, of course, for the large institutions (banks being the obvious example) who had invested millions in developing their core applications in COBOL running on mainframes. The cost of rewriting this for no reason (because the original still works) is prohibitive. Y2K gave them a shock, but also added to the sunk cost. They still run COBOL on mainframes, and still train up new developers in COBOL.
      - watt 5y ago
        > Writing an application in VB was several orders of magnitude cheaper Order of magnitude is big deal, can you be more specific, did you mean 2 or 3 orders of magnitude? Or what exactly did you mean. By this logic if writing an app in VB costs 20$ then COBOL equivalent cost 2000$ or 20 000$ ? Ok, 20$ seems a bit too cheap, let's see, if developing VB app cost 400$. So COBOL equivalent would have cost either 40 000$ or 400 000$ ? Is that what you mean?
        abernard1 5y ago
        > So COBOL equivalent would have cost either 40 000$ or 400 000$ ? Is that what you mean? If he didn't mean that, I will say, yes, that is correct. You could do things in VB in a day that teams of people in COBOL would take months to do. Many things in VB are basically impossible to do in COBOL. For some perspective, the reason why many Medicare reforms in the U.S. are not able to be implemented is because the government is unable to actually modify the software. The code that figures out how to bill Medicare is 50 years old, has 8 million lines of code and 1.5 million lines of assembly [1]. Another high profile failure was when CA couldn't furlough state employees because they couldn't figure out how to update the software (a feature, not a bug, for many). [1] https://www.programmableweb.com/news/how-usds-modernizing-medicares-50-year-old-payment-system/native-case-study/2018/11/13 https://www.programmableweb.com/news/how-usds-modernizing-me...
dboreham 5y ago
Nope.
mdoms 5y ago
Another story on the front page right now is "how I built a date picker". In 2022, people are still building date pickers. Imagine the progress our field could make were it not for the navel gazing, NIHing and pointless reworking.
- bluetomcat 5y ago
  > In 2022, people are still building date pickers. Automotive engineers in 2022 are still creating new designs of water pumps and alternators. The new design meets some new requirements for the particular application, be they thermal, spatial or cost-related. The date picker designed in 2022 is to be used with 2022 applications, IDEs, toolchains, etc.
- morsch 5y ago
  Hardly specific to software engineering, and maybe (but not certainly) that should tell us that there is something to navel-gazing and reworking of fundamentals and basics.
  - galaxyLogic 5y ago
    If creating furniture was physically as easy as writing software, I think many people would create their own furniture, just to create a chair that fits them exactly. Similarly in software there is not "one size that perfectly fits everybody". Therefore there is much need for customization. When you write software you are in effect not only creating the program. You are creating a system that consists of the computer, its program, and its user. The way the task is divided between what the human does and what the computer does can be done in very many ways. In some systems that results in a system where users need to do more but thus also have more options for controlling the system. Therefore there are so many different ways any "applications" can be written. No program is an island. It is something that must interact with its users. It is always designed to be a part of a bigger system consisting of itself and its users.
- choeger 5y ago
  Lookup YouTube on how many people built a chair, desk, or any other basic piece of furniture. In a craft you have to do the basics yourself sometimes.
galaxyLogic 5y ago
I think there's a myth that rewriting software is bad, or at least a symptom of badness, and "reusing" software is ideal. But writing software is really like creating plans: plans for what the machine should do. Of course there is much room for reusing old plans as components of your new plan. But still every plan must be about what is needed at the moment, not about reusing existing plan-components. In human communications we don't "reuse" old communications much do we? Well maybe we do a bit, using common utterances. But those are more like idioms of the language, not "subroutines".
- mettamage 5y ago
  I suppose if you squint, then rewriting software means: properly reading/understanding the old code. Sort of.
- 2muchcoffeeman 5y ago
  My feeling is that rewriting is used as blunt weapon when the developer lacks the inclination or skill to learn what was written before. I’m been guilty of this.
  - raverbashing 5y ago
    But even if you know what is written, good luck making that PHP 3 website keep up with times. Or that old C library written for old versions of OS/other libs.
    - islon 5y ago
      Agree, but you are giving a more extreme example. Most rewrites I saw was "wow, this Java code from the old team is a mess, let's rewrite it and, in a couple years, the new devs will think the same about our rewrite."
      - galaxyLogic 5y ago
        I think you're onto something there. It's hard to understand code. So maybe the best way to accomplish that is to rewrite it. That way you will have inhouse people who understand and can maintain the software your business is running on. Once those people are gone it's time for the new recruits to rewrite it again. It's a continuing process.
csande17 5y ago
Given the number of times we have failed to learn the lesson "downloading code from untrusted sources and running it is a bad idea" -- the log4j and NPM colors fiascos spring to mind -- I think it's fair to conclude that this industry is completely incapable of learning anything, ever.
- moring 5y ago
  I feel like the actual problem behind this is a useful definition of what "trusted" and "untrusted" mean that does not resolve to assigning blame for problems that have already happened.
  - csande17 5y ago
    I feel like "literally any URL supplied by anyone capable of visiting your website" and "some random guy from the Internet, with no connection to you or your company whatsoever, who was recently arrested for trying to burn his own house down" are both fairly obvious examples of sources from which you should not download and run random code without checking it first. But maybe that's the part that this industry is incapable of learning.
- capableweb 5y ago
  Ignoring the fact that basing ones opinion on an entire industry based on two "fiascos" seems drastic at best, who can we trust if we suddenly can't trust organizations like Apache? Do you trust the Linux Foundation? It's almost like the issue is not that code is available, but how people use the code that's available, and no one seemingly likes funding open source code.
  - mulmen 5y ago
    > and no one seemingly likes funding open source code. I’m not sure how this meme got started but it’s toxic. Why does free software need funding? Free software needs contributions. Big corporations make contributions by paying engineers. Everyone benefits in this ecosystem.
duped 5y ago
> As a result of how young the profession is, there are few universally accepted practices and standards. I wish the meme of "software engineering is young" would die. This profession is not younger than any other technical profession, in the modern sense. It was born out of prior practice and is still taught using the same techniques and philosophy. We are not special. Furthermore, modern engineering is a mid to late 20th century practice - across all domains - not just ECE. We inherited some practices of the first industrial revolution, but by and large the techniques and tradition of education of engineers is consistent across the disciplines and evolved out of the post-WW2 marketplace for new technical products. It would serve the author to remember this well coupled with the following : Engineering is the discipline of systematic problem solving. Computer science is the mathematical study of abstraction. Software development is the practice of applying computer science to the domain of engineering. This results in a force amplifier - much like a lever - but rather than creating torque what we're doing is amplifying the ability of creative problem solving to be applied and re-applied without re-engineering complexity. That abstraction and reuse propagates throughout entire industries in ways that are now invisible because they have become ubiquitous. "Does the software industry learn?" Well of course it does. We encode our learning of the problems we face everyday. The chips powering the computer I'm typing this on was designed with software that prototyped and ultimately yielded their design; packaged on circuit boards designed with software that automated hardware engineering checks to validate the design; encased in an enclosure designed with software that encompasses decades of 3D design and visualization technology; all fabricated using hardware and software tools with ambiguity resolved through digital communications that can be seen as a recursion of this paragraph. It's turtles all the way down. If you think we aren't learning everyday, propagating that learning, and preserving it for generations to come you do not understand the practices we undertake everyday. There is so much learned technique, practice, and theory encoded into the basis of our everyday lives it now seems so invisible you can blog about it.
- T-zex 5y ago
  Good point that software "industry" is not new, but I would argue that it is not an industry in a literal sense or at least it is a special industry. The special thing about software is that it is governed by the social aspects a lot. And because of this, software has poor standardization practices. Engineers in other sectors could prove their qualifications by showing certification diplomas, even education and years of experience would count. In software your qualification would be determined by some random dude and how well you fit his CS fetish of the month and that is it. Experience and education is only important while filtering CVs. If you designed a chip, an aircraft or skyscraper it is immediately impressive. If you built an FX trading platform it means almost nothing if you are a bit skeptical about pair programming. It also applies the other way round. If project uses microservice architecture, what exactly does that mean? The result would be a wildly different if you compare such projects at FANG and some investment bank. How software is tested and released will be different in any team. And how that affects the poor developer will vary too. What is a good software? We can have an opinion, but there is no formal definition. And because of this fragmentation and fluidity software as and industry is doing very different things. It will learn in some cases in others it will be incredibly ignorant.
grahamlee 5y ago
They do exist, but I do wish for more. We’re only a couple of decades away from software construction being a hundred years old (and a few from software _engineering_ being a century old) and I think it’d be wonderful to have some encyclopaedia-like resource that says “here’s everything we tried for the first hundred years, and why we do things this way at the end of it”. For my part, I try to contextualise the way I/my peers work now with the other things we tried over my career, and maybe the decade or two before that which contextualised _my_ junior experience. For example, I run a monthly magazine with a colleague, and in our devops issue we discuss how ideas in DevOps come from manufacturing, from Rapid Application Development, and other inspiration (https://deprogrammaticaipsum.com/play-it-again-sam/ https://deprogrammaticaipsum.com/play-it-again-sam/). I just recorded a podcast episode where I look at _what_ documentation was considered “comprehensive” at the time of the agile manifesto, and what docs are still useful despite the industry-wide tendency to eschew all prose (https://www.sicpers.info/podcast/episode-47-comprehensive-documentation/ https://www.sicpers.info/podcast/episode-47-comprehensive-do...).
MathMonkeyMan 5y ago
Physics progresses one funeral at a time.
fleddr 5y ago
We live in the age of shipping. Immediately. Not in the age of carefully considered best practices, architectural considerations, or sound engineering. Even top notch companies ship broken software, so it's not even about money. It's about speed. The problem is not with developers, it's in crappy foundational computing layers. NPM is somewhat the result of JavaScript itself being so bare bones. The cost of this community-based pseudo standard library is enormous: fragile, insecure, unstable. This doesn't mean we have to drop it, it means perhaps we do need a JavaScript standard library. Possibly mixed with NPM vendor packages that take responsibility of large clusters of functionality as a single dependency that is maintained over time. Almost all software now requires very frequent security updates and we even dispose physical products like smartphones because they no longer get security updates. Instead of accepting this reality, why is nobody working on a foundational software stack that isn't this damn insecure to begin with? Almost all websites on the web do not meet accessibility standards. And then we beat developers over the head in needing to learn proper accessibility best practices. Which doesn't work. Progress is near zero. So we might as well ask: why are our tools so primitive and bad, why do they not lead to accessible UX by default? We complain about performance, or the lack thereof. But the problem is mostly in the stack itself, crappy and slow abstraction layers. I could go on, but I hope you catch my drift. In development, you need to take the lowest common denominator. An average "bread programmer". We expect this person to personally dodge the many gaps in our crappy computing layers and it's far too easy to go wrong. This single individual is expected to be a top engineer, architect, security expert, performance expert, accessibility expert, and so on. And they also need to ship tomorrow. We basically expect 28 million software developers to all be super heroes. They're not, they keep making the same mistakes not because they suck, instead because we have computing layers that suck. No amount of "awareness" will fix that.
neuronic 5y ago
I think learning is subjective. The NodeJS world is learning a lot about backend development I guess. So, disclaimer, I am new to Typescript-NodeJS and the npm world but have a significant background in the JVM / Spring backend world with Kotlin and Java. I am also too young to be this cynic but I am honestly just baffled. To me the entire NodeJS backend movement seems like they are "Java sucks!" hipsters jumping into the trendy JavaScript ecosystem of the modern web era just to show how cool and progressive they are (???) - all while rediscovering and reinventing backend features that have been invented and in widespread and battle-hardened use for 2-3 decades now. Wow we are using NestJS, Jest, annotations, a dependency injection module and Typescript... congrats, you just rediscovered type systems, mocking, Maven and Spring Boot as they are very handy for backend development. All while bloating your fresh hip ecosystem to an even uglier Frankenstein monster than you can find anywhere in the Java enterprise, transpiling back and forth with 10,000 config files to make framework XYZ interact with each other. And then, JavaScript is running underneath with all its limitations and quirks that will reach feature parity with Kotlin with ES2048. But yea, I guess now you can send JSON natively between systems because it's first-class.
- gbuk2013 5y ago
  As someone with a significant background in Node.js and a bit of experience in Java I think Node.js is much better in that respect. Yes, we have things like NestJS, which I refuse to use precisely for the reasons that you point out: it looks like Java Spring and adds a lot of complexity where it is not needed. I am also painfully aware that TS can be abused to write Java-like code but that's on the developer. However, the abuse notwithstanding, the fact that JS is a simpler language protects it from Java's key problem: "Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle." My (limited) exposure to Java was quite painful because I had to deal with layers and layer of inheritance and abstractions that at some point made it very difficult to do what I needed to do. The language itself was fine except that it easily allowed for this sort of architecture. Also, the advantage of writing the frontend and backend in the same language should not be understated.
Tarucho 5y ago
It will never learn because there is no incentive to do so. Tech organizations value their managers in terms of their budget, headcount and being in fashion. As such the tech business model has always been about getting "good enough" results through a brute force process, possibly with the latest tech. Expertise/deep knowledge is only added/valued when the development gets stuck.
thrower123 5y ago
No. The burnout rate is sufficient that most people in the software industry have five or fewer years of experience, and very often it's not actually five years of cumulative experience, but five years of the same year of experience, served consecutively. This five year cycle turns up very often; that seems to be about the length of time that it takes for an idea to form, ride the hype wave, sink into the trough of disillusionment, and then be replaced by a new hot thing pushed by the next generation of fresh, idealistic programmers. Usually the new hot thing is the same basic idea as the hot thing one or two generations before the current hot thing, which always starts off bright-eyed and bushy-tailed before it runs into the wall of leaky abstractions and murderous corner-cases and the stark horror of being used in the real-world for real products. Then nearly all of the trade-offs and short-comings of the older iteration of the idea come back to haunt the new thing, and the gray-haired programmers over 35 who've seen this cycle come and go time after time have a chuckle.
lbriner 5y ago
I think the premise falls down with the example given about programming languages because in engineering everything is a trade-off. Cobol is not Go and would have made design choices based on not just Instituional Knowledge (or lack of it) but also because of the world at that time, the reality of hardware and software etc. Sure, GoLang could learn something like "don't make things nullable" but it would be easy to say, "yes I know that, but in our case, this creates a benefit because of X" Just like people who believe in TDD, DDD etc. they might all work, but that doesn't mean they are objectively the best way to do things in all circumstances. I think the better target is learning how to think and analyze more effectively so we know what we are trading off more clearly.
tpoacher 5y ago
“We learn one thing from history: that we learn nothing from history.” ― Georg Hegel
brettdeveffect 5y ago
Not to be too critical, but I feel like this article was a bit naive. Nothing works "optimally" and no-one knows the answers to lots of these questions. IMO its a great thing that people are out there trying new things and sometimes good practices stick around. We need the vanguard to be there so that we explore. "order for learning to most effectively improve our state of operation" Things don't move "most effectively" in anything. I guarantee that other engineering disciplines do not react to new tech / innovation in any sort of optimal way. People are driven to work in certain ways by all kinds of things, and mostly has to do with incentives and social reasons. "But I very rarely see articles looking back at past languages or technological fads and looking at current trends through that lens." There are historical surveys all over the place, even on this site. I don't want to attack this article too hard, and the motivation to ask whether or not we are are really learning from our experiences is good, I just feel like people are more flustered by the fact that there aren't magical best practices you can memorize and apply yet, and that most human knowledge comes from experience and intuition.
antishatter 5y ago
Does it forget?
CM30 5y ago
Part of me says "of course we learn, look at how many bad practices are at least significantly less common than they were in the olden days". It's quite rare you come across a decently run tech company or team that doesn't use version control, or makes changes live in production without a testing environment, or doesn't use automated tests at all, or doesn't have a deploy process at all beyond 'use FTP'. 10-20 years ago that wasn't really the case, and such practices were much rarer even in more tech savvy teams and organisations. Same with web development related stuff. People aren't using tables for layout anymore, nor are things like accessibility some sort of completely ignored concept that no one takes seriously. And people do actually use CSS for layout purposes rather than font tags and spacer images and whatever else the days of Geocities style web development had in store. So it's definitely advanced in many areas.
- mikewarot 5y ago
  In the 1980s, GIT didn't exist, and CVS was a complicated thing that Unix people used, so I didn't have "proper" version control. I did, however, have a stash of ZIP files of increasing sequence number on floppy disks. In the 1990s, We used FTP to send files to web servers because we didn't have WebDAV or any more secure protocols at the time. We used tables because that is what we had. We had fewer tools than today, it really doesn't amount to any learning, just better tool availability. Knowledge itself seems to have stayed about the same.
  - markmaglana 5y ago
    ...but aren't those tools a representation of the additional knowledge that the industry accumulated over time?
- jjav 5y ago
  > 10-20 years ago that wasn't really the case To state the obvious, 10 years ago was 2012. Absolutely all of the things you list were standard practice. 20 years ago was 2002, also same thing (no, git didn't invent version control). 30 years ago I was using version control at my first job, it was expected practice. As was having test coverage. None of these things are new. Deployment was different, granted. We didn't "deploy to production", we shipped a box of floppies to users.
commandlinefan 5y ago
Well, after 30 years of software development experience, I seem to be the only person I know who has learned that you have to be able to test everything outside of a production environment.
errantmind 5y ago
No, we mainly benefit from operating systems protecting us from bad developers (at the cost of performance). Take a look at all the LSPs on Linux ad an example.
mbrodersen 5y ago
The “industry” is a bunch of individuals who individually may or may not learn from the experience of others and themselves. And there are a lot more inexperienced developers than experienced. So on average I would say that yes the “industry” is learning but the average skill of developers might not be that high (yet).

I would also like to read a well-written analysis of COBOL, what it did right and what it did wrong. A problem the article doesn't mention, is that the percentage of still-programming developers who are young (<30 years) is higher than in other professions, for several reasons, so the lessons learned 15 years ago are often lost.

agumonkey 5y ago
The few I know is the language tried to be so easy to read by anybody that it made creating abstraction impossible. If the problem fits cobol, you get a nanoSQL like DSL to manage on disk records nicely. If it doesn't (controlling multiple sub systems, ad-hoc state, graphs) it becomes a monstrosity. Over the years IBM and others tried to fix that by making tooling or having nicer languages transpiling down to COBOL, which lead to ever bigger monstrosity. I was shown old code base blending manual code + generated code + manual patch of the two.. it was a kind of office prank to let new guys read that for a day.
- SwoopsFromAbove 5y ago
  You see, this sounds really cool. So what kinds of problems fit COBOL? And why did "we" conclude from this that general purpose languages were the solution, rather than building many different languages for specific problems? Did "we" even conclude that, or is that just my impression, 50 years later?
  - agumonkey 5y ago
    IMO, society doesn't conclude, it reacts, mostly by imitation and deviation. Among long term cycles
  - marcus_holmes 5y ago
    From what I remember of the times, it was the PC that killed COBOL. COBOL was considered a "serious" language, for "serious" business problems running on "serious" hardware that lived in a server room. PC's were not "serious", and lived on people's desks, running Lotus123 and other such trivial tools. Then VB/Delphi/etc came along, and a generation of developers writing "little" applications directly for the desktop (of which I was one, and it was fun). You could consider these as specialist DSL's for user interactive applications. But that was not the general view. Writing an application in VB was several orders of magnitude cheaper than writing the equivalent in COBOL, and if you squinted hard enough from a non-technical point of view they did the same job. Non-COBOL developers started writing server applications that lived on a PC in the server room for a fraction of the cost of a COBOL application living on a mainframe. And that was that. Except, of course, for the large institutions (banks being the obvious example) who had invested millions in developing their core applications in COBOL running on mainframes. The cost of rewriting this for no reason (because the original still works) is prohibitive. Y2K gave them a shock, but also added to the sunk cost. They still run COBOL on mainframes, and still train up new developers in COBOL.
    - watt 5y ago
      > Writing an application in VB was several orders of magnitude cheaper Order of magnitude is big deal, can you be more specific, did you mean 2 or 3 orders of magnitude? Or what exactly did you mean. By this logic if writing an app in VB costs 20$ then COBOL equivalent cost 2000$ or 20 000$ ? Ok, 20$ seems a bit too cheap, let's see, if developing VB app cost 400$. So COBOL equivalent would have cost either 40 000$ or 400 000$ ? Is that what you mean?
      - abernard1 5y ago
        > So COBOL equivalent would have cost either 40 000$ or 400 000$ ? Is that what you mean? If he didn't mean that, I will say, yes, that is correct. You could do things in VB in a day that teams of people in COBOL would take months to do. Many things in VB are basically impossible to do in COBOL. For some perspective, the reason why many Medicare reforms in the U.S. are not able to be implemented is because the government is unable to actually modify the software. The code that figures out how to bill Medicare is 50 years old, has 8 million lines of code and 1.5 million lines of assembly [1]. Another high profile failure was when CA couldn't furlough state employees because they couldn't figure out how to update the software (a feature, not a bug, for many). [1] https://www.programmableweb.com/news/how-usds-modernizing-medicares-50-year-old-payment-system/native-case-study/2018/11/13 https://www.programmableweb.com/news/how-usds-modernizing-me...

dboreham 5y ago

Nope.

mdoms 5y ago

Another story on the front page right now is "how I built a date picker". In 2022, people are still building date pickers. Imagine the progress our field could make were it not for the navel gazing, NIHing and pointless reworking.

bluetomcat 5y ago
> In 2022, people are still building date pickers. Automotive engineers in 2022 are still creating new designs of water pumps and alternators. The new design meets some new requirements for the particular application, be they thermal, spatial or cost-related. The date picker designed in 2022 is to be used with 2022 applications, IDEs, toolchains, etc.
morsch 5y ago
Hardly specific to software engineering, and maybe (but not certainly) that should tell us that there is something to navel-gazing and reworking of fundamentals and basics.
- galaxyLogic 5y ago
  If creating furniture was physically as easy as writing software, I think many people would create their own furniture, just to create a chair that fits them exactly. Similarly in software there is not "one size that perfectly fits everybody". Therefore there is much need for customization. When you write software you are in effect not only creating the program. You are creating a system that consists of the computer, its program, and its user. The way the task is divided between what the human does and what the computer does can be done in very many ways. In some systems that results in a system where users need to do more but thus also have more options for controlling the system. Therefore there are so many different ways any "applications" can be written. No program is an island. It is something that must interact with its users. It is always designed to be a part of a bigger system consisting of itself and its users.
choeger 5y ago
Lookup YouTube on how many people built a chair, desk, or any other basic piece of furniture. In a craft you have to do the basics yourself sometimes.

galaxyLogic 5y ago

I think there's a myth that rewriting software is bad, or at least a symptom of badness, and "reusing" software is ideal. But writing software is really like creating plans: plans for what the machine should do. Of course there is much room for reusing old plans as components of your new plan. But still every plan must be about what is needed at the moment, not about reusing existing plan-components. In human communications we don't "reuse" old communications much do we? Well maybe we do a bit, using common utterances. But those are more like idioms of the language, not "subroutines".

mettamage 5y ago
I suppose if you squint, then rewriting software means: properly reading/understanding the old code. Sort of.
2muchcoffeeman 5y ago
My feeling is that rewriting is used as blunt weapon when the developer lacks the inclination or skill to learn what was written before. I’m been guilty of this.
- raverbashing 5y ago
  But even if you know what is written, good luck making that PHP 3 website keep up with times. Or that old C library written for old versions of OS/other libs.
  - islon 5y ago
    Agree, but you are giving a more extreme example. Most rewrites I saw was "wow, this Java code from the old team is a mess, let's rewrite it and, in a couple years, the new devs will think the same about our rewrite."
    - galaxyLogic 5y ago
      I think you're onto something there. It's hard to understand code. So maybe the best way to accomplish that is to rewrite it. That way you will have inhouse people who understand and can maintain the software your business is running on. Once those people are gone it's time for the new recruits to rewrite it again. It's a continuing process.

csande17 5y ago

Given the number of times we have failed to learn the lesson "downloading code from untrusted sources and running it is a bad idea" -- the log4j and NPM colors fiascos spring to mind -- I think it's fair to conclude that this industry is completely incapable of learning anything, ever.

moring 5y ago
I feel like the actual problem behind this is a useful definition of what "trusted" and "untrusted" mean that does not resolve to assigning blame for problems that have already happened.
- csande17 5y ago
  I feel like "literally any URL supplied by anyone capable of visiting your website" and "some random guy from the Internet, with no connection to you or your company whatsoever, who was recently arrested for trying to burn his own house down" are both fairly obvious examples of sources from which you should not download and run random code without checking it first. But maybe that's the part that this industry is incapable of learning.
capableweb 5y ago
Ignoring the fact that basing ones opinion on an entire industry based on two "fiascos" seems drastic at best, who can we trust if we suddenly can't trust organizations like Apache? Do you trust the Linux Foundation? It's almost like the issue is not that code is available, but how people use the code that's available, and no one seemingly likes funding open source code.
- mulmen 5y ago
  > and no one seemingly likes funding open source code. I’m not sure how this meme got started but it’s toxic. Why does free software need funding? Free software needs contributions. Big corporations make contributions by paying engineers. Everyone benefits in this ecosystem.

duped 5y ago

> As a result of how young the profession is, there are few universally accepted practices and standards. I wish the meme of "software engineering is young" would die. This profession is not younger than any other technical profession, in the modern sense. It was born out of prior practice and is still taught using the same techniques and philosophy. We are not special. Furthermore, modern engineering is a mid to late 20th century practice - across all domains - not just ECE. We inherited some practices of the first industrial revolution, but by and large the techniques and tradition of education of engineers is consistent across the disciplines and evolved out of the post-WW2 marketplace for new technical products. It would serve the author to remember this well coupled with the following : Engineering is the discipline of systematic problem solving. Computer science is the mathematical study of abstraction. Software development is the practice of applying computer science to the domain of engineering. This results in a force amplifier - much like a lever - but rather than creating torque what we're doing is amplifying the ability of creative problem solving to be applied and re-applied without re-engineering complexity. That abstraction and reuse propagates throughout entire industries in ways that are now invisible because they have become ubiquitous. "Does the software industry learn?" Well of course it does. We encode our learning of the problems we face everyday. The chips powering the computer I'm typing this on was designed with software that prototyped and ultimately yielded their design; packaged on circuit boards designed with software that automated hardware engineering checks to validate the design; encased in an enclosure designed with software that encompasses decades of 3D design and visualization technology; all fabricated using hardware and software tools with ambiguity resolved through digital communications that can be seen as a recursion of this paragraph. It's turtles all the way down. If you think we aren't learning everyday, propagating that learning, and preserving it for generations to come you do not understand the practices we undertake everyday. There is so much learned technique, practice, and theory encoded into the basis of our everyday lives it now seems so invisible you can blog about it.

T-zex 5y ago
Good point that software "industry" is not new, but I would argue that it is not an industry in a literal sense or at least it is a special industry. The special thing about software is that it is governed by the social aspects a lot. And because of this, software has poor standardization practices. Engineers in other sectors could prove their qualifications by showing certification diplomas, even education and years of experience would count. In software your qualification would be determined by some random dude and how well you fit his CS fetish of the month and that is it. Experience and education is only important while filtering CVs. If you designed a chip, an aircraft or skyscraper it is immediately impressive. If you built an FX trading platform it means almost nothing if you are a bit skeptical about pair programming. It also applies the other way round. If project uses microservice architecture, what exactly does that mean? The result would be a wildly different if you compare such projects at FANG and some investment bank. How software is tested and released will be different in any team. And how that affects the poor developer will vary too. What is a good software? We can have an opinion, but there is no formal definition. And because of this fragmentation and fluidity software as and industry is doing very different things. It will learn in some cases in others it will be incredibly ignorant.

grahamlee 5y ago

They do exist, but I do wish for more. We’re only a couple of decades away from software construction being a hundred years old (and a few from software _engineering_ being a century old) and I think it’d be wonderful to have some encyclopaedia-like resource that says “here’s everything we tried for the first hundred years, and why we do things this way at the end of it”. For my part, I try to contextualise the way I/my peers work now with the other things we tried over my career, and maybe the decade or two before that which contextualised _my_ junior experience. For example, I run a monthly magazine with a colleague, and in our devops issue we discuss how ideas in DevOps come from manufacturing, from Rapid Application Development, and other inspiration (https://deprogrammaticaipsum.com/play-it-again-sam/ https://deprogrammaticaipsum.com/play-it-again-sam/). I just recorded a podcast episode where I look at _what_ documentation was considered “comprehensive” at the time of the agile manifesto, and what docs are still useful despite the industry-wide tendency to eschew all prose (https://www.sicpers.info/podcast/episode-47-comprehensive-documentation/ https://www.sicpers.info/podcast/episode-47-comprehensive-do...).

MathMonkeyMan 5y ago

Physics progresses one funeral at a time.

fleddr 5y ago

We live in the age of shipping. Immediately. Not in the age of carefully considered best practices, architectural considerations, or sound engineering. Even top notch companies ship broken software, so it's not even about money. It's about speed. The problem is not with developers, it's in crappy foundational computing layers. NPM is somewhat the result of JavaScript itself being so bare bones. The cost of this community-based pseudo standard library is enormous: fragile, insecure, unstable. This doesn't mean we have to drop it, it means perhaps we do need a JavaScript standard library. Possibly mixed with NPM vendor packages that take responsibility of large clusters of functionality as a single dependency that is maintained over time. Almost all software now requires very frequent security updates and we even dispose physical products like smartphones because they no longer get security updates. Instead of accepting this reality, why is nobody working on a foundational software stack that isn't this damn insecure to begin with? Almost all websites on the web do not meet accessibility standards. And then we beat developers over the head in needing to learn proper accessibility best practices. Which doesn't work. Progress is near zero. So we might as well ask: why are our tools so primitive and bad, why do they not lead to accessible UX by default? We complain about performance, or the lack thereof. But the problem is mostly in the stack itself, crappy and slow abstraction layers. I could go on, but I hope you catch my drift. In development, you need to take the lowest common denominator. An average "bread programmer". We expect this person to personally dodge the many gaps in our crappy computing layers and it's far too easy to go wrong. This single individual is expected to be a top engineer, architect, security expert, performance expert, accessibility expert, and so on. And they also need to ship tomorrow. We basically expect 28 million software developers to all be super heroes. They're not, they keep making the same mistakes not because they suck, instead because we have computing layers that suck. No amount of "awareness" will fix that.

neuronic 5y ago

I think learning is subjective. The NodeJS world is learning a lot about backend development I guess. So, disclaimer, I am new to Typescript-NodeJS and the npm world but have a significant background in the JVM / Spring backend world with Kotlin and Java. I am also too young to be this cynic but I am honestly just baffled. To me the entire NodeJS backend movement seems like they are "Java sucks!" hipsters jumping into the trendy JavaScript ecosystem of the modern web era just to show how cool and progressive they are (???) - all while rediscovering and reinventing backend features that have been invented and in widespread and battle-hardened use for 2-3 decades now. Wow we are using NestJS, Jest, annotations, a dependency injection module and Typescript... congrats, you just rediscovered type systems, mocking, Maven and Spring Boot as they are very handy for backend development. All while bloating your fresh hip ecosystem to an even uglier Frankenstein monster than you can find anywhere in the Java enterprise, transpiling back and forth with 10,000 config files to make framework XYZ interact with each other. And then, JavaScript is running underneath with all its limitations and quirks that will reach feature parity with Kotlin with ES2048. But yea, I guess now you can send JSON natively between systems because it's first-class.

gbuk2013 5y ago
As someone with a significant background in Node.js and a bit of experience in Java I think Node.js is much better in that respect. Yes, we have things like NestJS, which I refuse to use precisely for the reasons that you point out: it looks like Java Spring and adds a lot of complexity where it is not needed. I am also painfully aware that TS can be abused to write Java-like code but that's on the developer. However, the abuse notwithstanding, the fact that JS is a simpler language protects it from Java's key problem: "Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle." My (limited) exposure to Java was quite painful because I had to deal with layers and layer of inheritance and abstractions that at some point made it very difficult to do what I needed to do. The language itself was fine except that it easily allowed for this sort of architecture. Also, the advantage of writing the frontend and backend in the same language should not be understated.

Tarucho 5y ago

It will never learn because there is no incentive to do so. Tech organizations value their managers in terms of their budget, headcount and being in fashion. As such the tech business model has always been about getting "good enough" results through a brute force process, possibly with the latest tech. Expertise/deep knowledge is only added/valued when the development gets stuck.

thrower123 5y ago

No. The burnout rate is sufficient that most people in the software industry have five or fewer years of experience, and very often it's not actually five years of cumulative experience, but five years of the same year of experience, served consecutively. This five year cycle turns up very often; that seems to be about the length of time that it takes for an idea to form, ride the hype wave, sink into the trough of disillusionment, and then be replaced by a new hot thing pushed by the next generation of fresh, idealistic programmers. Usually the new hot thing is the same basic idea as the hot thing one or two generations before the current hot thing, which always starts off bright-eyed and bushy-tailed before it runs into the wall of leaky abstractions and murderous corner-cases and the stark horror of being used in the real-world for real products. Then nearly all of the trade-offs and short-comings of the older iteration of the idea come back to haunt the new thing, and the gray-haired programmers over 35 who've seen this cycle come and go time after time have a chuckle.

lbriner 5y ago

I think the premise falls down with the example given about programming languages because in engineering everything is a trade-off. Cobol is not Go and would have made design choices based on not just Instituional Knowledge (or lack of it) but also because of the world at that time, the reality of hardware and software etc. Sure, GoLang could learn something like "don't make things nullable" but it would be easy to say, "yes I know that, but in our case, this creates a benefit because of X" Just like people who believe in TDD, DDD etc. they might all work, but that doesn't mean they are objectively the best way to do things in all circumstances. I think the better target is learning how to think and analyze more effectively so we know what we are trading off more clearly.

tpoacher 5y ago

“We learn one thing from history: that we learn nothing from history.” ― Georg Hegel

brettdeveffect 5y ago

Not to be too critical, but I feel like this article was a bit naive. Nothing works "optimally" and no-one knows the answers to lots of these questions. IMO its a great thing that people are out there trying new things and sometimes good practices stick around. We need the vanguard to be there so that we explore. "order for learning to most effectively improve our state of operation" Things don't move "most effectively" in anything. I guarantee that other engineering disciplines do not react to new tech / innovation in any sort of optimal way. People are driven to work in certain ways by all kinds of things, and mostly has to do with incentives and social reasons. "But I very rarely see articles looking back at past languages or technological fads and looking at current trends through that lens." There are historical surveys all over the place, even on this site. I don't want to attack this article too hard, and the motivation to ask whether or not we are are really learning from our experiences is good, I just feel like people are more flustered by the fact that there aren't magical best practices you can memorize and apply yet, and that most human knowledge comes from experience and intuition.

antishatter 5y ago

Does it forget?

CM30 5y ago

Part of me says "of course we learn, look at how many bad practices are at least significantly less common than they were in the olden days". It's quite rare you come across a decently run tech company or team that doesn't use version control, or makes changes live in production without a testing environment, or doesn't use automated tests at all, or doesn't have a deploy process at all beyond 'use FTP'. 10-20 years ago that wasn't really the case, and such practices were much rarer even in more tech savvy teams and organisations. Same with web development related stuff. People aren't using tables for layout anymore, nor are things like accessibility some sort of completely ignored concept that no one takes seriously. And people do actually use CSS for layout purposes rather than font tags and spacer images and whatever else the days of Geocities style web development had in store. So it's definitely advanced in many areas.

mikewarot 5y ago
In the 1980s, GIT didn't exist, and CVS was a complicated thing that Unix people used, so I didn't have "proper" version control. I did, however, have a stash of ZIP files of increasing sequence number on floppy disks. In the 1990s, We used FTP to send files to web servers because we didn't have WebDAV or any more secure protocols at the time. We used tables because that is what we had. We had fewer tools than today, it really doesn't amount to any learning, just better tool availability. Knowledge itself seems to have stayed about the same.
- markmaglana 5y ago
  ...but aren't those tools a representation of the additional knowledge that the industry accumulated over time?
jjav 5y ago
> 10-20 years ago that wasn't really the case To state the obvious, 10 years ago was 2012. Absolutely all of the things you list were standard practice. 20 years ago was 2002, also same thing (no, git didn't invent version control). 30 years ago I was using version control at my first job, it was expected practice. As was having test coverage. None of these things are new. Deployment was different, granted. We didn't "deploy to production", we shipped a box of floppies to users.

commandlinefan 5y ago

Well, after 30 years of software development experience, I seem to be the only person I know who has learned that you have to be able to test everything outside of a production environment.

errantmind 5y ago

No, we mainly benefit from operating systems protecting us from bad developers (at the cost of performance). Take a look at all the LSPs on Linux ad an example.

mbrodersen 5y ago

The “industry” is a bunch of individuals who individually may or may not learn from the experience of others and themselves. And there are a lot more inexperienced developers than experienced. So on average I would say that yes the “industry” is learning but the average skill of developers might not be that high (yet).