7 ms·

I don't disagree with you and your examples are definitely over-engineering / busy work. In my experience a lot of it is driven by the desire for young engineer

by bko 5y ago

I don't disagree with you and your examples are definitely over-engineering / busy work. In my experience a lot of it is driven by the desire for young engineers to learn a new language. If someone paid me to move something to Rust, I would do it. I heard good things about Rust and I would love to get paid to learn it. But has being a software engineer become easier or harder over the last 30, 20, 10, 5 years? I wasn't an engineer for that long but my impression is that programming today is a lot easier. Dev tools, compilers and linters are very good. There's also a lot more community documentation on stack overflow. Some of the complexity is hidden from the developer, which is good and bad. It can bite you in the ass later, but in 95% of cases its a good trade off in my experience. For instance, my preferred stack is Serverless on AWS. I can set up a single config and have cloud storage, an api, a database, logging, auth all permissioned and in a file I can check in. And with a generous free tier, it's pretty much free. I'll admit if something goes wrong its not fun to debug, but it's remarkably fast and simple for me to spin up a CRUD api.

robertlagrant 5y ago
Some things are easier, but then we've made it so quick we become the lone typist implementing the vision of the all-important UX consultant.
dhosek 5y ago
I’ve been working in software engineering for 30+ years so I can say that yes, things are definitely much easier. Debugger in the 80s/90s were finicky beasts that were shrouded in esoterica and as a result, it was usually much easier to try to debug code by adding print statements than it was to actually use a debugger. I’m still somewhat amazed by the capabilities of contemporary debuggers. Libraries outside those provided by the OS/compiler tended to be hard to come by. Certainly the universe of freely available library code that we have now was nonexistent (I’d argue that CPAN was a big factor in the spread of Perl in the 90s—well, that an the assumption that was widespread back then that CGI scripts had to be written in Perl). As a community, we’ve collectively learned a lot of important lessons as well. Legacy systems like Perl and LaTeX tend to install common code in a single universal directory for everyone rather than having this be application/document specific as became the case with Java and the repositories will only give you the latest version of an artifact¹ which has tended to lead to stagnation since backwards compatibility becomes non-negotiable, although some lessons haven’t stuck (like the fact that Rust’s crates.io only has a single-level namespace for artifacts). ⸻ 1. Not sure if this is absolutely the case with CPAN. CTAN, which was quite possibly the first centralized repo² does not manage multiple versions of artifacts. 2. I remember when it was first being set up, since there was no guarantee that any tooling beyond TeX would be available to consumers, there being TeX scripts to do things like encode/decode binary files into emailable ASCII format. The original CTAN FTP server in the UK also had the remarkable feature that it would generate directory.zip for any directory that you requested.
mattmanser 5y ago
No, it definitely has got worse, I've been doing this 15 years, the sweet spot was the Rails revolution. Before that a lot of frameworks were a bit too much magic, and not enough understanding of how browsers, http and html worked. Simple MVC stacks went to all languages, jQuery front-ends doing enough but not a lot. JavaScript enhanced easy to reason about server-side stacks. You used to spend a couple of days a year, yes YEAR, mucking around with tooling. Now it wouldn't be too much of an exaggeration to say you spend a day or so a week fighting some part of your stack because it's been over-engineered. IDEs can't keep up so you have to run inscrutable command line tools that fail if you have deviated even slightly from whatever error-prone rain-dance some moron claiming 'best practice' has forced into the build process. Programming used to be about writing code to solve business problems. The shift to DevOps has been a massive productivity drain and most stacks are now incredibly brittle. The worse part has been debugging, which you touch upon. Native calls, simple call stacks, easy error logging. All gone. Moving everything into hard to debug http calls has been a disastrous productivity sink. The irony has been that as languages have got significantly better our productivity has actually dropped massively because of the ridiculous amounts of over-engineering in "modern" code bases. I recently worked on a project with 2 devs that took 3 months with a modern stack. The prototype in a standard MVC stack I'd made to demo to the client took 2 days. It's utterly ridiculous and sometimes I feel like the boy in the story about the emperor with no clothes.
- bko 5y ago
  > I recently worked on a project with 2 devs that took 3 months with a modern stack. The prototype in a standard MVC stack I'd made to demo to the client took 2 days. Can you expand on the modern stack and that standard stack? Is the "standard stack" you propose easier to work with now than it was 15 years ago? I guess the "standard stack" is just out of fashion? My preferred stack (aws serverless BE, nextjs front end or just aws app sync) keeps me in one language (typescript), with some stuff like GraphQL that you have to know. But the tooling around that helps keep my errors in compile time for the most part.
  - pc86 5y ago
    Is being in one language really that much of a benefit? I've worked on several large production apps with node backends and I'd rather use almost any other backend language. And I take issue with saying you "have to know" GraphQL, that's a pretty specialized tool for pretty specific problems. Most things are not graph structures.
abraxas 5y ago
I've been in the industry since 1998. Things have gotten worse. Part of it is that in certain ways more is expected of software today but that's not entirely true. Currently I'm involved in working on a "data intensive" app which munches a few dozen gigabytes of data and presents summarizations through a web based UI. Usually in the form of charts, time series plots, heatmaps etc. Adding small, incremental features to it is an ordeal. The "backend team" needs to mock and then provide the API endpoints, then the frontend team needs to "break down the work on the React components", then DBAs need to oversee the new queries. The security team needs to review risk impacts etc. One change is at least a few weeks of work. Contrast that with the work that I did in 1998. It was a "data heavy" desktop application that visualized in 3d seismic data collected by ultrasound probes. The data volumes were in the order of hundreds of megabytes up to a gigabyte. The app was built as a desktop app using FLTK and OpenGL. It took about a day or two to roll out an incremental feature once we knew what was being asked. By two weeks it was in all of our customers' hands. While there was no Stackoverflow.com and we relied primarily on offical documentation with the help of "Effective C++" and "More Effective C++" the build/debug/test cycle was much shorter, the tool chain was much simpler and more transparent, the UI toolkit was much easier to grok and even though it didn't have an "elegant state framework" somehow we made it work well without turning it into a mess. As an industry, we gave up on localized desktop processing in the name of "consumer preference" (or so we were told) while we assumed an immense amount of complexity in order to server the same computation from a centralized place.
native_samples 5y ago
Well, some things got easier and other things got harder. Some of the harder things are just due to higher consumer expectations or more competition, but, harder they are. The easier things are mostly obvious. Better languages, open source libs etc. Things that are harder now than before: 1. UI. The web is not a good UI platform, sorry. Designing UI in the 90s was easier, except for the need to do manual memory management if you weren't using Visual Basic. Partly because there was little expectation of branded UI, so you could easily re-use large control libraries that came with the OS which were/are pretty feature complete and well documented. 2. Cross language interop. Microsoft had this nailed. COM was a beast, but it worked and there was an actual real market of cross-language, auto-bound objects and GUI controls (COM objects, OCX controls). There was actually a thriving ecosystem of languages on Windows which are now mostly forgotten (Delphi, FoxPro, VB6, Paradox, Visual Prolog etc). Nowadays cross-language interop is a joke. Transpile to JavaScript or go via a C FFI, maybe, if you're lucky. 3. Expectation of supporting multiple platforms. This causes a lot of dysfunction because the lowest common denominator is really low, especially if you treat the browser as a "platform". In the 90s Microsoft had a monopoly. That had its own problems, but, it meant you could write a Windows app, once, and everyone would accept it. Nowadays you want web+mobile, and "web" is not really a platform in the sense Windows was. If you're doing anything serious you still need a desktop app. 4. Over-specialization/staffing. Back then everyone was "full stack". The developer was also a DBA, at least to some extent, and you could just throw up a quick GUI Windows app that connected directly to the database. The DBA would then manage security and backup. It wasn't really a full time job, at least not on a per app basis. Nowadays even simple projects feel absurdly over-staffed. Do you really need a backend guy, a frontend guy and a devops guy for an ordinary LOB app? Probably not. 5. Process overkill. Waterfall is underrated. It got a bad rap because it requires you to understand your customer, and for your customer to understand what they actually want, and for them to not change their mind every five minutes. Not always possible. Nonetheless, agile has become some kind of monster. Half the words in the average agile methodology are made up and a lot of it is really questionable. If your business domain weren't totally unstable and your users weren't totally incompetent (often the case!), then you could sit down and write an actual spec, which people would read and sign off on, and then you could build it. And it'd work, and after the initial debugging / shakeout period, people would be happy. Many of those apps never broke! Just imagine the output of the typical agile web AWS-based web stack teams today lasting 20 or 30 years without a dozen rewrites along the way. Very hard to imagine that.
selfhoster11 5y ago
> I wasn't an engineer for that long but my impression is that programming today is a lot easier. Dev tools, compilers and linters are very good. What sticks out to me when I'm looking at some older code editors/IDEs, is how crazily spartan they are - to the point where they are just inconvenient to use.

by bko 5y ago

robertlagrant 5y ago

Some things are easier, but then we've made it so quick we become the lone typist implementing the vision of the all-important UX consultant.

dhosek 5y ago

I’ve been working in software engineering for 30+ years so I can say that yes, things are definitely much easier. Debugger in the 80s/90s were finicky beasts that were shrouded in esoterica and as a result, it was usually much easier to try to debug code by adding print statements than it was to actually use a debugger. I’m still somewhat amazed by the capabilities of contemporary debuggers. Libraries outside those provided by the OS/compiler tended to be hard to come by. Certainly the universe of freely available library code that we have now was nonexistent (I’d argue that CPAN was a big factor in the spread of Perl in the 90s—well, that an the assumption that was widespread back then that CGI scripts had to be written in Perl). As a community, we’ve collectively learned a lot of important lessons as well. Legacy systems like Perl and LaTeX tend to install common code in a single universal directory for everyone rather than having this be application/document specific as became the case with Java and the repositories will only give you the latest version of an artifact¹ which has tended to lead to stagnation since backwards compatibility becomes non-negotiable, although some lessons haven’t stuck (like the fact that Rust’s crates.io only has a single-level namespace for artifacts). ⸻ 1. Not sure if this is absolutely the case with CPAN. CTAN, which was quite possibly the first centralized repo² does not manage multiple versions of artifacts. 2. I remember when it was first being set up, since there was no guarantee that any tooling beyond TeX would be available to consumers, there being TeX scripts to do things like encode/decode binary files into emailable ASCII format. The original CTAN FTP server in the UK also had the remarkable feature that it would generate directory.zip for any directory that you requested.

mattmanser 5y ago

No, it definitely has got worse, I've been doing this 15 years, the sweet spot was the Rails revolution. Before that a lot of frameworks were a bit too much magic, and not enough understanding of how browsers, http and html worked. Simple MVC stacks went to all languages, jQuery front-ends doing enough but not a lot. JavaScript enhanced easy to reason about server-side stacks. You used to spend a couple of days a year, yes YEAR, mucking around with tooling. Now it wouldn't be too much of an exaggeration to say you spend a day or so a week fighting some part of your stack because it's been over-engineered. IDEs can't keep up so you have to run inscrutable command line tools that fail if you have deviated even slightly from whatever error-prone rain-dance some moron claiming 'best practice' has forced into the build process. Programming used to be about writing code to solve business problems. The shift to DevOps has been a massive productivity drain and most stacks are now incredibly brittle. The worse part has been debugging, which you touch upon. Native calls, simple call stacks, easy error logging. All gone. Moving everything into hard to debug http calls has been a disastrous productivity sink. The irony has been that as languages have got significantly better our productivity has actually dropped massively because of the ridiculous amounts of over-engineering in "modern" code bases. I recently worked on a project with 2 devs that took 3 months with a modern stack. The prototype in a standard MVC stack I'd made to demo to the client took 2 days. It's utterly ridiculous and sometimes I feel like the boy in the story about the emperor with no clothes.

bko 5y ago
> I recently worked on a project with 2 devs that took 3 months with a modern stack. The prototype in a standard MVC stack I'd made to demo to the client took 2 days. Can you expand on the modern stack and that standard stack? Is the "standard stack" you propose easier to work with now than it was 15 years ago? I guess the "standard stack" is just out of fashion? My preferred stack (aws serverless BE, nextjs front end or just aws app sync) keeps me in one language (typescript), with some stuff like GraphQL that you have to know. But the tooling around that helps keep my errors in compile time for the most part.
- pc86 5y ago
  Is being in one language really that much of a benefit? I've worked on several large production apps with node backends and I'd rather use almost any other backend language. And I take issue with saying you "have to know" GraphQL, that's a pretty specialized tool for pretty specific problems. Most things are not graph structures.

abraxas 5y ago

I've been in the industry since 1998. Things have gotten worse. Part of it is that in certain ways more is expected of software today but that's not entirely true. Currently I'm involved in working on a "data intensive" app which munches a few dozen gigabytes of data and presents summarizations through a web based UI. Usually in the form of charts, time series plots, heatmaps etc. Adding small, incremental features to it is an ordeal. The "backend team" needs to mock and then provide the API endpoints, then the frontend team needs to "break down the work on the React components", then DBAs need to oversee the new queries. The security team needs to review risk impacts etc. One change is at least a few weeks of work. Contrast that with the work that I did in 1998. It was a "data heavy" desktop application that visualized in 3d seismic data collected by ultrasound probes. The data volumes were in the order of hundreds of megabytes up to a gigabyte. The app was built as a desktop app using FLTK and OpenGL. It took about a day or two to roll out an incremental feature once we knew what was being asked. By two weeks it was in all of our customers' hands. While there was no Stackoverflow.com and we relied primarily on offical documentation with the help of "Effective C++" and "More Effective C++" the build/debug/test cycle was much shorter, the tool chain was much simpler and more transparent, the UI toolkit was much easier to grok and even though it didn't have an "elegant state framework" somehow we made it work well without turning it into a mess. As an industry, we gave up on localized desktop processing in the name of "consumer preference" (or so we were told) while we assumed an immense amount of complexity in order to server the same computation from a centralized place.

native_samples 5y ago

Well, some things got easier and other things got harder. Some of the harder things are just due to higher consumer expectations or more competition, but, harder they are. The easier things are mostly obvious. Better languages, open source libs etc. Things that are harder now than before: 1. UI. The web is not a good UI platform, sorry. Designing UI in the 90s was easier, except for the need to do manual memory management if you weren't using Visual Basic. Partly because there was little expectation of branded UI, so you could easily re-use large control libraries that came with the OS which were/are pretty feature complete and well documented. 2. Cross language interop. Microsoft had this nailed. COM was a beast, but it worked and there was an actual real market of cross-language, auto-bound objects and GUI controls (COM objects, OCX controls). There was actually a thriving ecosystem of languages on Windows which are now mostly forgotten (Delphi, FoxPro, VB6, Paradox, Visual Prolog etc). Nowadays cross-language interop is a joke. Transpile to JavaScript or go via a C FFI, maybe, if you're lucky. 3. Expectation of supporting multiple platforms. This causes a lot of dysfunction because the lowest common denominator is really low, especially if you treat the browser as a "platform". In the 90s Microsoft had a monopoly. That had its own problems, but, it meant you could write a Windows app, once, and everyone would accept it. Nowadays you want web+mobile, and "web" is not really a platform in the sense Windows was. If you're doing anything serious you still need a desktop app. 4. Over-specialization/staffing. Back then everyone was "full stack". The developer was also a DBA, at least to some extent, and you could just throw up a quick GUI Windows app that connected directly to the database. The DBA would then manage security and backup. It wasn't really a full time job, at least not on a per app basis. Nowadays even simple projects feel absurdly over-staffed. Do you really need a backend guy, a frontend guy and a devops guy for an ordinary LOB app? Probably not. 5. Process overkill. Waterfall is underrated. It got a bad rap because it requires you to understand your customer, and for your customer to understand what they actually want, and for them to not change their mind every five minutes. Not always possible. Nonetheless, agile has become some kind of monster. Half the words in the average agile methodology are made up and a lot of it is really questionable. If your business domain weren't totally unstable and your users weren't totally incompetent (often the case!), then you could sit down and write an actual spec, which people would read and sign off on, and then you could build it. And it'd work, and after the initial debugging / shakeout period, people would be happy. Many of those apps never broke! Just imagine the output of the typical agile web AWS-based web stack teams today lasting 20 or 30 years without a dozen rewrites along the way. Very hard to imagine that.

selfhoster11 5y ago

> I wasn't an engineer for that long but my impression is that programming today is a lot easier. Dev tools, compilers and linters are very good. What sticks out to me when I'm looking at some older code editors/IDEs, is how crazily spartan they are - to the point where they are just inconvenient to use.