6 ms·

In the modern world of computing, the one thing I have seen literally everywhere I've been is people implementing systems that they absolutely do not understand

In the modern world of computing, the one thing I have seen literally everywhere I've been is people implementing systems that they absolutely do not understand. Most microservices I've seen implemented are not actually microservices. Most teams (unless they are developing "simple" web or mobile applications) have no idea who is consuming their services, or how, or if what they've made is working correctly. They frequently don't have an understanding of the different models of releasing software, much less of a complete system architecture, failure modes, consistency models, reliability estimates, performance limits, etc. Mostly what I see time and again is a team of people who just write some code that seems to work for them, and then go home, without ever considering how or if it's working in the real world. I don't know anything about modern computer science education. But it appears that new developers have absolutely no idea how anything but algorithms work. It's like we taught them how hammers and saws and wrenches work, and then told them to go build a skyscraper. There are only two ways I know of for anyone today to correctly build a modern large-scale computer system: 1) Read every single book and blog post, watch every conference talk, and listen to every podcast that exists about building modern large-scale computing systems, and 2) Spend 5+ years making every mistake in the book as you try to build them yourself. It feels like the industry mostly just re-learns the same mistakes over and over like we're in Groundhog's Day (we're the extras, not Bill Murray). But it's equally possible that I just lack perspective and am expecting too much. Maybe the auto industry at the turn of the 20th century also spent decades re-learning the same lessons over and over, as the novelty of mass-producing complex systems continued to elude us. Hell, the new auto companies still don't get it right.

pfarrell 5y ago
“There aren’t any new problems; only new engineers.” - Quote I read online a long time ago Agree with all your points, but there is a third way: apprenticeship with older engineers. I advise new engineers to find their balance between not accepting the status quo but also realizing software is way more complicated than they think. For example: successfully versioning an API over time while supporting live deployments is (I think) not something you’re going to learn in school. As a further aside... I’ve heard older attorneys complain about newly minted lawyers. They work to pass school and then the bar. But when they start lawyering, new grads don’t know the first thing about how to actually file something at the courthouse. I’m saying, I don’t think it’s constrained to our industry.
- sangnoir 5y ago
  > Agree with all your points, but there is a third way: apprenticeship with older engineers. That's unlikely to work: our industry has an ageism problem, combined with the low prospects of promotions/raises which results in engineers having to frequently job-hop to get ahead. This results in most organizations having very limited institutional memory. Other compounding problems are a discussion for another day (title inflation: e.g. "Senior engineer" with 2 years total experience)
wildrhythms 5y ago
I attended a community college and learned Java, C#, PHP, HTML, CSS, Javascript... the classes were more or less an introduction to full stack web development. I remember when I transferred to a four-year school for my Bachelor's the other students in the CS program (we were all Juniors) struggled to put together a year-end project requiring a GUI- like a text input and some buttons, simple HTML/CSS, or Swing (popular at the time). They didn't even know where to start with GUI development. Is this kind of thing still not being taught?
- swrj 5y ago
  I can only speak for my own experience (large, top 25 state school in the US) - this sort of thing is still not taught. There was maybe one or two optional courses that taught software engineering related material, the rest of it was just a lot of computer science theory. Definitely struggled the first couple of months at my first job out of college due to this.
- Izkata 5y ago
  My college, none of that was in the Computer Science (CS) curriculum - it was all part Information Technology and Management (ITM). CS was focused mostly on the broad underlying structure of computing, things like how software interacts with hardware (this was gutted a year or two before me), algorithms, programming paradigms (functional/etc), the theory behind databases (3rd normal form sound familiar?), and so on. The idea was to give you a really good foundation such that you could self-learn much much easier after college, but also unfortunately that you had to do some of that selfglearning right away because it didn't prep you for getting things done. I only took one or two ITM courses out of curiosity, so I'm less confident in describing it in broad strokes, but the impression I got is that it was for teaching specific skills while not "wasting" time on better understanding how they work. Very get-things-done oriented. The web development one, for example, used a lot of jquery, but never touched on what was going on under the hood. I can see it working as preparation for being able to do a job, but it left you kind of stuck if you hit a roadblock and didn't know where to look for answers. If I had to stereotype them, a CS student would spend time writing an algorithm for something instead of finding an existing library, while an ITM student would look for a library and if they can't find one say they can't do it. But the ITM student wouldn't get lost implementing and end-to-end project as long as they could use tech they already know, whereas the CS student has never done such a thing before.
akiselev 5y ago
> Maybe the auto industry at the turn of the 20th century also spent decades re-learning the same lessons over and over, as the novelty of mass-producing complex systems continued to elude us. Hell, the new auto companies still don't get it right. I think you have a misconception about what manufacturing is like. Waterfall development is just as unrealistic in manufacturing as it is in software, the scales are just different. People create CAD models of the machines in a factory all the time but those are architectural diagrams - when it comes time to build, the people on the ground (the equivalent of the software engineers) have to figure out all of the details, often using no more than trial and error guided by a little intuition. Experienced individuals know more about the pitfalls and gotchas but they're still creating something new even if they're going off some grand design. Becoming a competitive manufacturer in something simple like ball point pen balls is a years long process of mastering the "craft," gathering real world data, and incrementally improving. Factors like a single critical employee's physical disability can decide the layouts of entire factories so there are no cookie cutter factories for them to build. The real world is a lot messier than software. Just like we spend most of our time chasing bugs and implementing features, they spend most of their time building or troubleshooting interfaces between machines and processes, optimizing, and so on.
- collyw 5y ago
  > Waterfall development is just as unrealistic in manufacturing as it is in software Is it? Aren't a lot of the largest systems build using waterfall, or they certainly were in the past. When i was taught waterfall at university you were allowed to reiterate steps, yet these is a constant straw man that you only gte to move down the waterfall.
  - gonzo41 5y ago
    Well maybe the way to compare software waterfall to manufacturing waterfall is that the CI/CD pipeline that's in use is the waterfall. But even car companies have a development stage that's messy where they have to retool production lines. But they manage it the same way we do, by making things as standard as possible. Like how most SUV's have the same chassis and only really differ in internal fitout and body panels.
axaxs 5y ago
> It feels like the industry mostly just re-learns the same mistakes over and over like we're in Groundhog's Day This is true, but also in life. I'm so sad I can't just pass my knowledge to my daughter to give her a 30 year head start. I remember telling her 'dont touch that stove it's hot', she nods, and soon after is crying she burnt her hand. I wanted to be mad but it made me realize the one universal truth: advice and information is great, but humans just seem to learn most from mistakes. Made me realize how much time and energy and knowledge is lost, as every human learns a lot of the same things over and over. I'm sure there was a caveman telling his kid not to touch the fire thousands of years ago, too. And here we are. So don't be too cynical and depressed about this cycle, it's pretty natural.
- adjkant 5y ago
  I just want to give a giant explicit +1 to this sentiment and understanding of how humans learn, and then add a bit from there to connect it back to the problem highlighted in the first comment. IMO, there's a few ways to try and take a pragmatic approach to the problem above: 1. Make simple ways to do complex things. A library, a language, a framework, they all go after the same thing when designed well: batteries included complex functionality that the end user can't shoot themselves in the foot with. It seems like in this domain (microservices), we are still early and need better support for small companies to spin this up. I've found my current company runs microservices pretty well, but I know it took a TON of infrastructure, support, and education to get it to that point. Few companies can afford to have that. The danger with this approach is that people who understand the internals become rare, and surface level only knowledge can lead to issues when you get to edge cases. I'll still take it, but wanted to call it out. The linux kernel is a great example of what that looks like down the line. 2. Make as many places as possible for developers in training to fail and make those 5 years of mistakes with low stakes consequences. In some ways, the way the industry hires already does this, but I'm sure there are better ways we can come up with versus the current sandboxes. If your frustration boils over, this is a place to channel it positively.
  - axaxs 5y ago
    Thanks. I really like idea number 2. In training, have people fail nearly purposefully. I think some of the problem is that there's no real standardized training for developers. Many have 4 weeks of 'bootcamp' and are cut into the wild. And I can't be too negative about it, I didn't even have that. I would just say I learned from 20 years of reading and mistakes.
Spooky23 5y ago
I think computing is following industry. I’d argue that most companies are younger and dumber than they were when I entered the professional world about 20 years ago. A few months ago, i engaged to help fix a troubled system. The thing was built around kubernetes and the architecture was basically aligned with how a bunch of contractors were hired. Nobody really understood how it worked and it performed poorly. So poorly in fact that I reimplemented a core component in a shell script, mostly to quickly understand how it worked, and to my surprise, the thing ran faster on my laptop at moderate load than the actual system! Sounds like a bad IT project, but to be honest whole companies run this way, saving pennies while setting dollars on fire.
- mbreese 5y ago
  > most companies are younger and dumber than they were when I entered the professional world about 20 years ago Channeling a post from earlier today — I’m not sure if companies are younger and dumber today, but I’m sure that you are older and wiser now than you were then. And I’m sure you are better at seeing the idiocy in companies. Bad projects have always been around. But it isn’t until you have more experience that you can fully appreciate a bad project — particularly when you’re in the middle of it!
pm90 5y ago
Computer Science curriculum in most universities is focused on theory. Mine had a large component of working with and understanding digital electronics, Operating Systems, Networking, Discrete Math. Most of the ‘popular’ electives were AI related. Very few courses on actually building maintainable and functional systems. Most of what I learned I had to learn from experience and from others I worked with.
barbarbar 5y ago
Welcome to agile development. We fix it in the next sprint. Except we don't - since we need the new button.
jjav 5y ago
> But it appears that new developers have absolutely no idea how anything but algorithms work. Possibly a case of teaching to the test (interview, in this case)? My CS education is a couple decades back, but while we had an algorithm course, most credit hours were spent on more practical things like building compilers, building an OS, building a shell, SQL, networking stacks, encryption, software engineering complex systems and so forth. Is the current CS landscape more algorithm-focused because the FAANGs have made that the only stick to measure by?
- dagw 5y ago
  When I went to school they offered both Computer Engineering and Computer Science degrees[1]. The Computer Engineering degree was more like how you described your degree (plus hardware) and the Computer Science degree was more math, algorithms and theory focused. I personally feel that this is the obvious way to do it. [1]they later added a Software Engineering degree which focused more on architecture and designing large scale software system, software project management and so on.
diveanon 5y ago
Most software developers these days are just glorified technicians stringing together api's. There is a big bubble in tech salaries that is going to get popped as more and more people realize you just don't need a developer for most things and can get away with no-code for the majority of your needs. This will act as a force multiplier for some, but I think for most developers it will see a correction in their salaries to more reasonable levels.

pfarrell 5y ago

“There aren’t any new problems; only new engineers.” - Quote I read online a long time ago Agree with all your points, but there is a third way: apprenticeship with older engineers. I advise new engineers to find their balance between not accepting the status quo but also realizing software is way more complicated than they think. For example: successfully versioning an API over time while supporting live deployments is (I think) not something you’re going to learn in school. As a further aside... I’ve heard older attorneys complain about newly minted lawyers. They work to pass school and then the bar. But when they start lawyering, new grads don’t know the first thing about how to actually file something at the courthouse. I’m saying, I don’t think it’s constrained to our industry.

sangnoir 5y ago
> Agree with all your points, but there is a third way: apprenticeship with older engineers. That's unlikely to work: our industry has an ageism problem, combined with the low prospects of promotions/raises which results in engineers having to frequently job-hop to get ahead. This results in most organizations having very limited institutional memory. Other compounding problems are a discussion for another day (title inflation: e.g. "Senior engineer" with 2 years total experience)

wildrhythms 5y ago

I attended a community college and learned Java, C#, PHP, HTML, CSS, Javascript... the classes were more or less an introduction to full stack web development. I remember when I transferred to a four-year school for my Bachelor's the other students in the CS program (we were all Juniors) struggled to put together a year-end project requiring a GUI- like a text input and some buttons, simple HTML/CSS, or Swing (popular at the time). They didn't even know where to start with GUI development. Is this kind of thing still not being taught?

swrj 5y ago
I can only speak for my own experience (large, top 25 state school in the US) - this sort of thing is still not taught. There was maybe one or two optional courses that taught software engineering related material, the rest of it was just a lot of computer science theory. Definitely struggled the first couple of months at my first job out of college due to this.
Izkata 5y ago
My college, none of that was in the Computer Science (CS) curriculum - it was all part Information Technology and Management (ITM). CS was focused mostly on the broad underlying structure of computing, things like how software interacts with hardware (this was gutted a year or two before me), algorithms, programming paradigms (functional/etc), the theory behind databases (3rd normal form sound familiar?), and so on. The idea was to give you a really good foundation such that you could self-learn much much easier after college, but also unfortunately that you had to do some of that selfglearning right away because it didn't prep you for getting things done. I only took one or two ITM courses out of curiosity, so I'm less confident in describing it in broad strokes, but the impression I got is that it was for teaching specific skills while not "wasting" time on better understanding how they work. Very get-things-done oriented. The web development one, for example, used a lot of jquery, but never touched on what was going on under the hood. I can see it working as preparation for being able to do a job, but it left you kind of stuck if you hit a roadblock and didn't know where to look for answers. If I had to stereotype them, a CS student would spend time writing an algorithm for something instead of finding an existing library, while an ITM student would look for a library and if they can't find one say they can't do it. But the ITM student wouldn't get lost implementing and end-to-end project as long as they could use tech they already know, whereas the CS student has never done such a thing before.

akiselev 5y ago

> Maybe the auto industry at the turn of the 20th century also spent decades re-learning the same lessons over and over, as the novelty of mass-producing complex systems continued to elude us. Hell, the new auto companies still don't get it right. I think you have a misconception about what manufacturing is like. Waterfall development is just as unrealistic in manufacturing as it is in software, the scales are just different. People create CAD models of the machines in a factory all the time but those are architectural diagrams - when it comes time to build, the people on the ground (the equivalent of the software engineers) have to figure out all of the details, often using no more than trial and error guided by a little intuition. Experienced individuals know more about the pitfalls and gotchas but they're still creating something new even if they're going off some grand design. Becoming a competitive manufacturer in something simple like ball point pen balls is a years long process of mastering the "craft," gathering real world data, and incrementally improving. Factors like a single critical employee's physical disability can decide the layouts of entire factories so there are no cookie cutter factories for them to build. The real world is a lot messier than software. Just like we spend most of our time chasing bugs and implementing features, they spend most of their time building or troubleshooting interfaces between machines and processes, optimizing, and so on.

collyw 5y ago
> Waterfall development is just as unrealistic in manufacturing as it is in software Is it? Aren't a lot of the largest systems build using waterfall, or they certainly were in the past. When i was taught waterfall at university you were allowed to reiterate steps, yet these is a constant straw man that you only gte to move down the waterfall.
- gonzo41 5y ago
  Well maybe the way to compare software waterfall to manufacturing waterfall is that the CI/CD pipeline that's in use is the waterfall. But even car companies have a development stage that's messy where they have to retool production lines. But they manage it the same way we do, by making things as standard as possible. Like how most SUV's have the same chassis and only really differ in internal fitout and body panels.

axaxs 5y ago

> It feels like the industry mostly just re-learns the same mistakes over and over like we're in Groundhog's Day This is true, but also in life. I'm so sad I can't just pass my knowledge to my daughter to give her a 30 year head start. I remember telling her 'dont touch that stove it's hot', she nods, and soon after is crying she burnt her hand. I wanted to be mad but it made me realize the one universal truth: advice and information is great, but humans just seem to learn most from mistakes. Made me realize how much time and energy and knowledge is lost, as every human learns a lot of the same things over and over. I'm sure there was a caveman telling his kid not to touch the fire thousands of years ago, too. And here we are. So don't be too cynical and depressed about this cycle, it's pretty natural.

adjkant 5y ago
I just want to give a giant explicit +1 to this sentiment and understanding of how humans learn, and then add a bit from there to connect it back to the problem highlighted in the first comment. IMO, there's a few ways to try and take a pragmatic approach to the problem above: 1. Make simple ways to do complex things. A library, a language, a framework, they all go after the same thing when designed well: batteries included complex functionality that the end user can't shoot themselves in the foot with. It seems like in this domain (microservices), we are still early and need better support for small companies to spin this up. I've found my current company runs microservices pretty well, but I know it took a TON of infrastructure, support, and education to get it to that point. Few companies can afford to have that. The danger with this approach is that people who understand the internals become rare, and surface level only knowledge can lead to issues when you get to edge cases. I'll still take it, but wanted to call it out. The linux kernel is a great example of what that looks like down the line. 2. Make as many places as possible for developers in training to fail and make those 5 years of mistakes with low stakes consequences. In some ways, the way the industry hires already does this, but I'm sure there are better ways we can come up with versus the current sandboxes. If your frustration boils over, this is a place to channel it positively.
- axaxs 5y ago
  Thanks. I really like idea number 2. In training, have people fail nearly purposefully. I think some of the problem is that there's no real standardized training for developers. Many have 4 weeks of 'bootcamp' and are cut into the wild. And I can't be too negative about it, I didn't even have that. I would just say I learned from 20 years of reading and mistakes.

Spooky23 5y ago

I think computing is following industry. I’d argue that most companies are younger and dumber than they were when I entered the professional world about 20 years ago. A few months ago, i engaged to help fix a troubled system. The thing was built around kubernetes and the architecture was basically aligned with how a bunch of contractors were hired. Nobody really understood how it worked and it performed poorly. So poorly in fact that I reimplemented a core component in a shell script, mostly to quickly understand how it worked, and to my surprise, the thing ran faster on my laptop at moderate load than the actual system! Sounds like a bad IT project, but to be honest whole companies run this way, saving pennies while setting dollars on fire.

mbreese 5y ago
> most companies are younger and dumber than they were when I entered the professional world about 20 years ago Channeling a post from earlier today — I’m not sure if companies are younger and dumber today, but I’m sure that you are older and wiser now than you were then. And I’m sure you are better at seeing the idiocy in companies. Bad projects have always been around. But it isn’t until you have more experience that you can fully appreciate a bad project — particularly when you’re in the middle of it!

pm90 5y ago

Computer Science curriculum in most universities is focused on theory. Mine had a large component of working with and understanding digital electronics, Operating Systems, Networking, Discrete Math. Most of the ‘popular’ electives were AI related. Very few courses on actually building maintainable and functional systems. Most of what I learned I had to learn from experience and from others I worked with.

barbarbar 5y ago

Welcome to agile development. We fix it in the next sprint. Except we don't - since we need the new button.

jjav 5y ago

> But it appears that new developers have absolutely no idea how anything but algorithms work. Possibly a case of teaching to the test (interview, in this case)? My CS education is a couple decades back, but while we had an algorithm course, most credit hours were spent on more practical things like building compilers, building an OS, building a shell, SQL, networking stacks, encryption, software engineering complex systems and so forth. Is the current CS landscape more algorithm-focused because the FAANGs have made that the only stick to measure by?

dagw 5y ago
When I went to school they offered both Computer Engineering and Computer Science degrees[1]. The Computer Engineering degree was more like how you described your degree (plus hardware) and the Computer Science degree was more math, algorithms and theory focused. I personally feel that this is the obvious way to do it. [1]they later added a Software Engineering degree which focused more on architecture and designing large scale software system, software project management and so on.

diveanon 5y ago

Most software developers these days are just glorified technicians stringing together api's. There is a big bubble in tech salaries that is going to get popped as more and more people realize you just don't need a developer for most things and can get away with no-code for the majority of your needs. This will act as a force multiplier for some, but I think for most developers it will see a correction in their salaries to more reasonable levels.