14 ms·

For a while now I've had a feeling that all the comments about lack of engineers(especially in software) are vastly underestimated. Probably around 10% of us is

For a while now I've had a feeling that all the comments about lack of engineers(especially in software) are vastly underestimated. Probably around 10% of us is capable of doing actual software development. The rest writes plumbing and can handle the project for only as long as abstractions available through libraries can hold the complexity. If we assume most of us don't really know what we're doing, that totally explains language preferences. We don't choose the best tools, we choose the best tooling. And best tooling is the one which we can comfortably fit in our minds, so advanced concepts are mostly ignored and the hype wheel constantly turns. To put it shortly - smart people write better and better tools, so the rest can handle bigger and bigger projects writing mediocre code. Of course, as every programmer, I live in constant fear that I am part of the plumbing crowd waiting to be exposed.

kbwt 9y ago
Reminds me of this picture entitled Life of a game programmer: https://i.imgur.com/sBih7ol.jpg https://i.imgur.com/sBih7ol.jpg
DanielBMarkham 9y ago
Nerds screw up everything we touch. I don't think we mean to. Whatever the system, we make it more complicated, er featureful. Then we add an abstraction layer. Then we make that layer more complicated. Repeat and rinse. Some abstraction layers help more than they hurt, but the ratio is about 1-in-10 or so. For any given project, there are probably a dozen cool-sounding frameworks or layers, one of which is absolutely needed. The rest are there because somebody wants to be promised that even crappy programmers can use this to make cool stuff happens. The sales pitch is clear: don't become a better programmer, get a better toolkit. I have been quite fortunate to have come into computers before this cloud of marketing madness overtook us. I got to watch the layers roll out one-by-one. Honestly I have no idea how I would learn programming if I had to do it again today. It's just too much of a byzantine mess. Hell if I would start with the top layers, though. I'd rather know how to write simple code in BASIC than have a grasp of setting up a CRUD website using WhizBang 4.7 When you learn programming, well you should be learning how to program. Not how to breeze through stuff. Breezing through stuff is great when you already know what's going on -- but by the time you already know what's going on, it's unlikely you'll need the layer or framework. (Sadly, the most likely scenario is that you've invented yet another framework and are busy evangelizing it to others.) This guy's story strikes me as poignant and indicative of where the industry is. They don't care if you can solve people's problems. They care if you're one of the cool kids. It's nerd signaling.
- d3ckard 9y ago
  I'd put it differently. Software development severely lacks any objective metrics of performance and quality. We just haven't invented any(at least any practical enough to become mainstream). As a result, we quite often misjudge our(and others') skill and make bad decisions. It's the goal of software to provide more and more features. The problem is, we usually achieve those features by abusing abstractions we are using. Once the problem becomes apparent, somebody writes a library, which allows us to write a couple of times more crappy code before everything collapses. Rinse and repeat. Since we are unable on scale to choose right abstractions when necessary, we quite often get ourselves in situations, when it's actually preferable to rewrite everything using lessons learnt. Rewriting comes with its own set of new problems and cycle is complete. As a result, we are fundamentally unable to reuse already written code on scale, crumbling development and putting the whole industry in constant early-stage.
  - DanielBMarkham 9y ago
    >>It's the goal of software to provide more and more features. That's the goal. The key question is: are we writing features that users find valuable or are we writing features that developers find valuable? They're two different things. What's happened is that the community has become incestuous. Instead of focusing on value to the user we're focusing on selling frameworks to one another. >>Software development severely lacks any objective metrics of performance and quality. Agreed here as well. But this is related to my first point. Nobody wants to focus on the users. It's far too easy to focus on the technology (or other developers). You write a feature for a user, you can instrument it and see whether users use it or not. You write a feature for a developer, even if nobody uses it you can argue that somewhere, somehow, that feature is going to be critical. It's an abstract value argument -- which is intractable. >>As a result, we are fundamentally unable to reuse already written code on scale, crumbling development and putting the whole industry in constant early-stage. It's a sad state of affairs. I suggest that your conclusion continues the broken thinking I'm describing above. We shouldn't be striving for reusable code at scale. We should be striving for people who have strong basic programming skills and are experts in some business domain. Then we would shoot for the minimum amount of technological abstraction necessary for these programmers to solve problems in that particular domain. Because that's really what the whole point is, not creating large codebases that last twenty years, creating tiny bits of code almost instantly that solve business problems for twenty years. We've got our head stuck in the wrong bucket.
crispyambulance 9y ago
AFAIK, the guy has had a marvelous and productive career. When he says "...knowing Lisp destroyed my programming career" he just means that at some point in time he switched to other things. There was no "destruction". It was "a pivot"-- to use HN lingo. I think anyone who makes programming (let alone programming in a particular tool/language) the absolute focus of their career is in for a major disappointment. The OP was NOT crushed by his realization. He just moved on and appears to have been very successful regardless. Not a big deal unless one is obsessed with Lisp.
pjc50 9y ago
Maintaining a distinction between "actual software development" and "plumbing" is elitist, even if you're placing yourself on the downside of that comparison and setting yourself up for imposter syndrome. You can get an awful lot done by "plumbing". Entire businesses like SAP are built on it. It can also be mission critical; in SpaceX, is the literal plumbing of hydraulic fluid and fuel flow unimportant? No.
- Angostura 9y ago
  > "actual software development" and "plumbing" is elitist, Perhaps, but you are the person who prepended 'mere' in front of plumbing.
  - pjc50 9y ago
    I'll take it out again if people are going to argue by string matching rather than reading the tone of the original post.
    - Angostura 9y ago
      It might be best.
- golergka 9y ago
  > Maintaining a distinction between "actual software development" and "plumbing" is elitist You are correct - however, it doesn't mean that this statement is necessarily wrong. Usually when I see a statement that reeks of elitism, I immediately assume lower probability of it being true, because elitism of a statement correlates with falsehood - but it's worth remembering that this correlation is not absolute, and sometimes (although rare), elitism is indeed deserved and true. And in this particular case, out of my personal professional experience, elitism is absolutely deserved. There are a lots and lots of software engineers out there that can't or just won't fit more complex ideas into their minds. And yet, you are of course, correct: these developers can, and will, do good work. They can have different skills, like knowing the users, or creating beautiful animations, or having a great game design sense. These developers don't need to be looked down upon. And yet it would be very useful for all of us to acknowledge that these developers think in a different way and require different tools - they're not "worse" than "real" engineers, but they are different.
shanxS 9y ago
for any given level we use abstractions, those abstractions maybe in form of algorithms or tools - so isn't everything plumbing?
wglb 9y ago
So let's think about this in a different way. Rather than classify programmers in buckets, e.g., plumber, non-plumber. Instead, the tasks of vast teams of programmers has grown to include a significantly richer environment. Skills beyond pure programming that we all need to do include connecting vast sets of existing environments together to build the next thing that we are working on. Few environments that we work on don't have networking, acres of existing APIs, already built ecosystems of data. In today's environment, 'pure programming' is a small fraction of what needs to get done. As a long-time programmer, the most leverage I have had is when my work connected to some business objective and produced a result. My engineering background, which emphasized a "problem-solving attitude" helps.
flarg 9y ago
As someone not even worthy of the status of a plumber, I quite agree. But the wonder of it all is that we can attend on the shoulders of giants so easily.
lr4444lr 9y ago
Your post resonates, but let's be fair: the business of a lot of the money in software now - users downloading deploys that change daily or more frequently, on multiple devices, with expectations to interoperate with so much other software - has really changed the incentives of what to focus on.
watwut 9y ago
Why do you think that writing those tools is harder? Oftentimes it is not. A lot of it is easy. Yes, well done abstractions are how we keep large projects manageable. No one can learn all of them at once. Neither those writing "plumbing" nor those doing libraries.
barrkel 9y ago
I reckon all software is plumbing. Plumbing as in taking data in, processing it, and pushing the data out. Doesn't matter if it's a database, a compiler, or a CRUD app; it's all just plumbing.
- segmondy 9y ago
  No, plumbing is taking prebuilt pieces like pipes and elbows, joints, valves and just putting them together. There are plenty of software that is not about taking pre-existing packages and gluing them together, but rather involves construction from the ground up. Building a CRUD app using an existing framework, and an existing rest API with little to no custom business logic is plumbing. Using sci-kit for your ML app is plumbing. Writing your own novel ML/deep learning algorithm, shoving the data in and out of GPUs is not plumbing.
  - contingencies 9y ago
    Your distinction doesn't hold up to analysis: at some level any system can be said to consist of elbow joints - eg. "processor opcodes are just elbow joints." The same can be said not only of the subsystems but also the intellectual heritage of so-called "original" systems. Nothing and everything is new. News at nine. Someone watches the news. They create a new system ever so subtly influenced by yours. Ad infinitum.
  - VonGallifrey 9y ago
    > Writing your own novel ML/deep learning algorithm, shoving the data in and out of GPUs is not plumbing. This is plumbing as well by your definition. Writing your own novel ML/deep learning algorithm also take "pre-existing packages" and glues them together. Unless you are actually going to write your own custom OS, Language and drivers you are going to reuse stuff. I don't think there is anything wrong with using pre-existing packages and the like. We all are doing that anyways unless your are Terry A. Davis off course.
kamaal 9y ago
Plumbing does get very complicated when your building size increases. And after a while its not just plumbing. Ensuring no pipe leaks in a skyscraper is no easy task.
YeGoblynQueenne 9y ago
>> Of course, as every programmer, I live in constant fear that I am part of the plumbing crowd waiting to be exposed. Well, the cheapest, safest assumption is that your skill level is about average and the majority of the programmers you 'll meet are going to have the same kind of skills as you.
segmondy 9y ago
I sort of agree, but I also disagree. I think 90% of us are capable of doing "actual software development." However the market does't care for that. The market prefers short term gains over long term gains. Perhaps we can blame wallstreet? Due to the demand for short term gains, the ask from most developers is "how fast can you build this" not "how can you build this to be most efficient and cheapest in the long run and lifetime of the product" To move quick, developers must then employ abstractions layered upon abstractions. The short term winners get lots of money for demonstrating wins quickly. The losers conform to keep up.
- mywittyname 9y ago
  > "how can you build this to be most efficient and cheapest in the long run and lifetime of the product" I couldn't answer this question accurately. I can't even give remotely accurate time estimates for projects larger than "build a CLI tool to do this one thing," much less give an informed estimate about the tool's TCO. I feel like I'm just floating down the river, incrementally improving upon the stuff that we've already built and that nothing we planned to accomplish ever is. I'm sure lots of people here have executed their Grand Vision for a project. But I'm also certain many of us never have.
- piva00 9y ago
  I don't agree with this view. Most jobs working with software development is about creating business value. Either as an end-user product that your customers are going to use or with internal tooling that will help the business have more access to data, streamline processes, increase efficiency, etc. This "no true Scotsman" approach to software development is actually quite funny after a while being a professional, it's a huge industry, there are terrible companies, there are terrible managers, product managers, etc., but there are also great ones. You can work for good ones if your current gig is mostly being pushed around by unreal expectations from your stakeholders. The demand is not for short-term gains, how can you justify that it's going to take a year to build your perfectly architected software if the business really needs that done in 3 months and you assess it's quite doable if you decide on some constraints? Your job as a professional engineer is to be able to find ways to do your work as best as possible given constraints, to design something to be improved on over time, to communicate with stakeholders and, given your area of expertise (software), give valuable input to the business decision so they can be the best as possible at that moment. Seeing software engineering as some grandiose goal by itself is quite wrong, software exists MOSTLY to fulfil business purposes. It's not about "conforming", there is software that is fun to work, that are intellectual challenges by themselves but that really have no way to be justified on a business level. This defensiveness against "business" is part of a mindset that should be broken among engineers, we should embrace we are part of a much larger process, not that we are the golden nugget and the top of the crop at a company. Our work is to enable others' work.
vijaybritto 9y ago
You seem to be experienced and ignorant at the same time
- contingencies 9y ago
  You can accurately attribute that quote to any external culture, re: another culture. Hunter-gatherers on capitalist democracy, Capitalist democracy on hunter-gatherers, China on India, India on China, US on India, India on Iran, Iran on Israel... You get the picture. Perhaps what it really means is "communications failure: exception thrown in cultural assumptions".
ubermonkey 9y ago
There's a real tension in this field between capital-C Computer Science and the work of writing code for businesses. The business doesn't care about your toolset, probably. What they care about is solving a problem. I've met no small number of very gifted, very creative developers that had this same mindset -- didn't give a crap about the Cool New Language or pure CS, but really DID get excited about building connections and features within existing systems to solve business problems. These two visions of development are in tension. Few folks can get jobs writing Haskell or Lisp, but there are LOTS of jobs for .NET developers. Neither path necessarily makes a better developer, though.
- threatofrain 9y ago
  It's like saying a doctor isn't doing interesting things if they're just content to apply medical knowledge to improve community health. The doctor is just using tools developed by other people.
  - Bartweiss 9y ago
    I'd feel better about this if it weren't for the growing evidence that an awful lot of what we actually get is cargo-cult medicine - procedures that have been supplanted or invalidated, but are still widely used by non-research doctors who don't know better. Obviously doctors are still good, and don't need to be doing research to make the world a better place. I agree that dismissing 'plumbing' programmers or 'rote' doctors would be a serious mistake. But... well, I can't help drawing some connections between programmers implementing already-broken security, and doctors putting in heart stents that don't actually help patients. I don't think we're just being metaphorical here, I think research vs practice doctors often show the same patterns as programmers, for the same reasons. Creating new knowledge is neither necessary nor sufficient for keeping up with other people's knowledge, but we seem to be worryingly bad at keeping people who implement that knowledge up to date. (Context on the doctors: https://www.theatlantic.com/health/archive/2017/02/when-evidence-says-no-but-doctors-say-yes/517368/?single_page=true https://www.theatlantic.com/health/archive/2017/02/when-evid...)
pulsarpietro 9y ago
>Of course, as every programmer, I live in constant fear that I am part of the plumbing crowd waiting to be exposed. How true, the 50% of jobs I see are actually semi-skilled overpayed jobs which won't last. In many cases you configure somebody's else product and you manage it at best, it won't last. It is already happening.
ziikutv 9y ago
Any suggestions on how I can get into that 10%?
- eropple 9y ago
  Write more code. Stretch yourself. Learn how other people work and figure out how you can improve what they do, because that tends to be the bailiwick of what he's referring (mistakenly, IMO, but the bucket is probably fine even if the label isn't) as "real software development". I didn't write Auster[0] for me, I wrote it because other people needed a tool and it fit the parameters. I'm not writing Modern[1] for me, I'm writing it because there's a hole in the ecosystem that somebody has to solve, and I'm a someone. [0] - https://github.com/eropple/auster https://github.com/eropple/auster [1] - https://github.com/modern-project/modern-ruby https://github.com/modern-project/modern-ruby
- d3ckard 9y ago
  If I knew, I would do it myself and wouldn't be afraid ;) Probably the best advise is to learn as much as possible, but focus on basics. I feel people often mistake knowledge for skill and that harms them in the long run. If you learn a concept, you can use it in anything you create. If you learn a framework, you will have to learn a new one in 5 years. Focus on transferable skills.
jgh 9y ago
For the past four years I've architected, built, and maintained a system largely designed using the pipes and filters pattern. Does that make me Mario?
Waterluvian 9y ago
"actual software engineering" is like saying that you're not doing "actual farming" unless you're using some oxen and an iron plough.
agumonkey 9y ago
I agree 80% with this, I'll add something. I always had a lispy side, I hated Java, PHP, C and others prevalent languages that most companies would bet their money on, I like APL, I like Forth. That said I found something weird, sometimes, and with a little bit of adequate libraries (guava for instance) I enjoy doing some Java. It's verbose, way more than lisp, clojure, kotlin (not even mentionning mr Haskell of course). But I find a little pleasure in doing code there. It's manual, it requires doing a lot of things but even though its slower and more "work" it's another kind of stimulation, that I think is one large factor of people doing code in subpar languages. They're just happy doing things and solving things their own way. Some say that lisp and haskell can't be mainstream because their power is best fit for complex problems, and I think that hints to my previous point. People who need more than mainstream have the brain power and desire to solve non mainstream things. ps: about the plumbing thing, you might have heard that MIT switched to python exactly for that reason. I was and still am stumped that the people would brought SICP decided that plumbing was the way to go.
hoplite009 9y ago
As my professor used to say, Computer Scientists are merely merchants of abstraction. Layers of abstraction is what we all deal with, albeit at different levels.
rjhallsted 9y ago
So how does one make sure they're not plumbing?
jack6e 9y ago
It's easy to differentiate: real software developers use butterflies [0]. [0] https://www.xkcd.com/378/ https://www.xkcd.com/378/
astine 9y ago
I think that no matter where you are as a programmer, you're part of the plumbing crowd whether you like it or not. If you hack together websites using a web framework like Rails or whatever, then that web framework is likely your plumbing. If however you write a web framework, then the language (Ruby, etc) is your plumbing. Those who write languages have operating systems and those who write operating systems have hardware. Even hardware has different levels of abstraction in it from folks designing instruction sets down to folks who have to actually deal with the physics of circuits on silicon. The things is though, a lot of those folks wouldn't be able to deal with the stuff that a typical web developer deals with because they don't understand the domain. Writing a highly performant web server is a different task from writing a complete web application that uses that web server. I think that's it's useful to have a deeper understanding of the tools we use, but ultimately we all have to work at the level of abstraction that makes sense for the work we do and that's not a problem.
swat535 9y ago
Sigh.. I for one will never understand why everyone is obsessed about titles. Is doing front end not actual software development? What does that even mean? Are you considered a real software engineer if you can write code in a certain language? .. Or does it mean you are really good at O(1) problems? I think we should all agree that software development is a team effort, it requires vast knowledge and skills that different individuals bring to the table. Doing CSS adds just as much value as writing the backend API. Here is a good analogy. For an aircraft to function, pilots, aircraft mechanists, aerospace engineers all come together. Does that mean one is doing "plumbing"? Certainly not, they all add value.
Pokepokalypse 9y ago
The real problem is that the majority of us probably are actually capable of doing actual software development right when we come out of school. Most of us get slotted into crappy plumbing and middleware jobs, and after 5-10 years those skills get rusty. They can come back; but if you end up out on the job market, being asked some of the advanced questions in interviews. . . well good luck. That situation exposes one fairly quickly.
JustSomeNobody 9y ago
The 10% of developers you mention are the ones who don't cargo cult program. They don't seek "Neo" architectures. They are the ones who knuckle down and solve actual, real problems, not some fantasy generalized version of it. They are the ones who code and hone their craft. They don't spend any time arguing over React vs VueJs, Rust vs C++, etc. I think the average developer can do this too if they'd just get out of the programming echo chambers and trust their gut.

kbwt 9y ago

Reminds me of this picture entitled Life of a game programmer: https://i.imgur.com/sBih7ol.jpg https://i.imgur.com/sBih7ol.jpg

DanielBMarkham 9y ago

Nerds screw up everything we touch. I don't think we mean to. Whatever the system, we make it more complicated, er featureful. Then we add an abstraction layer. Then we make that layer more complicated. Repeat and rinse. Some abstraction layers help more than they hurt, but the ratio is about 1-in-10 or so. For any given project, there are probably a dozen cool-sounding frameworks or layers, one of which is absolutely needed. The rest are there because somebody wants to be promised that even crappy programmers can use this to make cool stuff happens. The sales pitch is clear: don't become a better programmer, get a better toolkit. I have been quite fortunate to have come into computers before this cloud of marketing madness overtook us. I got to watch the layers roll out one-by-one. Honestly I have no idea how I would learn programming if I had to do it again today. It's just too much of a byzantine mess. Hell if I would start with the top layers, though. I'd rather know how to write simple code in BASIC than have a grasp of setting up a CRUD website using WhizBang 4.7 When you learn programming, well you should be learning how to program. Not how to breeze through stuff. Breezing through stuff is great when you already know what's going on -- but by the time you already know what's going on, it's unlikely you'll need the layer or framework. (Sadly, the most likely scenario is that you've invented yet another framework and are busy evangelizing it to others.) This guy's story strikes me as poignant and indicative of where the industry is. They don't care if you can solve people's problems. They care if you're one of the cool kids. It's nerd signaling.

d3ckard 9y ago
I'd put it differently. Software development severely lacks any objective metrics of performance and quality. We just haven't invented any(at least any practical enough to become mainstream). As a result, we quite often misjudge our(and others') skill and make bad decisions. It's the goal of software to provide more and more features. The problem is, we usually achieve those features by abusing abstractions we are using. Once the problem becomes apparent, somebody writes a library, which allows us to write a couple of times more crappy code before everything collapses. Rinse and repeat. Since we are unable on scale to choose right abstractions when necessary, we quite often get ourselves in situations, when it's actually preferable to rewrite everything using lessons learnt. Rewriting comes with its own set of new problems and cycle is complete. As a result, we are fundamentally unable to reuse already written code on scale, crumbling development and putting the whole industry in constant early-stage.
- DanielBMarkham 9y ago
  >>It's the goal of software to provide more and more features. That's the goal. The key question is: are we writing features that users find valuable or are we writing features that developers find valuable? They're two different things. What's happened is that the community has become incestuous. Instead of focusing on value to the user we're focusing on selling frameworks to one another. >>Software development severely lacks any objective metrics of performance and quality. Agreed here as well. But this is related to my first point. Nobody wants to focus on the users. It's far too easy to focus on the technology (or other developers). You write a feature for a user, you can instrument it and see whether users use it or not. You write a feature for a developer, even if nobody uses it you can argue that somewhere, somehow, that feature is going to be critical. It's an abstract value argument -- which is intractable. >>As a result, we are fundamentally unable to reuse already written code on scale, crumbling development and putting the whole industry in constant early-stage. It's a sad state of affairs. I suggest that your conclusion continues the broken thinking I'm describing above. We shouldn't be striving for reusable code at scale. We should be striving for people who have strong basic programming skills and are experts in some business domain. Then we would shoot for the minimum amount of technological abstraction necessary for these programmers to solve problems in that particular domain. Because that's really what the whole point is, not creating large codebases that last twenty years, creating tiny bits of code almost instantly that solve business problems for twenty years. We've got our head stuck in the wrong bucket.

crispyambulance 9y ago

AFAIK, the guy has had a marvelous and productive career. When he says "...knowing Lisp destroyed my programming career" he just means that at some point in time he switched to other things. There was no "destruction". It was "a pivot"-- to use HN lingo. I think anyone who makes programming (let alone programming in a particular tool/language) the absolute focus of their career is in for a major disappointment. The OP was NOT crushed by his realization. He just moved on and appears to have been very successful regardless. Not a big deal unless one is obsessed with Lisp.

pjc50 9y ago

Maintaining a distinction between "actual software development" and "plumbing" is elitist, even if you're placing yourself on the downside of that comparison and setting yourself up for imposter syndrome. You can get an awful lot done by "plumbing". Entire businesses like SAP are built on it. It can also be mission critical; in SpaceX, is the literal plumbing of hydraulic fluid and fuel flow unimportant? No.

Angostura 9y ago
> "actual software development" and "plumbing" is elitist, Perhaps, but you are the person who prepended 'mere' in front of plumbing.
- pjc50 9y ago
  I'll take it out again if people are going to argue by string matching rather than reading the tone of the original post.
  - Angostura 9y ago
    It might be best.
golergka 9y ago
> Maintaining a distinction between "actual software development" and "plumbing" is elitist You are correct - however, it doesn't mean that this statement is necessarily wrong. Usually when I see a statement that reeks of elitism, I immediately assume lower probability of it being true, because elitism of a statement correlates with falsehood - but it's worth remembering that this correlation is not absolute, and sometimes (although rare), elitism is indeed deserved and true. And in this particular case, out of my personal professional experience, elitism is absolutely deserved. There are a lots and lots of software engineers out there that can't or just won't fit more complex ideas into their minds. And yet, you are of course, correct: these developers can, and will, do good work. They can have different skills, like knowing the users, or creating beautiful animations, or having a great game design sense. These developers don't need to be looked down upon. And yet it would be very useful for all of us to acknowledge that these developers think in a different way and require different tools - they're not "worse" than "real" engineers, but they are different.

shanxS 9y ago

for any given level we use abstractions, those abstractions maybe in form of algorithms or tools - so isn't everything plumbing?

wglb 9y ago

So let's think about this in a different way. Rather than classify programmers in buckets, e.g., plumber, non-plumber. Instead, the tasks of vast teams of programmers has grown to include a significantly richer environment. Skills beyond pure programming that we all need to do include connecting vast sets of existing environments together to build the next thing that we are working on. Few environments that we work on don't have networking, acres of existing APIs, already built ecosystems of data. In today's environment, 'pure programming' is a small fraction of what needs to get done. As a long-time programmer, the most leverage I have had is when my work connected to some business objective and produced a result. My engineering background, which emphasized a "problem-solving attitude" helps.

flarg 9y ago

As someone not even worthy of the status of a plumber, I quite agree. But the wonder of it all is that we can attend on the shoulders of giants so easily.

lr4444lr 9y ago

Your post resonates, but let's be fair: the business of a lot of the money in software now - users downloading deploys that change daily or more frequently, on multiple devices, with expectations to interoperate with so much other software - has really changed the incentives of what to focus on.

watwut 9y ago

Why do you think that writing those tools is harder? Oftentimes it is not. A lot of it is easy. Yes, well done abstractions are how we keep large projects manageable. No one can learn all of them at once. Neither those writing "plumbing" nor those doing libraries.

barrkel 9y ago

I reckon all software is plumbing. Plumbing as in taking data in, processing it, and pushing the data out. Doesn't matter if it's a database, a compiler, or a CRUD app; it's all just plumbing.

segmondy 9y ago
No, plumbing is taking prebuilt pieces like pipes and elbows, joints, valves and just putting them together. There are plenty of software that is not about taking pre-existing packages and gluing them together, but rather involves construction from the ground up. Building a CRUD app using an existing framework, and an existing rest API with little to no custom business logic is plumbing. Using sci-kit for your ML app is plumbing. Writing your own novel ML/deep learning algorithm, shoving the data in and out of GPUs is not plumbing.
- contingencies 9y ago
  Your distinction doesn't hold up to analysis: at some level any system can be said to consist of elbow joints - eg. "processor opcodes are just elbow joints." The same can be said not only of the subsystems but also the intellectual heritage of so-called "original" systems. Nothing and everything is new. News at nine. Someone watches the news. They create a new system ever so subtly influenced by yours. Ad infinitum.
- VonGallifrey 9y ago
  > Writing your own novel ML/deep learning algorithm, shoving the data in and out of GPUs is not plumbing. This is plumbing as well by your definition. Writing your own novel ML/deep learning algorithm also take "pre-existing packages" and glues them together. Unless you are actually going to write your own custom OS, Language and drivers you are going to reuse stuff. I don't think there is anything wrong with using pre-existing packages and the like. We all are doing that anyways unless your are Terry A. Davis off course.

kamaal 9y ago

Plumbing does get very complicated when your building size increases. And after a while its not just plumbing. Ensuring no pipe leaks in a skyscraper is no easy task.

YeGoblynQueenne 9y ago

>> Of course, as every programmer, I live in constant fear that I am part of the plumbing crowd waiting to be exposed. Well, the cheapest, safest assumption is that your skill level is about average and the majority of the programmers you 'll meet are going to have the same kind of skills as you.

segmondy 9y ago

I sort of agree, but I also disagree. I think 90% of us are capable of doing "actual software development." However the market does't care for that. The market prefers short term gains over long term gains. Perhaps we can blame wallstreet? Due to the demand for short term gains, the ask from most developers is "how fast can you build this" not "how can you build this to be most efficient and cheapest in the long run and lifetime of the product" To move quick, developers must then employ abstractions layered upon abstractions. The short term winners get lots of money for demonstrating wins quickly. The losers conform to keep up.

mywittyname 9y ago
> "how can you build this to be most efficient and cheapest in the long run and lifetime of the product" I couldn't answer this question accurately. I can't even give remotely accurate time estimates for projects larger than "build a CLI tool to do this one thing," much less give an informed estimate about the tool's TCO. I feel like I'm just floating down the river, incrementally improving upon the stuff that we've already built and that nothing we planned to accomplish ever is. I'm sure lots of people here have executed their Grand Vision for a project. But I'm also certain many of us never have.
piva00 9y ago
I don't agree with this view. Most jobs working with software development is about creating business value. Either as an end-user product that your customers are going to use or with internal tooling that will help the business have more access to data, streamline processes, increase efficiency, etc. This "no true Scotsman" approach to software development is actually quite funny after a while being a professional, it's a huge industry, there are terrible companies, there are terrible managers, product managers, etc., but there are also great ones. You can work for good ones if your current gig is mostly being pushed around by unreal expectations from your stakeholders. The demand is not for short-term gains, how can you justify that it's going to take a year to build your perfectly architected software if the business really needs that done in 3 months and you assess it's quite doable if you decide on some constraints? Your job as a professional engineer is to be able to find ways to do your work as best as possible given constraints, to design something to be improved on over time, to communicate with stakeholders and, given your area of expertise (software), give valuable input to the business decision so they can be the best as possible at that moment. Seeing software engineering as some grandiose goal by itself is quite wrong, software exists MOSTLY to fulfil business purposes. It's not about "conforming", there is software that is fun to work, that are intellectual challenges by themselves but that really have no way to be justified on a business level. This defensiveness against "business" is part of a mindset that should be broken among engineers, we should embrace we are part of a much larger process, not that we are the golden nugget and the top of the crop at a company. Our work is to enable others' work.

vijaybritto 9y ago

You seem to be experienced and ignorant at the same time

contingencies 9y ago
You can accurately attribute that quote to any external culture, re: another culture. Hunter-gatherers on capitalist democracy, Capitalist democracy on hunter-gatherers, China on India, India on China, US on India, India on Iran, Iran on Israel... You get the picture. Perhaps what it really means is "communications failure: exception thrown in cultural assumptions".

ubermonkey 9y ago

There's a real tension in this field between capital-C Computer Science and the work of writing code for businesses. The business doesn't care about your toolset, probably. What they care about is solving a problem. I've met no small number of very gifted, very creative developers that had this same mindset -- didn't give a crap about the Cool New Language or pure CS, but really DID get excited about building connections and features within existing systems to solve business problems. These two visions of development are in tension. Few folks can get jobs writing Haskell or Lisp, but there are LOTS of jobs for .NET developers. Neither path necessarily makes a better developer, though.

threatofrain 9y ago
It's like saying a doctor isn't doing interesting things if they're just content to apply medical knowledge to improve community health. The doctor is just using tools developed by other people.
- Bartweiss 9y ago
  I'd feel better about this if it weren't for the growing evidence that an awful lot of what we actually get is cargo-cult medicine - procedures that have been supplanted or invalidated, but are still widely used by non-research doctors who don't know better. Obviously doctors are still good, and don't need to be doing research to make the world a better place. I agree that dismissing 'plumbing' programmers or 'rote' doctors would be a serious mistake. But... well, I can't help drawing some connections between programmers implementing already-broken security, and doctors putting in heart stents that don't actually help patients. I don't think we're just being metaphorical here, I think research vs practice doctors often show the same patterns as programmers, for the same reasons. Creating new knowledge is neither necessary nor sufficient for keeping up with other people's knowledge, but we seem to be worryingly bad at keeping people who implement that knowledge up to date. (Context on the doctors: https://www.theatlantic.com/health/archive/2017/02/when-evidence-says-no-but-doctors-say-yes/517368/?single_page=true https://www.theatlantic.com/health/archive/2017/02/when-evid...)

pulsarpietro 9y ago

>Of course, as every programmer, I live in constant fear that I am part of the plumbing crowd waiting to be exposed. How true, the 50% of jobs I see are actually semi-skilled overpayed jobs which won't last. In many cases you configure somebody's else product and you manage it at best, it won't last. It is already happening.

ziikutv 9y ago

Any suggestions on how I can get into that 10%?

eropple 9y ago
Write more code. Stretch yourself. Learn how other people work and figure out how you can improve what they do, because that tends to be the bailiwick of what he's referring (mistakenly, IMO, but the bucket is probably fine even if the label isn't) as "real software development". I didn't write Auster[0] for me, I wrote it because other people needed a tool and it fit the parameters. I'm not writing Modern[1] for me, I'm writing it because there's a hole in the ecosystem that somebody has to solve, and I'm a someone. [0] - https://github.com/eropple/auster https://github.com/eropple/auster [1] - https://github.com/modern-project/modern-ruby https://github.com/modern-project/modern-ruby
d3ckard 9y ago
If I knew, I would do it myself and wouldn't be afraid ;) Probably the best advise is to learn as much as possible, but focus on basics. I feel people often mistake knowledge for skill and that harms them in the long run. If you learn a concept, you can use it in anything you create. If you learn a framework, you will have to learn a new one in 5 years. Focus on transferable skills.

jgh 9y ago

For the past four years I've architected, built, and maintained a system largely designed using the pipes and filters pattern. Does that make me Mario?

Waterluvian 9y ago

"actual software engineering" is like saying that you're not doing "actual farming" unless you're using some oxen and an iron plough.

agumonkey 9y ago

I agree 80% with this, I'll add something. I always had a lispy side, I hated Java, PHP, C and others prevalent languages that most companies would bet their money on, I like APL, I like Forth. That said I found something weird, sometimes, and with a little bit of adequate libraries (guava for instance) I enjoy doing some Java. It's verbose, way more than lisp, clojure, kotlin (not even mentionning mr Haskell of course). But I find a little pleasure in doing code there. It's manual, it requires doing a lot of things but even though its slower and more "work" it's another kind of stimulation, that I think is one large factor of people doing code in subpar languages. They're just happy doing things and solving things their own way. Some say that lisp and haskell can't be mainstream because their power is best fit for complex problems, and I think that hints to my previous point. People who need more than mainstream have the brain power and desire to solve non mainstream things. ps: about the plumbing thing, you might have heard that MIT switched to python exactly for that reason. I was and still am stumped that the people would brought SICP decided that plumbing was the way to go.

hoplite009 9y ago

As my professor used to say, Computer Scientists are merely merchants of abstraction. Layers of abstraction is what we all deal with, albeit at different levels.

rjhallsted 9y ago

So how does one make sure they're not plumbing?

jack6e 9y ago

It's easy to differentiate: real software developers use butterflies [0]. [0] https://www.xkcd.com/378/ https://www.xkcd.com/378/

astine 9y ago

I think that no matter where you are as a programmer, you're part of the plumbing crowd whether you like it or not. If you hack together websites using a web framework like Rails or whatever, then that web framework is likely your plumbing. If however you write a web framework, then the language (Ruby, etc) is your plumbing. Those who write languages have operating systems and those who write operating systems have hardware. Even hardware has different levels of abstraction in it from folks designing instruction sets down to folks who have to actually deal with the physics of circuits on silicon. The things is though, a lot of those folks wouldn't be able to deal with the stuff that a typical web developer deals with because they don't understand the domain. Writing a highly performant web server is a different task from writing a complete web application that uses that web server. I think that's it's useful to have a deeper understanding of the tools we use, but ultimately we all have to work at the level of abstraction that makes sense for the work we do and that's not a problem.

swat535 9y ago

Sigh.. I for one will never understand why everyone is obsessed about titles. Is doing front end not actual software development? What does that even mean? Are you considered a real software engineer if you can write code in a certain language? .. Or does it mean you are really good at O(1) problems? I think we should all agree that software development is a team effort, it requires vast knowledge and skills that different individuals bring to the table. Doing CSS adds just as much value as writing the backend API. Here is a good analogy. For an aircraft to function, pilots, aircraft mechanists, aerospace engineers all come together. Does that mean one is doing "plumbing"? Certainly not, they all add value.

Pokepokalypse 9y ago

The real problem is that the majority of us probably are actually capable of doing actual software development right when we come out of school. Most of us get slotted into crappy plumbing and middleware jobs, and after 5-10 years those skills get rusty. They can come back; but if you end up out on the job market, being asked some of the advanced questions in interviews. . . well good luck. That situation exposes one fairly quickly.

JustSomeNobody 9y ago

The 10% of developers you mention are the ones who don't cargo cult program. They don't seek "Neo" architectures. They are the ones who knuckle down and solve actual, real problems, not some fantasy generalized version of it. They are the ones who code and hone their craft. They don't spend any time arguing over React vs VueJs, Rust vs C++, etc. I think the average developer can do this too if they'd just get out of the programming echo chambers and trust their gut.