18 ms·

The Bulk of Software Engineering in 2018 Is Just Plumbing

• js8 8y ago
  I think the article is quite right, and David Graeber made a similar observation when he talks about bullshit jobs (companies would rather hire someone to tapeduct the system rather than redesign it). But I feel part of the problem is that the companies want to rather hire a plumber (I mean, software engineer) than to buy an off-the-shelf solution. It seems to me that off-the-shelf solutions are very costly and many companies think they can do it cheaper if they build it themselves. Which to me sounds like that the market is not functioning properly, for whatever reason.
  • maltalex 8y ago
    > But I feel part of the problem is that the companies want to rather hire a plumber (I mean, software engineer) than to buy an off-the-shelf solution. Consider Rosenberg’s Law: Software is easy to make, except when you want it to do something new. The corollary is, The only software that’s worth making is software that does something new.
    • kornish 8y ago
      Never heard this before, and a quick Google around doesn't turn up anything - what's the origin of this? I dig it.
      • maltalex 8y ago
        It's from Dreaming in Code [0] by Scott Rosenberg. [0]: https://www.amazon.com/Dreaming-Code-Programmers-Transcendent-Software/dp/1400082471 https://www.amazon.com/Dreaming-Code-Programmers-Transcenden...
        • kornish 8y ago
          Right on — thanks!
  • jstimpfle 8y ago
    In many cases, for an off-the-shelf solution to work for you, you need control over it. You need to be able to change it. So it is required to be open source. I guess most off-the-shelf solutions aren't. Also, to be able to change the solution, you need to find one with just about the required feature set. In other words, you need a solution with minimal complexity that fulfills your needs. Because otherwise changing quickly becomes too painful. On top of that, trying existing solutions and finding out why they don't work for you is not fun. It's much more fun to explore the solution space on your own.
    • Bahamut 8y ago
      Open source isn’t a silver bullet - if the library isn’t updated frequently, or has some bugs in design, it can be tough to get them fixed. Maintainers tend to be very protective of the codebases they steward too.
      • bigger_cheese 8y ago
        Support is a big thing as well for business software. When you buy "off the shelf" package from commercial vendor a large chunk of the cost is usually in support contracts. If you have a bespoke system built from open source components it can be difficult to find people able to support it. I think there are pros and cons to both approaches. Sometimes it can be frustrating trying to shoehorn the vendor supplied peg into your business's particular hole. I've heard a lot of people over the years complain to my manager "XYZ is crap I could write a better solution using ABC" the reply is usually along the lines of "Yeah but are you willing to spent the next 10 years supporting it."
      • jdbernard 8y ago
        Yes, I agree with GP that open source is often necessary. I agree with you that it is also not always sufficient. So companies do what they can to have the necessary control and also sufficient support: build it themselves.
    • wolco 8y ago
      I see this a lot. Large companies buying large expensive solutions that do very little of what they need. Many would be better off building themselves. Smaller companies leveraging existing products makes more sense in terms of budget and team size.
  • swills 8y ago
    > many companies think they can do it cheaper if they build it themselves. Sometimes it seems this is because someone who is more interested in building it themselves than doing more mundane work tells them so. They have no way to know for sure if it will be, and often it isn't, but the prospect of cheaper sounds good.
  • dyeje 8y ago
    This is exactly the trap companies fall into. They see a solution ready to buy and think, "Oh we can build that ourselves for cheaper." The thing is, they can't. I worked in a niche of the medical industry and we saw this so much we made a white paper on it. The number of times people decided to build something themselves and came back to us in a year was amusing. They don't realize: 1. The amount of man hours necessary to build a reliable, effective system. 2. The expertise and insight needed to make it actually solve the problem. 3. The ongoing costs of maintenance, adding features, and support. So, in my opinion, it's not that the market isn't functioning it's just that most companies are delusional about what it costs to build a complex software tool.
    • F_J_H 8y ago
      In the area I'm most family with, (business applications), many, many systems/applications are basically relatively simple CRUD apps with reporting and dashboard features, and maybe a bit of work flow management. With the rise of low-code tools/platforms, building such apps is almost trivial. So, the make versus buy decision is not as straightforward as it once might have been, especially for companies who already might have developers on staff. *Edit: typo and clarity
    • pbecotte 8y ago
      Well...yes and no. For example, page scraping is something I have been pricing recently. The companies I have spoken to cost WAY more than it would cost me to build...because I need one simple solution with limited accuracy...but their company is built around an advanced high accuracy solution. The off the shelf thing is rarely the simplest thing possible, because once they built that they had to keep adding features and engineers and costs :)
• laogai 8y ago
  To use the plumbing analogy, even if most or all of the job will be maintenance, you still want engineers who knows the internals enough to build their own “plumbing” if/when needed. Maybe an apt comparison is how astronauts are trained heavily for the 1% of off script situations that might occur even if it doesn’t happen in the span of their careers. Companies should feel able to innovate when appropriate by having hired engineers capable of doing so.
  • adamsea 8y ago
    Not to knock this comment, which makes a valid point, but I couldn’t help but smile at the implicit comparison of software engineers to astronauts when the article is preaching the value of developers thinking of themselves as the “humble plumber”.
    • TeMPOraL 8y ago
      You can never have people who actually build stuff think too highly of themselves, lest they start thinking about whether what they're building is actually useful or helpful at all...
      • falcolas 8y ago
        Or that what they're building has value - otherwise they'd want a piece of that value in addition to having high self esteem.
    • dgreensp 8y ago
      The complexity, reliability, and user-friendliness (or technician-friendliness) of the kinds of software components you need to plug together to make a software solution are more like that of a space shuttle or fighter jet than, say, a car, let alone a plumbing fixture. The glamour (and risk to life and limb) of doing so is that of plumbing.
  • wmccullough 8y ago
    I’m not fully willing to accept the plumbing analogy myself. While I do agree that most of our work is CRUD over REST, I think there’s another element of skill that isn’t uncovered in the article. If we are humble plumbers, why does it take me six months to find engineers that understand basic concepts like cohesion and loose coupling beyond a text book level (and I live in a metropolitan area with several software schools!)? Perhaps a more apt analogy is that we are more like home renovation specialists. If things go well, we can update your plumbing to get away from lead pipes, but if we have to route the new plumbing through a joist, we are going to have to educate you on why we should do something a certain way. If we find a water leak and rot that will put your home (business) as risk, count on us to tell you about the risk and let you decide if you want to address it. I’d argue that we are simply doing plumbing when things are going really well, perhaps even when the house is small and was well built to start with. If plumbing was all we did, I’d argue that our field would be way more saturated than it is.
• mborch 8y ago
  He wants to make that case that software development is a more pedestrian affair than it might seem, without much of an argument. But it's not true. Even a small shop needs to live up to pretty much a global standard, not least in terms of security. In the EU you must engineer with a security perspective in mind. The field is constantly moving. People come and go faster than most other industries. I think there's an analogy to getting a better bike: it doesn't get easier, you just go faster.
  • mannykannot 8y ago
    Security is a valid counter-example, but, significantly, we are (collectively) very bad at it in practice, repeatedly re-implementing the same old mistakes in each new technology.
  • qaq 8y ago
    Working at security related company even Fortune 500 are not really living up to "global standard" in terms of security. Smaller shops can't afford proper security solutions and def. can't afford a competent SOC.
  • betterworldb 8y ago
    I recently got a better bicycle and it got easier and faster. Well lubricated bottom brackets and a new chain makes a massive difference in speed / resistance.
• amelius 8y ago
  Yeah, in a way it is the curse of "open source": there are so many libraries and tools freely available that the only problem left is to tie them together.
• segmondy 8y ago
  This is a good way to offend most developers since most like to see themselves as rocket scientists instead of plumbers. But I have to agree, it's MOSTLY plumbing. Thus the reason we see people argue, "Don't make me leetcode during an interview bro! Who ever implements their own algorithm these days?"
  • IshKebab 8y ago
    I asked that during my last interview but it turns out the job actually is a lot of new algorithm development, which is fun and challenging.
    • Apocryphon 8y ago
      Lucky! What kind of role is it?
      • IshKebab 8y ago
        Helping to write a sort of compiler for a new chip architecture. The previous job I had (I've only had two programming jobs - used to do engineering) also wasn't CRUD - it wasn't as algorithmic but it did involve lots of low level USB programming which was interesting. Are non-CRUD jobs really that rare? Surely if you don't want a CRUD job just find a company that wouldn't need to write any...
  • sidlls 8y ago
    I agree that most development doesn't require sophisticated research quality abilities. But that has little to do with what you're talking about. Leetcode style questions are valuable for testing a recent graduate on his textbook knowledge, not for what you imply.
• inertiatic 8y ago
  >"This isn’t a popular opinion among software engineers, but I believe that many in the field are overpaid relative to the difficulty of the work they do." But no one is paid relative to the difficulty of their job, dummy!
  • TeMPOraL 8y ago
    Or actual importance. It's always a combination of what salary you can command (function of your soft skills) and how much money you can make the person paying you (function of market dynamics).
    • petra 8y ago
      Good points, but it's also about how much would your replacement cost.
  • tootie 8y ago
    People should be paid relative to their accountability.
  • arkis22 8y ago
    And if people were paid relative to the difficulty of their job no one would have an incentive to make things simple or easy.
• nobody271 8y ago
  The writing has been on the wall for at least a decade. I remember hearing about how programmers would be more like plumbers all the way back in 2008. Sure enough, that really has come true for a lot of developers. But I agree that this is good. The job is just changing. For example, maybe you wrote a custom DAL back in 2010. Now what you likely have is a mess that you have to pay several people a senior salary for because they are the only ones who understand how it works. Meanwhile a general purpose plug and play solution like entity framework is ubiquitously understood among .net developers. So now we don't need those "senior" DAL guys simply because they understand the mess they helped create a decade ago (also, about the last time they read any books). I see it as more a levels of software development thing. low-level > medium-level > high-level > plumbing.
• ken 8y ago
  > I believe that many in the field are overpaid relative to the difficulty of the work they do. I believe that every programmer is overpaid relative to the difficulty of the work they do -- or, more accurately, that The Market doesn't pay based on difficulty of work. Software pays so well because the product scales so well. Successful pop musicians don't do work that's 100 times more 'difficult' than software engineering. They do work that scales even better (copying digital music; playing to arenas; branding on merch). Having worked outside the software world, I absolutely do not believe that software people are any smarter, on average, than anyone else. Ever see a plumbing or electrical or structural system fail spectacularly? Other types of workers absolutely need to understand interactions between multiple complex systems, deal with obsolete and incompatible systems, and deal with changing and conflicting requirements and regulations. If software is any more complex to deal with than physical systems, it's only because the architects and implementors let it get that way.
  • RobertDeNiro 8y ago
    Keep in mind that it's easy to you because you're from that particular field. For a lot of people software development is still basically witchcraft.
    • cbluth 8y ago
      "In obscurity lies opportunity"
      • booleandilemma 8y ago
        Job security by obscurity? :)
    • c22 8y ago
      So is plumbing.
      • minor3rd 8y ago
        I had a plumber over for some fixes about a month ago, and he saw me doing some coding on my laptop. He made a comment like "I have absolutely no clue how any of that works... it is wizardry to me". I said "I'm watching you fix plumbing issues around my house, and that is wizardry to me". I think we could probably learn each others' jobs if we really tried, but the point is that all things can seem like magic until you learn them.
• Daishiman 8y ago
  "Just Plumbing" is not really an apt analogy. Because, judging from all the people I've been working with throughout the years, it's amazingly difficult to write a clean, cohesive, maintainable, scalable CRUD app that will actually satisfy your customers' demands. Making things that work is easy. Making things that work well, that's another story. And it's the reason why there's so many open spots and high salaries for senior web devs who can deliver.
  • stevehiehn 8y ago
    Installing a toilet is easy. Planning/creating the infastructure for a large building complex is another story :)
• dilatedmind 8y ago
  my friend is a pipefitter in chicago, installing piping in a new 60 story building. plumbers make more than software engineers, when they have work to do.
• danharaj 8y ago
  If what we do is plumbing, then what we're pumping is salt water. Some of us get to build better pipes. Software in 2018 is better built than in 2008 for a lot of sectors. In 2028 I expect our pipes to be more robust. I also expect a single engineer to be able to do the plumbing 5 developers do now. At least with the right technologies.
• funfunfunction 8y ago
  This article has an interesting point. I think the corollary to the point made is that web software, which is what is providing most of the plumber-type jobs this article references, has reached a point of maturation where certain solutions have been accepted as best practices. This lowers costs for businesses and lowers the barrier to entry of the web software industry, which I think of as generally good, but it also cements the positions of certain organizations that are foundational to the infrastructure on which these best practices are built. The saddest part about this for me is the realization that there is little room left for truly innovated web technologies that have real impact to businesses. Of course there are counter-examples to this (I would count WebRTC, WebSub, and a few others as having potential to impact some usecase specific best practices) but the majority of new software I see being hyped is either a new implementation of an old idea (React and Vue for FRP on the front end) or something touting decentralizing as the next major step which so far hasn't proved itself valuable in a commercial sense.
  • AznHisoka 8y ago
    It also doesnt help that building valuable software these days requires lots of data, especially if you are doing any machine learning. And most startups lack any data. its all owned by the monopolies of Google, facebook, Amazon and Apple. So the rich will just get richer, and the chances of startups succeeding in any field is as low as ever.
• notacoward 8y ago
  Makes me glad I work in a specialty (data storage) which provides the parts and tools the plumbers need. Sure, there's a shared grab-bag of techniques etc. that we all use, but it's far from plug and play. While the market for people like me will never grow anywhere near as big as the front-end plumbing stuff, demand and pay have been much more consistently good for the last couple of decades and are likely to remain so for a while. My advice to new software engineers is to get into one of the more "boring" specialties. You might not be in the lottery for the really big bucks, but you probably weren't going to win anyway. There will still be plenty of interesting and fun challenges, with greater stability and (TBH IMO) more mature coworkers.
  • jaiprabhu 8y ago
    Genuinely curious - can you give me some examples of these specialities where one can go deeper? In am in this boat now where I have worked on pretty much on all layers of the software stack and I want to now narrow down my focus.
    • notacoward 8y ago
      Some of these map to classes you might have taken in college - compilers, databases, AI, etc. I'd divide operating systems into more than one part: * Core kernel (schedulers, memory management, virtualization) * Drivers * Filesystems * Networking Then there are some specialties that cut across more than one of these - high performance computing, financial tech (not even counting blockchain stuff), security engineering, performance engineering. Then there's a whole world of embedded stuff, from teeny-tiny microcontrollers up to high-performance medical imaging or genomics, plus large distributed systems to control things like military equipment or nuclear power plants. Really, there are so many different kinds of computing that we just never even hear of on a site like this. It's really the web/app/mobile front-end stuff that's a niche technology-wise. (On my less charitable days I'd call it a ghetto.) It's a hugely profitable niche, to be sure, but to me it still seems to get a disproportionate share of attention and funding compared to other stuff that's just as hard, just as fun, and just as important.
• tootie 8y ago
  For most projects I've worked on in the past 10 years, the energy has majorly shifted towards front-end development. Building blocks have improved dramatically, but expectations have been rising even faster. Backends are getting more and more homogenized. There's still a fair amount of work that goes into accurately implementing business rules and a much bigger focus on quality and maintainability than ever before. But there's not a ton of serious thinking involved. I can pretty much follow a script for implementing microservices that will scale up to millions of users. I somehow ended up with a high-paying enterprise architect job and when they ask how we can build a system that will have 1000 users, the answer is that you can do pretty much whatever the hell you want so long as the framework is less than 5 years old and it will be fine. Then I go get a coffee.
  • StaticRedux 8y ago
    Not to be snarky, but this sounds like the viewpoint of a front end developer. One I disagree with. It would be like someone saying front end developers are unnecessary bc anyone can spin up a WordPress site and then go get a coffee. Can you get a backend going for a thousand users fairly quickly and easily? Sure. But that's the WordPress equivalent of describing back end development. If you have any kind of process based workflow with varying parameters, integration with hardware, detailed reporting / analytics, hell, if you even deal with money or dates/ times / timezones, which is any non-trivial application, backend development gets real complex real fast. It may be trivial to spin up a WordPress site, but it isn't too put together a solid, reliable React / Redux PWA with isomorphic rendering and offline support. It may be trivial to serve a few API endpoints straight out of the database, but it isn't when you have workflows and processes and states and have to deal with shit like GDPR. > But there's not a ton of serious thinking involved. Give a fucking break. The only way this could ever be true is if you only have experience working on extremely trivial applications with no complex functionality.
    • tootie 8y ago
      I'm strictly backend and I've myself becoming less and less useful. Complex requirements are one thing, but implementations just require diligence, not aptitude. I've built loads of apps and sites. Never needed a data structure or algo that wasn't in a standard lib. Most complexity is in integration. Most of my co-workers have no formal CS education and are at no obvious disadvantage.
• ramoz 8y ago
  If devs see their jobs as "plumbing" then that is a problem with their current culture & environment. Most devs should be focused on delivering business/mission value. This means custom code, unique ip, algos, GUIs, etc. However, the current state of devops has thrown all sorts of engineering challenges at devs that lose sight of the actual value that should be continuously delivered. It becomes plumbing when we get caught up in redundant tasks such as dependency management, tests, logging, containers, runtimes, integrations, cloud provider specifics for scaling/ha/sre, and connecting them all to create "the platform" that is supposed to enable true continuous delivery of value. We end up more focused on this internal platform development even though the tooling is pretty advanced. We still have to configure/orchestrate/manage that advanced tooling, and this, in turn, creates the plumping effect. We greatly underestimate the work involved with all of these things. It is not plumbing when we are focused solely on the continuous delivery of value. We remain lean and agile, constantly testing a hypothesis and developing our "uniqueness".
  • matte_black 8y ago
    Very few developers deliver direct business value or revenue. For every rockstar dev who gets paid a lot of money to make unique solutions and use his brain, there’s dozens of unsung heroes who do the grunt work reading from docs and setting up plumbing so the rockstar can deliver his value effectively. These developers are not working in cool languages or solving new problems, and many of them will never do so in their life. They are just expendable workers swinging hammers and writing JavaScript.
    • ramoz 8y ago
      But isn't it a problem that most devs are setting up the plumbing vs already being enabled to deliver rockstar/valuable solutions? i.e I've worked at a large enterprise and we all (product devs & ops) became so focused on our grand platform (continuous delivery, heavy integration with AWS, container orch, all using OSS) that product development stalled. Though we were doing "rockstar" things with cool tech and our resumes have a bunch of cool words, but it's true that most of that work was plumbing. In the end that tooling became a bch to manage and continually developing upon. The product continues to stall. Wouldn't it be better for all if we weren't so caught up in the plumbing of these advance/cool tools? -- that's the thought that actually leads into something like building your own platform, vs something provided. Ops then become more like an overseer sre type of role, and most engineering shifts back to direct value.
• alexanderdmitri 8y ago
  When I was in college, one summer I worked up in Vermont putting together and maintaining trailer park homes. Plumbing was one of the best parts of the jobs in my opinion. I'd wager the person who wrote this article has never had to put a plumbing system together from scratch. There is an art and a science to it. The guy I worked with put a lot of thought into the best way to design these things, often because he'd be the one who would then have to maintain them. He'd also get very upset when we go out to a trailer with a poor plumbing system. The main giveaway would be a mess of pipes going every which way without rhyme or reason. These systems were the ones that broke the most and were also the hardest to fix. It might just be "plumbing" to someone that doesn't have to do it, but there is a whole lot that goes into it and you don't want a plumber who approaches his/her work with the catch phrase: "Well, it's just plumbing."
  • cousin_it 8y ago
    You just made me feel inspired in a new way about the job I've been doing for many years. Thank you!
  • kashyapc 8y ago
    There's a reason why parts of the Linux community has an event named: The Linux Plumbers Conference :-) https://linuxplumbersconf.org https://linuxplumbersconf.org A reputed community conference, co-organized by contributors and maintainers of the Linux kernel and related low-level "plumbing" utilities and libraries.
  • rootusrootus 8y ago
    And plumbers get paid pretty well, just like programmers. I talked to the plumber who did my house and he loved his job. He was reluctantly retiring and handing over the day-to-day operations to his son, but if he had been physically able he'd keep doing it. He also said he refused to do work on existing plumbing. He loved to build new systems but hated maintaining old ones. Where have I heard that before...?
    • chiefalchemist 8y ago
      In a previous professional life I hired an electrician to put in some outlets in my warehouse. Long to short, he said he mainly worked solo (read: didn't want to deal with employee bullshit), and tended to work for 4 to 6 weeks and then go on vacation with his wife for a week. I think it's the only time I said "THAT is the kinda setup I want."
• matchagaucho 8y ago
  Kind of a cyclical theme... but this is why the job titles of "Developer" and "Engineer" were created. "Engineers" design/build components that perform with repeatable results. "Developers" work out the plumbing details between components in a system. When hiring, don't mislead people into thinking they're interviewing for an Engineering role when you actually need a Developer.
  • jxub 8y ago
    I think people need to stop seeing software as punching coloured characters on a screen and start considering it as an occupation that varies widely in skills, wages, and status and distinguish between the blue collar "plumbing" and white collar "architecting" parts, with a smooth array of options between both. This, of course, is hard if you don't program and it's not really meaningful to evangelise about, so I'll leave off this rant here.
    • matchagaucho 8y ago
      For a recent home remodeling project I paid an Architect $90 hr and the Plumber $130. There's nothing diminishing about being a Plumber. "Software is eating the world" means there is more demand for developing systems than designing them.
      • jxub 8y ago
        Yes, I guess developing/architecting are largely orthogonal to paycheck, it's more about the current market forces than the position in this scale. However, our lizard brain status system still associates directing and leading things with more status than just requirements implementation and there is no way around that yet. Also, less demand means that this more "abstract thought" type of positions are more prestigious by the virtue of scarcity.
  • ccccccccccccc 8y ago
    eh, i don't know about that one. In most countries outside of America, "engineer" is a reserved word for licensed engineers and Software Engineers can not legally call themselves engineers, thus developers.
• falcolas 8y ago
  I have to be honest - I don't see this. Maybe I'm lucky. My current company is basically creating Microsoft Office on the web - because for the targeted niche, Microsoft Office (Google sheets, Libre office) wasn't cutting it. There aren't libraries for that. My previous company was applying machine learning to the twitter firehose, and present the results in real time. The existing libraries weren't up to the task (or the volume). Before that, we built tooling to allow ten people to administrate 40 plus companies' on-premise databases. There were a few libraries and tools we used, but nothing that we could just "plumb together". I saw in the Reddit thread that the "plumbing" argument applies mostly to backend developers who just put REST libraries on top of databases. OK, I can kinda see that, but it's not like the complexity of the programs has disappeared, it was just moved to the frontend. And in a way, that's a benefit to the company (and the employees) - the resources previously tied up in re-creating REST libraries before REST libraries existed can now do other things. The value of programmers has never been in the movement of data from point A to point B, it's been in the application of business logic to that data.
  • drb91 8y ago
    > There aren't libraries for that. Sure there is—textarea and http. The rest is just plumbing. I’m only half-joking. I just don’t see a huge difference in difficulty between business logic and plumbing—they’re two sides of laying out the program, and neither one is more difficult than the other. Both lead to complexity and confusion if you lay out poorly. I think that if you are good enough at googling, the vast majority of problems businesses are solving have already been solved (or the problems are well-mapped) and you just need to adapt the successful patterns to your domain.
• thegabez 8y ago
  If software started to become as reliable as PVC piping and standardized and regulated by a governing body like it is with plumbing, then all your problems are solved. Until then, keep learning and make sure your engineers aren't over-engineering the crap out of everything.
• okket 8y ago
  "just plumbing"
• quotemstr 8y ago
  The transformation of the software profession into law is progressing. As we engineer most moderate-difficulty tasks away, what remains is the trivial or the challenging, and these two classes of task correspond to the peaks of the new bimodal salary distribution. We have a glut of people in the field capable only of the former class of task, so that peak is going to rather low. This dynamic isn't speculative. It's _exactly_ what happened to law. It's what happens to any field that goes through a boom and attracts a certain kind of person who's more motivated by immediate financial reward than talent or passion.
  • arwhatever 8y ago
    What exactly did happen in the field of law? Split into lawyers / paralegals?
• lgleason 8y ago
  Beyond the ludicrous nature of the authors sttatement, a look at his Linkedin shows that he is a non-technical CTO. IE: A CTO who has has never really worked as an engineer for an extended period of time and din't come from that background. Thus, consider the source and move on...
• chiefalchemist 8y ago
  But it's not just tech / engineer hiring that has this problem. Add on a manager that has peter principle'd out and there are plenty of opportunity to talent mismatches. That aside, the truth is, the vast majority of what's asked to be done has been done before. Even the particlars are likely to be wheel reinvention. Mind you, it hasn't happen yet, but an object driven drag & drop (or VR) driven solution / application builder is only a matter of time. Splash on some AI and the role of engineers not in the top percentile is going to trend to zero
  • chrisco255 8y ago
    Visual drag and drop application builders have existed for some time. The problem is that as soon as your app needs to go "off rails" which all apps do at some point, you won't have the skills or knowledge to tweak your program with some visual editor tool. And I don't see AI doing anything remotely close to solving for that. You would need to be able to communicate with an AI on a deep level of understanding of business requirements and the AI would also need to understand all the layers of the software stack to be able to produce code with AI that even rivals average individuals. That's something akin to general AI and we are a long ways off if it's even possible.
    • chiefalchemist 8y ago
      Yes. There have been attempts. I see AI as being the collection and resoltion tool. Imaging GitHub with a brain. The current problem is far too much wheel reinvention. Far too much language specific solutions when - ultimately - the language doesn't matter.
• simplecomplex 8y ago
  Bullshit. Don’t trivialize software development or plumbing. Even a blog is not simple at all. Case in point: there’s numerous bugs on his own website! For example, on my mobile browser his “subscribe” button text is cutoff and misaligned from poor CSS work. But it’s “just plumbing” right?
• focal-point 8y ago
  "Just plumbing" which requires at times detailed detective work and/or the ability to prove that many invariants hold, all while communicating effectively with several people of different backgrounds.
• xkcdefgh 8y ago
  My employer used to have devs from service based companies working on contracts. Later they started cutting the amount of vendors and hiring good talent with attractive wages. The amount of stupid code I encountered when I joined was ridiculous, now that we have smarter engineers over here, everything works like a breeze.
• arcticbull 8y ago
  100%, we're Dell - we grab the bits from the different manufacturers, design a motherboard that connects it all in a way that ideally keeps them from falling down unexpectedly. Sometimes we make a custom doohickey. Probably not though. And if we do, maybe we shouldn't. That is, until you get to a certain scale. As you get bigger and bigger, it makes more and more sense to roll your own of everything to squeeze that last little bit of situational performance.
• plesiv 8y ago
  Blunt generalization incoming... it's naive to read this article (similar to the majority of others) as an info-piece. It's a marketing piece, any informational value is a by-product.
• whatshisface 8y ago
  Isn't this a description of what IT people and operators are supposed to do? "A bunch of services, set up with a little glue and a lot of configuration," is exactly what ops is about. Customarily, programmers run the show and tell the operators what's needed, but perhaps for a small business it would be best for an operator to act as "software concierge," arranging for new software to be engineered only when necessary or helpful.
  • mmt 8y ago
    The "concierge" idea is a new one that I hadn't heard before. It helps describe a piece that I feel is missing from what it is I feel that I do, because, so often, perhaps because the terms "operations" or "system administrator" don't have the word "software" in them imply that I don't understand it or can't apply engineering to it. Although I don't feel comfortable going as far as dictating the outcome of some version of the build vs buy decision, I agree that, when I happen to be on staff, tremendous value is lost by not at least consulting me as to what "buy" [1] alternatives are available. As an experienced technology (not just software!) concierge, I often know even more than one. Unfortunately, by the time I'm hired, it's almost always too late. Especially at modern startups, programmers very much run the show, so the focus is very much on writing new software. That said, I don't think this notion is unique to ops engineers. It can be extended to programmers, as well, in that very senior ones tend to follow similar models, of simpicity, code reuse, libraries, and, ultimately, eschewing actually writing (especially rewriting) new software unless necessary. [1] quotes because it's often free-gratis, merely not custom-written there
• stakhanov 8y ago
  ...that argument has been around for as long as computing itself, and it's no more true today than it was at any point in history. If the microcosm you're in is CRUD systems, that argument may be valid, but I think the author is overgeneralizing if he equates CRUD to the sum total of everything there is to Software Engineering. There is a constant lifecycle in this: A new idea is born or first becomes technologically possible. At that point it's very difficult to put the idea into reality, but the demand is so pressing that the market will pay whatever it takes for individuals to live up to the challenge. Then, gradually, it becomes commoditized and, eventually, everything that's left to do is "plumbing". ...but at that point, a new idea will have been born or will have become technologically possible for the first time ELSEWHERE, and the REAL MEN will have moved on to that (leaving behind only the QUICHE EATERS to do the plumbing work). So, if you've been in the field for a long time, your career path might have looked like: Writing CRUD systems on IBM host systems in the 80s, when that was still an intellectually demanding job, because you wouldn't have learned programming at university as programming wasn't yet a thing, but you still managed to learn what bytestreams to send down some line to make an IBM terminal show a number in red or green, and you did it in PL/2, a programming language that hadn't yet had a chance to learn the lessons learned from several decades of software systems gone wrong. Writing some of the first pieces of internet banking software in the 90s. Your employer, the bank, wanted to make damn sure they were not going to make fools of themselves by using this new and unproven technology in a way that would allow funds to be misappropriated. So they paid you handsomely to learn everything you could about telecommunication technology, security engineering, encryption, etc because you actually had to manually code all that as there were no frameworks or libraries for that just yet. Doing algorithmic trading software in the 00s. Everyone else was ripping databases out of their systems because XML was the future, and the XML was processed by a java programme to run in a virtual machine to run in a container to run in a virtual machine to run on god knows what. Since you still knew how to do real programming for real machines in a way that didn't involve converting your data to XML and back, your software executed orders to buy and sell equity on the world's major exchanges faster than anyone else's, and you were paid handsomely for it! Doing data science & data engineering in the 10s. Organizations had been run for several decades by numbers-driven people whose computing competence only reached as far as Excel. The numbers-driven mindset did not go away. But the data was now there in such quantities, and stored in such nonsensical ways, that everyone who was previously an Excel-guy now needed one or two of those data engineers at their side. Also, because it now became possible to use the data to answer some pretty complex questions, those questions were now seriously being asked for the first time, and everyone discovered that they had been asleep at university during the lecture where someone explained what a standard deviation was. So that's why you learned everything you could about statistics, machine-learning etc etc. You applied it, and you got paid very well. So, what really sets you apart if you're not just a software engineer, but a GOOD and SUCCESSFUL one, is that the stuff you're working on is always the stuff that's NOT YET plumbing, because it's a major force right here and now despite the fact that nobody has seen it coming. ...I don't know what the future holds, but I think I'll be well equipped to deal with it, because THAT's what real software engineers do and there's no framework to do THAT. On a side-note: Stack Overflow developer survey results routinely indicate that like half of the nation of software developers have been writing code for a ridiculously small number of years (like two? don't remember the number now). If you're one of them: The above argument does not include you. You're not a software engineer yet in the sense of the argument that I'm making, above. But don't feel bad about that. Next year, you'll probably switch to sales or become the pointy-haired boss of some REAL software engineers.
• learc83 8y ago
  From looking at the Author's linkedin, it doesn't look like he's had much experience working as a software dev. Funny, a CTO thinks programmers are overpaid and our work isn't really that hard. Who would have thought?
  • dang 8y ago
    Please don't get personal. Even if you're right—and the odds are you're not—the damage it does to the container here is greater than any information it adds. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    • learc83 8y ago
      I don't think pointing out that there is a conflict of interest by the author of a public marketing piece is getting personal. If Mark Zuckerberg posts something to the effect of: "guys what we do isn't that hard, we should probably get paid less", I'd call him out on it too and I hope other people would as well. I certainly wouldn't consider that a personal attack.
      • dang 8y ago
        The dude wrote a blog post. You're welcome to respond to what he said in the post. Digging up personal details on the internet to use against him is not cool. Please don't do it again.
        • learc83 8y ago
          I didn't dig up anything. His linkedin profile is the 3rd link at the top of the post before the content. There's a signup for a newsletter-- also before the content. The whole blog is marketing for his CTO advice company. If someome sets up a marketing page, sells themself as an expert, and then posts a link to a resume backing up that expertise, it's fair game to click the link to the resume and verify.
• tomohawk 8y ago
  If you want to be better at building software, pick up one or more of the building trades. There is much to learn about building things from electricians, plumbers, carpenters, and masons. Working with tradesmen and seeing how teams of tradesmen work can be enlightening. I can't believe this guy said "just plumbing".
• kpwagner 8y ago
  These comments crack me up. Hacker News readers seem to have a lot of opinions about plumbing, the plumbing trade, the difficulty of plumbing tasks both intellectually and physically, pay scales of plumbers, personal experiences with plumbers, etc. Did you get that "plumbing" was just, like, an analogy? The article wasn't really about plumbing.
  • RichardCA 8y ago
    It resonates for a reason. In plumbing everyone understands the difference between what's theoretical (flow rate equations, hydrodynamics, metallurgy) and what's practical (nearly everything else). Computer science has always had this tension between being an actual "hard" science and an applied engineering discipline. A lot of otherwise intelligent people pretend to understand that distinction when they really don't and it leads to no end of problems.
    • wgerard 8y ago
      > Computer science has always had this tension between being an actual "hard" science and an applied engineering discipline I actually would go much further: I would say that the tension is between an applied engineering discipline and a vocational degree ala, well, plumbing. I don't think there's much question that fields like quantum computing belong more in the "applied engineering" category (or even the "hard science" category). There's much more question about whether professional software development has any significant relation to the field of Computer Science other than the bare fundamentals. My hunch in the future is that we'll see the field bifurcate into two very different professions, ala EE and electricians: "Computer Engineering" (or whatever it's called) will become a much more limited field that is overseen by the NSPE and has a licensure system, and "Software Development" will become a vocational degree (if that, even) and represent the vast majority of software development jobs we see today. That obviously has problems, but I see it as an inevitable compromise that might eventually help resolve questions like, "this set of customer personal data was leaked because of a known but unpatched vulnerability, who signed off on this software release?" and also relives a lot of the pressure on regular software developers to understand these fundamental CS concepts that don't come up much in practice.
• mongol 8y ago
  The bulk of IT is certainly data plumbing. The question is what the engineering part is of the total. Compare with physcial logistics, you have the lorry drivers, the mechanics, and the engineers that develop the next lorry model. There will always be a need for tools and algorithms to work on the data in innovative ways. That is probably where the engineering skills are the most essential. But these jobs are likely fewer than there are people wanting to think of themselves as software engineers.
• sonnyblarney 8y ago
  "The Bulk of Software Engineering in 1XXX is Just Plumbing" When was it not? What were these guys thinking? The vast majority of software is taking real world applications and dealing with the 'kind of a mess' of situations that can arise. Even login/credentials can be a minor pain. Doing it clearly and succinctly is the challenge. Though it's often more 'solving problems' than 'building something' we should maybe think of ourselves more as construction workers than research scientists. Upon graduation my skills were filled with so many holes I suggest that every CS grad takes a 3-month long 'power course' wherein the get all the basics, best practices and common tooling down.
• jorblumesea 8y ago
  Engineers are paid to solve business problems, not just "code things" and "create products". I am always perplexed as to how engineers see their own field. You are not there to create some novel framework, unless the situation absolutely calls for it.
• booleandilemma 8y ago
  Be careful with the word “just”. I’ve found that people who use it often are blissfully unaware of the details of things and the work involved.
• Animats 8y ago
  Overcomplication is sometimes an issue. Remember Soylent, the hipster's Ensure? They were always talking about their overdesigned server infrastructure, even though you could calculate from their sales volume that they did only a few transactions per minute. They could have used low-end shared hosting and CGI to handle that. But no, they had to be a "tech" company. (It looks like they've finally accepted that they're in the food business and are selling by the pallet to WalMart.) Plumbing has standard ways of doing things, and standard parts that rarely change. Web services need to settle down like that. Really, few sites are doing anything very interesting.
  • overkalix 8y ago
    There can't be a similar process of standardization if there is no cooperation between supply actors, and there will not be cooperation if the market is completely open. There is a tradeoff between scalability and competitiveness. If your product can easily penetrate a market without any more than a few tweaks it means it is also easily replaceable, and you'll be competing through non-functional aspects.
• matz1 8y ago
  Funny, I actually been using plumbing analogy all this time when talking to non tech people. Many people think too highly of what I actually did. I too feel like using the word engineer is sugar coating it.
  • Apocryphon 8y ago
    The internet is a series of tubes meme aside, I've felt that a lot of modern engineering, at least tying together REST APIs with web/mobile GUIs to do CRUD, is really just data plumbing. We're just packaging and unpacking blobs of information. That's it. And the difficulty involved is that we're not yet smart enough to build efficient, easily-adopted systems that use natural language and don't contain a million dependencies and hacky workarounds.
• matz1 8y ago
  I have no problem when getting paid engineer job when I'm just doing plumbing :-) but the problem is when I go look for a job or interview, there is literally no job ad for software plumber so I have to pretend to be engineer.
• adim86 8y ago
  I don't feel the author is belittling plumbers or is saying software engineering in 2018 is less glamorous. I think the point he was getting at is that lots of software engineering recruiters and employees treat software like we are making the next PageRank Algorithm or inventing the next rocket. He is pointing out that today lots of what we do is using the already built software to create new things. This is ok, this is still hard, this is still worth a lot but we need to work on our perspective, both for hiring and also for job expectation.
• et-al 8y ago
  "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." - Fred Brooks Dug up this old HN thread about Data First if anyone else is interested: https://news.ycombinator.com/item?id=10291688 https://news.ycombinator.com/item?id=10291688
• danesparza 8y ago
  So for application development, I tend to agree. Interviewers should focus less on computer science questions and more on application development questions. But ... with advances in AI / Machine learning, drone tech, and self-driving vehicles, I think there is a growing niche in software development for more serious science / data analysis. Investor expectations are being set more and more by this niche -- and causing regular application developers to have to push the envelope.
• zelos 8y ago
  I constantly have this feeling: software development as a job, relative to the amount it pays, just feels ridiculously easy most of the time. Churn out some API that takes data from one place and exposes it in a slightly different form over here, write some unit tests and docs, done. Then other days I'll spend an hour doing nothing but thinking, staring out the window, trying to figure out the best way to do solve some complex multi-threaded issue, before writing any code, and I feel like a real engineer again.
• greenhatman 8y ago
  >other engineers insisted they overbuild things such that junior devs will take a long time to understand it Anywhere that I've worked this will get your PRs or architectural ideals rejected.
• _bxg1 8y ago
  One of the greatest personal flaws your average engineer has (I can say this because I fall in this category), is the constant desire to do more engineering. We have a need to make newer, cleverer stuff ad infinitum, whether or not it's needed, whether or not it's even useful. This is why there are so many frameworks, this is why people make their systems scalable before they need to be and bloated beyond their use case. We like shiny toys and we like to use them to make more shiny toys. It's hard to be someone with that mentality when most of the major problems in your field are pretty much already solved.
• _bxg1 8y ago
  Right now anybody working in software (outside of startups) makes an extremely comfortable living. I think we're headed for a bubble-burst as the job market approaches saturation. The "bootcamps" are so successful because you don't have to be a genius to do the plumbing, but you get paid as if you are one. I've considered going back for a master's degree for this reason alone.
• InclinedPlane 8y ago
  > "This isn’t a popular opinion among software engineers, but I believe that many in the field are overpaid relative to the difficulty of the work they do." This is straight up classism, period. Software devs want to pretend that their high (or even just decent) salaries are due to membership in an elite, highly skilled class. Software development isn't some brain dead job like burger flipping or some gross blue collar job like, ugh, plumbing, right? And there you see the classism raise its ugly head. The denigration of entire job fields. Forcing a separation between "true" developers and other workers also requires denigrating those other workers, how else can "true" developers be raised up higher without pushing lesser workers lower? A lot of engineering is "just" plumbing. So what? A lot of real-engineering is also "just" plumbing (or the equivalent). So what? Does that mean it's valueless? Does that mean people who aren't in the elite should be treated like second class citizens or that they shouldn't make a decent living for their work? The truth is that people who do "just" plumbing work (whether in software development, in engineering, or actual plumbers) still do valuable, high quality work. Still do skilled work (not that that even matters or is a meaningful term). And still deserve to be compensated well for that work. Let's stop shitting on burger flippers and plumbers, let's stop trying to define hard class boundaries between some hypothetical elite of "true" highly skilled knowledge workers and everyone else. Let's stop calling for people to be paid less for their work, almost everyone (even software devs) are being underpaid today. It's unhelpful, it's divisive, it's prejudicial, and it's just mean.
• yakshaving_jgt 8y ago
  Most people who buy SUVs don't drive off-road. Most people who buy diving watches don't dive. They do expect those things to perform if pushed to those lengths though. It's possible the same mentality carries through when hiring programmers. I'm not sure it's a problem either.
• auvrw 8y ago
  i frequently compare workaday web-dev to general contracting (a favorable description from my perspective). no issues with the article title. here's my issue with the article text: it describes I came to the Graide Network and immediately started breaking our legacy app into microservices. as "overengineering," and perhaps it was ... or perhaps it was just not-great planning. after all, the premise of the article is that the microservices are now incurring costs that can be described more like plumbing than engineering. here's where some of my experience partially aligns with the article's complaints Investors are guilty of pushing it on us this has happened to CEOs at startups i've been at: [some] investors asked for diagrams, etc., and we made decisions to build invisible fanciness instead of product for users. here's where the author begins upon the path to enlightenment we can build more with less
• analog31 8y ago
  After reading a number of comments... I wonder if we in the tech sector tend to romanticize the trades. I don't necessarily disagree, but would only offer a caution: Look at their hands. I had a new screen door installed on my house, ordered it through Home Depot, and they contracted it to a father-son business. The father was not much older than me (mid 50s), but was broken and hobbling. There's also a widespread view that we will always need tradespeople, e.g., to build houses. But the trades are being replaced by automation in subtle ways that could eat away at their livelihoods bit by bit. Examples include plumbing fittings where you just cut a plastic tube and squeeze it onto a fitting, rather than sweating it together with a torch. Another is the ability to design and order a "custom" house that's built in a factory and assembled on site.
  • notacoward 8y ago
    I think it's a form of overcompensation. We all know deep down that we're pampered and privileged and frankly not contributing as much to the world as we're taking out of it. Thank you, supply and demand and barely-free-enough markets! But instead of just accepting that and making our peace with it, many of us try to make up for it with our choice of sports, hobbies, or accoutrements. There are way more techies obsessed with martial arts or brewing or sports cars than even other demographic factors such as income would predict. I think it's the same with trade and craft knowledge. Any techie with any exposure at all to the more traditionally "manly" trades (I'd include things like military/police work or firefighting here) tends to flaunt it as a way to show that they're more worldly and "authentic" than all those other techies who have never known any other life. I used to do it myself, except that my "trade" was the simple one of actually being poor. Then I realized that it was just another kind of virtue signaling, and it lost its appeal.
• stevehiehn 8y ago
  I find this article interesting largely because it appears to tease out class & elitism.
• snarfybarfy 8y ago
  I once wrote a class called SuperMario that did some plumbing between the database and the HTML template and people didn't think it was funny at all. Contrary to a lot of comments, I still think the word "plumbing" fits well with connecting specific columns in the DB with specific holes in the template. Like in real plumbing, you get shit when you connect the wrong things:-)