6 ms·

Programming has turned into gluing together mystery meat. You can't make a business case for essentially writing every library you need. It would take too long

Programming has turned into gluing together mystery meat. You can't make a business case for essentially writing every library you need. It would take too long to have anything like modern stuff. So you always deal with a huge pile of unknowns all of the time. The job consists more of the meta-skill of navigating unchartered waters intelligently, but you get dinged for doing the real job and people doing a terrible job get the rewards. The bad people glue the stuff together without trying to understand it, don't document stuff, pull in any dependency that gets them closer to their goal, and put up a victory flag when the thing does what they want it to do. Then the war goes on day by day dealing with under-documented stuff and a giant pile of dependencies with no internal logic. The hero goes on to do it again, and a team of saps tries to make sense of a Gordian knot for the rest of the product's lifetime.

ozim 4y ago
We have tools to fight it - code reviews, dependency scanning, QA. I am guilty as the next guy of throwing stuff that "just works" out there. It is also important what kind of software you write - if you write frameworks, you better get unit tests and full blown process in place to make sure your framework will work in 10 years time. If you work on a business application - in 6 months time your application might be gone or requirements change in a way that everything you wrote does not matter anymore and future proofing was just a waste of time. I would like that more devs/software engineers understood which one they are writing.
- wizofaus 4y ago
  > in 6 months time your application might be gone or requirements change in a way that everything you wrote does not matter anymore I'd say if you write code with the assumption this might be true, you're almost guaranteeing it will be - business requirements do often change but typically that means having to modify/refactor existing code so that core components continue to work the same way, because those core requirements haven't changed. While you can make occasional assumptions about which requirements are more likely to change than others I'd much rather write all my code on the assumption the requirements won't change too significantly and that the tools/automated testing and other processes are in place to support that. There's far too much "throwaway/POC" code that ends up getting used in production, often for years, then mysteriously stops working down the line because nobody ever assumed it would need to survive that long. Whereas I don't believe I've seen a project fail because too much time was spent ensuring code was well written and well tested.
  - wpietri 4y ago
    > Whereas I don't believe I've seen a project fail because too much time was spent ensuring code was well written and well tested. This does indeed happen. Kent Beck tells a story of being called in to help a European insurance project where, sick of all the legacy code, they spun off a new company to start fresh. Teams of bright people spent years building The Right Thing and doing it The Right Way. But they never delivered anything actually useful, and by the time the sponsors were entirely fed up, they still couldn't promise anything soon. So they fired everybody, hired back one team's worth, and then started fresh on well-written code that actually did something immediately useful. I think there's a middle path between "write garbage that succeeds" and "write perfect code that never gets used". I think it's narrow but doable, and it requires dogged attention to both "ship early and often" and "build in a way that's sustainable over the long term".
    - ozim 4y ago
      I argue that real dichotomy is: "write perfect code that gets used a lot, frameworks, libraries" "write throw away glue code with use of libraries and frameworks" Because for me "write garbage that succeeds" vs "perfect code that never gets used" is false dichotomy. Where most devs want to write libraries and frameworks because these are places where "eternal fame" is - even better if you can write your own programming language that is the highest echelon of software engineering. Ugly truth is most devs are driving beat up honda - where Linus Torvalds, Anders Hejlsberg, Bjarne Stroustrup are F1 drivers - you can rev up your honda on street lights all you want - but that is not the same league. So most devs write business apps and many of them overshoot quality.
      - wpietri 4y ago
        > So most devs write business apps and many of them overshoot quality. I think this can be true for sufficiently bad definitions of quality, which I agree are very common. To me, though, overengineering something doesn't really increase quality, because actual need puts an upper bound on quality.
    - wizofaus 4y ago
Joeri 4y ago
I can’t shake the feeling we are inching closer to the programmer-archeologist from a deepness in the sky, where gluing together mystery meat is the only sensible thing that still needs doing. “We should rewrite it all,” said Pham. “It’s been done,” said Sura, not looking up. She was preparing to go off-Watch, and had spent the last four days trying to root a problem out of the coldsleep automation. “It’s been tried,” corrected Bret, just back from the freezers. “But even the top levels of fleet system code are enormous. You and a thousand of your friends would have to work for a century or so to reproduce it.” Trinli grinned evilly. “And guess what—even if you did, by the time you finished, you’d have your own set of inconsistencies. And you still wouldn’t be consistent with all the applications that might be needed now and then.” https://www.goodreads.com/quotes/9427225-pham-nuwen-spent-years-learning-to-program-explore-programming-went-back https://www.goodreads.com/quotes/9427225-pham-nuwen-spent-ye...
wpietri 4y ago
I agree with a lot of what you say! When I was a kid, my first computer had 4k of RAM; my next one had 48k. I learned a lot about how everything worked, because there was very little to it. Over the years things have grown, but I've had a chance to grow along with it. But somebody starting out today start in the middle of vast layers of complexity. From processors and hardware that are hugely more complicated up through virtualization, containerization, rich OSes, languages and standard libraries with decades of history, tons of add-on libraries, and then out to user platforms with their own tangled ecosystems and decades of history. It's a lot! I think it's so much harder now for developers to develop "mechanical sympathy", that intuitive understanding of the rhythms of machinery. Apparently slight shifts in code can be 3 orders of magnitude difference in performance. So much of what they deal with is historically determined. (E.g., what percentage of working developers has every actually seen a carriage returning or a line feeding?) And it's all running on much shorter cycles with an ever-updating mass of tools, libraries, frameworks, and operating systems. On the one hand, people can do some amazing¸valuable stuff with very little training. That's great! But I think it's a lot harder to truly master the craft, and I'm concerned that a lot of programmers spend their time in professional contexts where the feedback loops are long or broken such that they are encouraged to be less analytical than superstitious, cargo-culting their way to from one overly-aggressive sprint deadline to the next.
- Tildey 4y ago
  > I think it's so much harder now for developers to develop "mechanical sympathy" This has always been something that’s fascinated me, how some people don’t have this feeling of “mechanical sympathy”. Feeling “bad” (not the exact right term but I don’t know how best to describe this very intangible feeling) for a high-revving engine, or a CPU wasting cycles, or a structural component being under too much stress. Even if all of those things are within spec. It feels like the drive to avoid that feeling ends up creating better solutions: more efficient code, a better distribution of stress across a structure, etc. But I wonder if it’s something learned, or something that people just “have” to a certain degree. Not trying to pathologise everything, but it feels like the sort of thing that would be correlated with ASD
  - 4y ago
bentlegen 4y ago
In the past you just wrote the mystery meat yourself, and I assure you it didn’t taste better. Like, I literally wrote and deployed my own garbage JavaScript dependency manager in the late 2000s and we all thought it was great. It wasn’t.
deleted 4y ago
[deleted]
xhrpost 4y ago
This sums up mostly what I've been feeling for the last 10+ years. When I started "web 2.0" development in 2005, there was a much smaller library ecosystem. I had to use what tools were available, most of which were developed by for-profit companies and thus heavily documented. If the feature I needed didn't exist, I had to build it. I spent much more time reading documentation and specifications than I do today. Now, I do try to read docs first for whatever FOSS I'm using, but often they are lacking the specificity I need and I end up just running through a bunch of Google searches and SO questions hoping to find my specific "glue case". Whatever challenging "science" aspect I was expecting from my "computer science" skills just doesn't seem to exist. The concerns of logic and program efficiency/structure are still important, just not what I have to do most of the time when gluing stuff together. I'm trying to accept my fate these days and just be thankful I work in a well-paid field with good people, but it's tough as I feel it does affect my overall performance and personal enjoyment as flow is so hard to achieve anymore.
dehrmann 4y ago
Programming has always been about gluing together mystery meat. Now we just glue together bigger parts.
314 4y ago
This should be carved into stone somewhere. It is the most accurate description of modern software development that I've come across.

ozim 4y ago

We have tools to fight it - code reviews, dependency scanning, QA. I am guilty as the next guy of throwing stuff that "just works" out there. It is also important what kind of software you write - if you write frameworks, you better get unit tests and full blown process in place to make sure your framework will work in 10 years time. If you work on a business application - in 6 months time your application might be gone or requirements change in a way that everything you wrote does not matter anymore and future proofing was just a waste of time. I would like that more devs/software engineers understood which one they are writing.

wizofaus 4y ago
> in 6 months time your application might be gone or requirements change in a way that everything you wrote does not matter anymore I'd say if you write code with the assumption this might be true, you're almost guaranteeing it will be - business requirements do often change but typically that means having to modify/refactor existing code so that core components continue to work the same way, because those core requirements haven't changed. While you can make occasional assumptions about which requirements are more likely to change than others I'd much rather write all my code on the assumption the requirements won't change too significantly and that the tools/automated testing and other processes are in place to support that. There's far too much "throwaway/POC" code that ends up getting used in production, often for years, then mysteriously stops working down the line because nobody ever assumed it would need to survive that long. Whereas I don't believe I've seen a project fail because too much time was spent ensuring code was well written and well tested.
- wpietri 4y ago
  > Whereas I don't believe I've seen a project fail because too much time was spent ensuring code was well written and well tested. This does indeed happen. Kent Beck tells a story of being called in to help a European insurance project where, sick of all the legacy code, they spun off a new company to start fresh. Teams of bright people spent years building The Right Thing and doing it The Right Way. But they never delivered anything actually useful, and by the time the sponsors were entirely fed up, they still couldn't promise anything soon. So they fired everybody, hired back one team's worth, and then started fresh on well-written code that actually did something immediately useful. I think there's a middle path between "write garbage that succeeds" and "write perfect code that never gets used". I think it's narrow but doable, and it requires dogged attention to both "ship early and often" and "build in a way that's sustainable over the long term".
  - ozim 4y ago
    I argue that real dichotomy is: "write perfect code that gets used a lot, frameworks, libraries" "write throw away glue code with use of libraries and frameworks" Because for me "write garbage that succeeds" vs "perfect code that never gets used" is false dichotomy. Where most devs want to write libraries and frameworks because these are places where "eternal fame" is - even better if you can write your own programming language that is the highest echelon of software engineering. Ugly truth is most devs are driving beat up honda - where Linus Torvalds, Anders Hejlsberg, Bjarne Stroustrup are F1 drivers - you can rev up your honda on street lights all you want - but that is not the same league. So most devs write business apps and many of them overshoot quality.
    - wpietri 4y ago
      > So most devs write business apps and many of them overshoot quality. I think this can be true for sufficiently bad definitions of quality, which I agree are very common. To me, though, overengineering something doesn't really increase quality, because actual need puts an upper bound on quality.
  - wizofaus 4y ago

Joeri 4y ago

I can’t shake the feeling we are inching closer to the programmer-archeologist from a deepness in the sky, where gluing together mystery meat is the only sensible thing that still needs doing. “We should rewrite it all,” said Pham. “It’s been done,” said Sura, not looking up. She was preparing to go off-Watch, and had spent the last four days trying to root a problem out of the coldsleep automation. “It’s been tried,” corrected Bret, just back from the freezers. “But even the top levels of fleet system code are enormous. You and a thousand of your friends would have to work for a century or so to reproduce it.” Trinli grinned evilly. “And guess what—even if you did, by the time you finished, you’d have your own set of inconsistencies. And you still wouldn’t be consistent with all the applications that might be needed now and then.” https://www.goodreads.com/quotes/9427225-pham-nuwen-spent-years-learning-to-program-explore-programming-went-back https://www.goodreads.com/quotes/9427225-pham-nuwen-spent-ye...

wpietri 4y ago

I agree with a lot of what you say! When I was a kid, my first computer had 4k of RAM; my next one had 48k. I learned a lot about how everything worked, because there was very little to it. Over the years things have grown, but I've had a chance to grow along with it. But somebody starting out today start in the middle of vast layers of complexity. From processors and hardware that are hugely more complicated up through virtualization, containerization, rich OSes, languages and standard libraries with decades of history, tons of add-on libraries, and then out to user platforms with their own tangled ecosystems and decades of history. It's a lot! I think it's so much harder now for developers to develop "mechanical sympathy", that intuitive understanding of the rhythms of machinery. Apparently slight shifts in code can be 3 orders of magnitude difference in performance. So much of what they deal with is historically determined. (E.g., what percentage of working developers has every actually seen a carriage returning or a line feeding?) And it's all running on much shorter cycles with an ever-updating mass of tools, libraries, frameworks, and operating systems. On the one hand, people can do some amazing¸valuable stuff with very little training. That's great! But I think it's a lot harder to truly master the craft, and I'm concerned that a lot of programmers spend their time in professional contexts where the feedback loops are long or broken such that they are encouraged to be less analytical than superstitious, cargo-culting their way to from one overly-aggressive sprint deadline to the next.

Tildey 4y ago
> I think it's so much harder now for developers to develop "mechanical sympathy" This has always been something that’s fascinated me, how some people don’t have this feeling of “mechanical sympathy”. Feeling “bad” (not the exact right term but I don’t know how best to describe this very intangible feeling) for a high-revving engine, or a CPU wasting cycles, or a structural component being under too much stress. Even if all of those things are within spec. It feels like the drive to avoid that feeling ends up creating better solutions: more efficient code, a better distribution of stress across a structure, etc. But I wonder if it’s something learned, or something that people just “have” to a certain degree. Not trying to pathologise everything, but it feels like the sort of thing that would be correlated with ASD
- 4y ago

bentlegen 4y ago

In the past you just wrote the mystery meat yourself, and I assure you it didn’t taste better. Like, I literally wrote and deployed my own garbage JavaScript dependency manager in the late 2000s and we all thought it was great. It wasn’t.

deleted 4y ago

[deleted]

xhrpost 4y ago

This sums up mostly what I've been feeling for the last 10+ years. When I started "web 2.0" development in 2005, there was a much smaller library ecosystem. I had to use what tools were available, most of which were developed by for-profit companies and thus heavily documented. If the feature I needed didn't exist, I had to build it. I spent much more time reading documentation and specifications than I do today. Now, I do try to read docs first for whatever FOSS I'm using, but often they are lacking the specificity I need and I end up just running through a bunch of Google searches and SO questions hoping to find my specific "glue case". Whatever challenging "science" aspect I was expecting from my "computer science" skills just doesn't seem to exist. The concerns of logic and program efficiency/structure are still important, just not what I have to do most of the time when gluing stuff together. I'm trying to accept my fate these days and just be thankful I work in a well-paid field with good people, but it's tough as I feel it does affect my overall performance and personal enjoyment as flow is so hard to achieve anymore.

dehrmann 4y ago

Programming has always been about gluing together mystery meat. Now we just glue together bigger parts.

314 4y ago

This should be carved into stone somewhere. It is the most accurate description of modern software development that I've come across.