5 ms·

I’ve been programming for over 30 years, and never been as productive as before. Need to load and decode a jpeg/png, I grab stb_image.h. Need to decode an ogg f

I’ve been programming for over 30 years, and never been as productive as before. Need to load and decode a jpeg/png, I grab stb_image.h. Need to decode an ogg file, libogg. Need to decompress, libz. Need to decode video, libavformat. Need physics, libbullet. Need truetype fonts, freetype. Need a GUI, Qt. Need SSL, libssl. My day becomes selecting libraries, integrating them, and testing integration. My business code is < 20% of our entire codebase. And I’m much more productive in 2022 than in 1992.

swatcoder 4y ago
That’s funny because I feel like my day became cleaning up those kinds of projects as their countless dependencies rot. It feels anything but productive.
- mynegation 4y ago
  What is your alternative then? Vendored-in dependencies with their ossified security vulnerabilities? Or figuring out homegrown code of dubious quality for the functionality that is not core business of the company/product?
  - swatcoder 4y ago
    Nothing so dramatic. The alternative is judicious inclusion of dependencies rooted in thoughtful, experienced engineering. Robust, comprehensive libraries from reliable vendors can provide a lot of value and are slow to rot. These can be anticipated and added early so that they’re made good use of and can become the first tool to reach for before adding other dependencies. Think React, lodash, QT, boost, etc. Meanwhile, I acknowledge that most other packages and repos are of far more dubious quality than anything my team would write, have no accountability to my team or stakeholders, receive few/no code reviews when added or updated, introduce conflicting style/semantic conventions, and generally expand the surface area for bugs and vulnerabilities by including many lines of code that have no relevance to the project. There are no strict rules, but these are the sort of considerations that weigh in.
    - chrismarlow9 4y ago
      My experience in engineering tells me that vendors and dependencies come and go and get acquired and all of them develop their own set of problems over the years. I get what you're saying, but you picked projects that are easy to see as popular in hindsight, but not so much in early adoption. Why react over angular? There is a problem with picking vendors outside of just "picking the best" that has multiple elements: - Age of the company - Complexity of integration with the vendor - Stability in early adoption If you adopt the tech too early, you risk instability during a time where you don't have the size as a business to afford that. Nobody that's a customer of Google is going to leave their contract because it was down for an hour. However your new 20 employee startup goes down for an hour? And you just signed your largest customer yet? And it's because you decided to early adopt some tech because it simply "looks good and efficient?". No thanks, that customer will be looking at alternatives when renewal time comes, if they don't just pay out the contract and leave on you already. And good luck getting funding with that blemish that you blew it by adopting cool new trends. Okay, so being early adopter at a startup is risky. So maybe you go with something that is "kind of" new, and is being used by other large companies. Great, so you start building, and the feature system grows, and the integration grows, and now some new system comes along that is way better. Think about life before React/AngularJS and people who built systems with plain jQuery. There was a time where that was the hottest thing and nothing like React/Angular even existed. So you pick that and then 3 years later you have piles of jQuery and AngularJS/React/whatever else comes out, but remember the maturity thing mentioned above. So now you watch it grow over 2 years and now you have 5 years worth of jQuery and the amount of work it will take to switch to React/Angular is a year long project. You can't just stop everything else to work on this, so it's 20% of your time, nevermind that it's a moving target as you fix bugs/release new things in jQuery over that year. So now you're maintaining both for a year, and then 3 years later it becomes clear that React is the winner over Angular, and you picked the wrong one! At the time you picked they both seemed pretty mature (Google w/Angular and FB with React). Oh jeeze, do we migrate everything again? And from the business perspective how do you even begin to sell this to the people in charge when it results in no benefits to the end user. And by the time you finish it will these even be the tech you should be using? Or will something else rise up to replace that? The complexity grows with the integration as the age of the company increases. And adding headcount and expecting to solve the problem just introduces more problems. Now your team size is 10 instead of 5 and standup is taking an hour everyday, do you break it up into other teams? What are those teams called? What if nobody wants to work on the upkeep of the legacy? Why would they as an IC? So now how do you split the work up fairly? I'm not saying "don't use vendors", "don't adopt early" or "don't vet your vendors", but what I'm saying is the problem isn't the tech. It's the fact that the "obvious correct solution" is a changing answer over time. And the integration and added complexity combined with team dynamics eventually becomes the hardest problem a company has to solve.
- selfmodruntime 4y ago
  The alternative is shipping your own, which most of the time is a) a bad idea since you don't have domain specific knowledge and b) the customer is not paying for you to reimplement the 10th jpeg parsing function.
whartung 4y ago
> My day becomes selecting libraries, integrating them, and testing integration. Funny, being an old, cranky, gray-no-beard, if I wanted to spend my days selecting libraries, integrating them, and testing integration, I would have become a digital electronics engineer. Your point is well taken, it's just our modern truth. It's just not was attracted me to this world in the first place.
dehrmann 4y ago
You could have said the same thing 30 years ago because of the C standard library. People used to hand-roll a lot of code you get for free from it.
Supermancho 4y ago
> Writing tests is time-consuming, usually doesn't scale, and it easily creates tight coupling with implementations After thousands of years, humans still use double accounting in finance. This is because it's effective. Unsurprisingly, it's tightly coupled to the original transaction recording. The constant assertion that tight coupling is bad, because it creates more work which can be error prone. This is a fundamental problem with how people discuss software testing. It's not a panacea, but a guardrail. Guardrails aren't indestructible. That isn't the point. Developers have an obsession with making things as easy as possible (be that simplifying or codifying) and this is not a domain where that makes for a better solution, all other qualities being equal.
- panstromek 4y ago
  I think "double accounting" is a good analogy, and if you look at it that way, some tests that look like the same thing written twice in a different way start to make more sense. The problem is just with effectivity - it'd be fine if the time spent on test would be the same as time on code, but very often the effort on test is much bigger while not even covering all important cases. I think we need to look for methods to get a good case coverage without spending majority of our time writing tests.
  - Too 4y ago
    Hit the nail on the head. For same reason requirements writing is so difficult, to explain every edge case you end up writing the whole software again, using human language. Writing software is the process of explaining a set of conditions and conversions from input to output. We should assume the programmer does this in as short code as possible (unless we are using a low level language like C which adds a lot of accidental complexity like memory management). To repeat all these conditions in test-code, would be to write the same thing twice, in a different way. Not saying this is wrong, it's just important to acknowledge it. Testing often becomes more like sampling, with hard coded or generated known set of inputs/outputs, often taken from a scenario derived from a common user journey. This verifies that this particular journey works, but it would never be a guarantee that other edge cases of it does, to do that you do end up in the write-twice situation.
lazyjones 4y ago
And still you're typing repetitive text. My dad was punching cards and able to read code from punched tape. I grew up writing code on a home computer's display. And now, 35 years later, it's long overdue that some visual NoCode tool should be the default for mostly repetitive programming jobs. If I want to build a Finder Quick Action to resize an image, for example, I certainly won't open a C editor, I'll just use Automator.
hsn915 4y ago
> My day becomes selecting libraries, integrating them, and testing integration. This is why I hate modern programming. When was the last time you had a job where you could actually learn how to actually program something? The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library. Which means the more time you spend at a regular job, the worse your brain will rot. Just talk to any programmer at any company and you will notice that almost none of them know how to program anything. They just know how to stitch together libraries with some glue code.
- selfmodruntime 4y ago
  > The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library. They are right, even if you do not agree. Handrolling is incredibly cost ineffective, yet another source of bugs and can be seen as a bad idea when it comes to security. Huge third party libraries like LibOgg are maintained by gigantic industry players. Chances are, they are more knowledgeable in that specific domain than you are and they can focus more developer time on what is just a small part in your business application.
  - hsn915 4y ago
    How many of the people who are working on any given library actually are domain expert on the subject matter as you imagine? This might have been true 20 years ago. But now the cultural shift affects even the people who work on the lower levels. A sort of famous example is how slow Visual Studio has gotten compared to 20 years ago (when it was running on slower hardware even!) where it was nearly instant. The refterm saga has demonstrated that people employed by Microsoft and paid very high salaries to work on a terminal emulator have no idea how to make a half decent terminal renderer. > handrolling is incredibly cost ineffective, yet another source of bugs and can be seen as a bad idea when it comes to security. Without context this statement is blatantly false. Many libraries are full of bugs and security problems.
- chakkepolja 4y ago

swatcoder 4y ago

That’s funny because I feel like my day became cleaning up those kinds of projects as their countless dependencies rot. It feels anything but productive.

mynegation 4y ago
What is your alternative then? Vendored-in dependencies with their ossified security vulnerabilities? Or figuring out homegrown code of dubious quality for the functionality that is not core business of the company/product?
- swatcoder 4y ago
  Nothing so dramatic. The alternative is judicious inclusion of dependencies rooted in thoughtful, experienced engineering. Robust, comprehensive libraries from reliable vendors can provide a lot of value and are slow to rot. These can be anticipated and added early so that they’re made good use of and can become the first tool to reach for before adding other dependencies. Think React, lodash, QT, boost, etc. Meanwhile, I acknowledge that most other packages and repos are of far more dubious quality than anything my team would write, have no accountability to my team or stakeholders, receive few/no code reviews when added or updated, introduce conflicting style/semantic conventions, and generally expand the surface area for bugs and vulnerabilities by including many lines of code that have no relevance to the project. There are no strict rules, but these are the sort of considerations that weigh in.
  - chrismarlow9 4y ago
    My experience in engineering tells me that vendors and dependencies come and go and get acquired and all of them develop their own set of problems over the years. I get what you're saying, but you picked projects that are easy to see as popular in hindsight, but not so much in early adoption. Why react over angular? There is a problem with picking vendors outside of just "picking the best" that has multiple elements: - Age of the company - Complexity of integration with the vendor - Stability in early adoption If you adopt the tech too early, you risk instability during a time where you don't have the size as a business to afford that. Nobody that's a customer of Google is going to leave their contract because it was down for an hour. However your new 20 employee startup goes down for an hour? And you just signed your largest customer yet? And it's because you decided to early adopt some tech because it simply "looks good and efficient?". No thanks, that customer will be looking at alternatives when renewal time comes, if they don't just pay out the contract and leave on you already. And good luck getting funding with that blemish that you blew it by adopting cool new trends. Okay, so being early adopter at a startup is risky. So maybe you go with something that is "kind of" new, and is being used by other large companies. Great, so you start building, and the feature system grows, and the integration grows, and now some new system comes along that is way better. Think about life before React/AngularJS and people who built systems with plain jQuery. There was a time where that was the hottest thing and nothing like React/Angular even existed. So you pick that and then 3 years later you have piles of jQuery and AngularJS/React/whatever else comes out, but remember the maturity thing mentioned above. So now you watch it grow over 2 years and now you have 5 years worth of jQuery and the amount of work it will take to switch to React/Angular is a year long project. You can't just stop everything else to work on this, so it's 20% of your time, nevermind that it's a moving target as you fix bugs/release new things in jQuery over that year. So now you're maintaining both for a year, and then 3 years later it becomes clear that React is the winner over Angular, and you picked the wrong one! At the time you picked they both seemed pretty mature (Google w/Angular and FB with React). Oh jeeze, do we migrate everything again? And from the business perspective how do you even begin to sell this to the people in charge when it results in no benefits to the end user. And by the time you finish it will these even be the tech you should be using? Or will something else rise up to replace that? The complexity grows with the integration as the age of the company increases. And adding headcount and expecting to solve the problem just introduces more problems. Now your team size is 10 instead of 5 and standup is taking an hour everyday, do you break it up into other teams? What are those teams called? What if nobody wants to work on the upkeep of the legacy? Why would they as an IC? So now how do you split the work up fairly? I'm not saying "don't use vendors", "don't adopt early" or "don't vet your vendors", but what I'm saying is the problem isn't the tech. It's the fact that the "obvious correct solution" is a changing answer over time. And the integration and added complexity combined with team dynamics eventually becomes the hardest problem a company has to solve.
selfmodruntime 4y ago
The alternative is shipping your own, which most of the time is a) a bad idea since you don't have domain specific knowledge and b) the customer is not paying for you to reimplement the 10th jpeg parsing function.

whartung 4y ago

> My day becomes selecting libraries, integrating them, and testing integration. Funny, being an old, cranky, gray-no-beard, if I wanted to spend my days selecting libraries, integrating them, and testing integration, I would have become a digital electronics engineer. Your point is well taken, it's just our modern truth. It's just not was attracted me to this world in the first place.

dehrmann 4y ago

You could have said the same thing 30 years ago because of the C standard library. People used to hand-roll a lot of code you get for free from it.

Supermancho 4y ago

> Writing tests is time-consuming, usually doesn't scale, and it easily creates tight coupling with implementations After thousands of years, humans still use double accounting in finance. This is because it's effective. Unsurprisingly, it's tightly coupled to the original transaction recording. The constant assertion that tight coupling is bad, because it creates more work which can be error prone. This is a fundamental problem with how people discuss software testing. It's not a panacea, but a guardrail. Guardrails aren't indestructible. That isn't the point. Developers have an obsession with making things as easy as possible (be that simplifying or codifying) and this is not a domain where that makes for a better solution, all other qualities being equal.

panstromek 4y ago
I think "double accounting" is a good analogy, and if you look at it that way, some tests that look like the same thing written twice in a different way start to make more sense. The problem is just with effectivity - it'd be fine if the time spent on test would be the same as time on code, but very often the effort on test is much bigger while not even covering all important cases. I think we need to look for methods to get a good case coverage without spending majority of our time writing tests.
- Too 4y ago
  Hit the nail on the head. For same reason requirements writing is so difficult, to explain every edge case you end up writing the whole software again, using human language. Writing software is the process of explaining a set of conditions and conversions from input to output. We should assume the programmer does this in as short code as possible (unless we are using a low level language like C which adds a lot of accidental complexity like memory management). To repeat all these conditions in test-code, would be to write the same thing twice, in a different way. Not saying this is wrong, it's just important to acknowledge it. Testing often becomes more like sampling, with hard coded or generated known set of inputs/outputs, often taken from a scenario derived from a common user journey. This verifies that this particular journey works, but it would never be a guarantee that other edge cases of it does, to do that you do end up in the write-twice situation.

lazyjones 4y ago

And still you're typing repetitive text. My dad was punching cards and able to read code from punched tape. I grew up writing code on a home computer's display. And now, 35 years later, it's long overdue that some visual NoCode tool should be the default for mostly repetitive programming jobs. If I want to build a Finder Quick Action to resize an image, for example, I certainly won't open a C editor, I'll just use Automator.

hsn915 4y ago

> My day becomes selecting libraries, integrating them, and testing integration. This is why I hate modern programming. When was the last time you had a job where you could actually learn how to actually program something? The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library. Which means the more time you spend at a regular job, the worse your brain will rot. Just talk to any programmer at any company and you will notice that almost none of them know how to program anything. They just know how to stitch together libraries with some glue code.

selfmodruntime 4y ago
> The prolifiration of libraries means that all your bosses and managers would extremely frown upon anyone handrolling a solution instead of using an existing library. They are right, even if you do not agree. Handrolling is incredibly cost ineffective, yet another source of bugs and can be seen as a bad idea when it comes to security. Huge third party libraries like LibOgg are maintained by gigantic industry players. Chances are, they are more knowledgeable in that specific domain than you are and they can focus more developer time on what is just a small part in your business application.
- hsn915 4y ago
  How many of the people who are working on any given library actually are domain expert on the subject matter as you imagine? This might have been true 20 years ago. But now the cultural shift affects even the people who work on the lower levels. A sort of famous example is how slow Visual Studio has gotten compared to 20 years ago (when it was running on slower hardware even!) where it was nearly instant. The refterm saga has demonstrated that people employed by Microsoft and paid very high salaries to work on a terminal emulator have no idea how to make a half decent terminal renderer. > handrolling is incredibly cost ineffective, yet another source of bugs and can be seen as a bad idea when it comes to security. Without context this statement is blatantly false. Many libraries are full of bugs and security problems.
chakkepolja 4y ago