6 ms·

Something that has also worked well for me historically is aggressively updating tool chains as an integral part of the development process. I've seen far too m

Something that has also worked well for me historically is aggressively updating tool chains as an integral part of the development process. I've seen far too many companies de-prioritize upgrading their tool chains until it got to the point where they were so out-of-date that things broke through decay and obsolescence, and often had security vulnerabilities with no easy path toward remedying them. At this point, it usually requires heroic efforts to bring everything up to a vaguely modern standard. I've adopted a policy that for every significant compiler or build system release, we create a branch that verifies a good build. If it doesn't build without warnings or errors, or fails in testing, that is treated the same as a bug in the code to be dealt with ASAP. Prioritizing this also tends to enforce good build configuration hygiene and automation. Not being able to do this is a bit of an architecture/code smell anyway. This also makes it much easier to incrementally modernize and refactor the code base with new language features -- the tool chain supports the latest and greatest even if the code base doesn't yet. I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks. Anecdotally, this policy has not materially reduced development velocity; the scope of useful functionality can usually be replaced with a small amount of code that has good mechanical sympathy with the rest of the code base because it was designed with the code base in mind. I've found in many cases you'll write just as much code trying to interface a third-party library. That said, for new projects it is often useful to use a few third-party dependencies to plug holes with a clear plan to replace them in the near- to mid-term. Stopgap dependencies have been a reasonable practice in my experience.

wutwutwat 2y ago
> I also find it difficult to rationalize third-party dependencies these days Your home grown auth code doesn’t have the minds and the scrutiny of security experts like the go-to third party auth lib everyone uses does, nor does it have the free pen testing, security fixes, maintenance, optimizations, refactors, ever collaborative maintainers, or corporate sponsorships. Auth is a solved problem by minds much smarter than my own. I’m glad to outsource things like that to third party deps.
- skydhash 2y ago
  Is auth that difficult? The truly difficult problem is cryptography and that's the code you really don't want to write. But most authentication schemes are well-known. No need to import a library, and if you do, you should be prepared to vendor it in.
  - wutwutwat 2y ago
    > most authentication schemes are well-known so why maintain your own implementation of the scheme/spec when a community already does, as well as handles the vulns as they pop up and generally has more eyes looking (and fixing) issues? Here's how I look at it; Trust my own NIH implementation of a spec and hope a user reports a vuln instead of it going undisclosed/unnoticed in the wild for years, or use a community lib that is always being scrutinized and vetted and has security professionals involved. It's a no brainer for me.
- cowsandmilk 2y ago
  I would go beyond auth to areas where there historically have been security bugs. Parsing JPEG files for instance, I can write a parser and can probably choose a modern safe language to avoid many classes of bugs, but a popular open source library is more likely to have been fuzzed by dozens of techniques from multiple companies and universities. The code is more resilient because it has been out there to be experimented on in public.
- bsder 2y ago
  > Auth is a solved problem by minds much smarter than my own. Is it? It seems like all the outsourced auth has lots and lots and lots of problems, issues, etc. along with being remarkably expensive. I'm not saying I, personally, can do better, but it seems like auth is far from a solved problem. Especially since it seems like the difficult part of "auth" is actually "customer support" rather than any technical issue. (I would argue that the lack of an open-source "Auth in a Box" seems to also argue that auth is far from "solved".)
  - wutwutwat 2y ago
rramadass 2y ago
> Something that has also worked well for me historically is aggressively updating tool chains as an integral part of the development process. ... I've adopted a policy that for every significant compiler or build system release, we create a branch that verifies a good build. If it doesn't build without warnings or errors, or fails in testing, that is treated the same as a bug in the code to be dealt with ASAP. ... This also makes it much easier to incrementally modernize and refactor the code base with new language features -- the tool chain supports the latest and greatest even if the code base doesn't yet. Good Point.
cpeterso 2y ago
> I've adopted a policy that for every significant compiler or build system release, we create a branch that verifies a good build. Mozilla's Firefox CI builds with multiple compilers, including clang trunk and (IIRC) Rust Nightly compilers to identify code issues that will need to be fixed before Firefox can update to the new compiler versions and to catch regressions in the compilers themselves.
szvsw 2y ago
> I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks. What kind of software development do you do? As someone who is in ML/data science and works primarily in python, occasionally TS/React, at a small company (about 5 FTE, one round of funding), that sentence is so completely foreign to me. Again, this might just be because I am working primarily on ML systems in Python on relatively small teams (also in academia at the same time), where the most basic building blocks like Pandas (or Polars), PyTorch, XGBoost, SKLearn, etc are so essential and so it’s pretty hard to imagine ever being at a scale where they would be feasible or worth it to replace with our own code… and anyways if support for libraries like those collapses, the only explanation is that the entire landscape has shifted so much that it’s hard to imagine the underlying tech of the company not also having naturally shifted with it… Maybe I am misunderstanding how you are using “3rd party,” or maybe you are just operating in a completely different staffing and financial context which I don’t have experience with. Just curious for more details about what kind of world you operate in where that philosophy is viable!
- all2 2y ago
  I do SW in a hardware world that restricts a lot of dependencies. We vendor some, but mostly roll or own libraries. For time intensive things peripheral to our work we will rely on 3rd party modules (ie, network stack or UI).
- crabbone 2y ago
  You chose the field with one of the worst rates of churn and brokeness of third-party libraries, and by some kind of magic managed to avoid it? You should be buying lottery tickets by a truckload! All the projects you listed (Pandas, PyTorch, SKLearn) have a huge list of compatibility issues, on top of compatibility issues of Python itself. And I know this because I have to support various research projects, which typically use this stuff. My estimate is that the shelf-life of a project using just these three will not exceed 3 years in >90% of cases. My typical situation with these projects is that I'm called in because some former PhD student, who graduated a year or two ago left a project that now another student is picking up. And nothing works. Results cannot be reproduced. The dependencies cannot be installed. And so on. And then, depending of the amount of effort I'm allowed to put into fixing the project, I end up either crafting some sort of a spec that tries to find a set of versions of used libraries that seem to make the project work again. Or, more commonly, have to vendor some code and add fixes on top to make the packages fit together at least somehow. Sometimes I even end up repackaging someone else's code (a dependency), where the version used before cannot be found anymore, or I cannot figure out the combination of version that could possibly make the program work. I hear that it's worse in JavaScript world. And, my experience with LaTeX packages has also been marred by similar problems (but not anywhere near the extent of how bad it is in Python). On the other hand, I had to build absolutely atrocious (math operations on a mix of signed and unsigned ints, modifying const char etc.) C code (pre-K&R, the kind where they write function argument types after parenthesis), and it was much easier to deal with than any "modern" Python (i.e. 3-4 years old). If anything, Python is the poster child of the problem of unreliability of third-party dependencies.
t-writescode 2y ago
> I also find it difficult to rationalize third-party dependencies these days I would love a deeper insight into what you mean by this. I can't imagine you're rolling your own http server, database communication layer (as in like, postgresql's jdbc driver), json parsing engine, etc.
- throwaway290 2y ago
  JSON parser, http server are provided by many runtimes and if a runtime does not provide something it provides primitives that make it often trivial to build
  - lmz 2y ago
    I'm not saying it's impossible, but writing your own HTTP server ("trivial to build"?) to avoid bugs in other implementations doesn't sound like a good idea to me.
    - throwaway290 2y ago
      Read more carefully. We talk about "third-party dependencies" not about "other implementations". Using HTTP server from your runtime is not against the recommendation of avoiding third-party dependencies. Your runtime is usually not counted as "third-party dependency", normal runtimes don't have breaking changes every other moon (Even if you need to implement something like that from scratch it may be OK, your requirements don't need all of the functionality a third package includes because someone else needed it)
      - lmz 2y ago
        It's the "trivial to build" thing I have issues with - especially for the HTTP server. Not runtime vs third party.
    - rurban 2y ago
      I had to write one recently. One third page of code, multi-threaded. Really trivial, and no security nightmares as with overblown 3rdparty deps.
Lutger 2y ago
Fully agree, though I feel there are some nuances with third party dependencies. Consider the following: Some third party dependencies are so widely adopted and supported, they are on-par with the main framework you are using in terms of support and development. They often don't lag the main framework in supporting new versions for long. I don't think these kinds of dependencies have the same drawbacks. You only have a couple at this level anyway, and should choose them carefully. There are also dependencies that solve one particular problem well and very narrowly couple to a specific feature in your product. If they become problematic over time, it is easy to just replace them or even rewrite it yourself. Specifically, you could select these judging how well you could maintain the sourcecode yourself in such an event. Often, these are easy to extend as the focus is so specific. You can consider just adding it to your codebase and thinking of these dependencies as your code, except that it is written by someone else.
jcarrano 2y ago
I had the same experience with Buildroot. The project was stuck at an older release and it was kind of a time-bomb: as time went on it became harder to update because customizations kept piling on but at the same time there were obsolete packages (some with CVEs) which had to be upgraded ASAP.
vv_ 2y ago
> I also find it difficult to rationalize third-party dependencies these days This depends on what kind of dependency you're talking about. For example using Zephyr RTOS significantly reduces time to market and it brings in a lot of 3rd party libraries that are written specifically for it. In other fields this might differ, but fields like Web development are a nightmare to begin with.
MetaWhirledPeas 2y ago
> I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks. I think I understand your sentiment here. For me the risk is that you are not just trusting the original author, you're trusting all future maintainers, especially if you follow your first (and good) suggestion of "aggressively updating tool chains". The other risk is what you are mitigating by aggressively updating: things break through decay and obsolescence. That can usually be mitigated through updates, but if you introduce a dependency that is eventually abandoned and not given a quality fork then you are suddenly in need of solving this dependency in an entirely new way.
Olreich 2y ago
Tool-chain dependencies are still dependencies. If you need autoconf, cmake, python, perl, and gcc to build, you’ve made yourself a giant pile of garbage to deal with compared to just having a bash script that you spent two days getting into shape to send incantations to gcc.

wutwutwat 2y ago

> I also find it difficult to rationalize third-party dependencies these days Your home grown auth code doesn’t have the minds and the scrutiny of security experts like the go-to third party auth lib everyone uses does, nor does it have the free pen testing, security fixes, maintenance, optimizations, refactors, ever collaborative maintainers, or corporate sponsorships. Auth is a solved problem by minds much smarter than my own. I’m glad to outsource things like that to third party deps.

skydhash 2y ago
Is auth that difficult? The truly difficult problem is cryptography and that's the code you really don't want to write. But most authentication schemes are well-known. No need to import a library, and if you do, you should be prepared to vendor it in.
- wutwutwat 2y ago
  > most authentication schemes are well-known so why maintain your own implementation of the scheme/spec when a community already does, as well as handles the vulns as they pop up and generally has more eyes looking (and fixing) issues? Here's how I look at it; Trust my own NIH implementation of a spec and hope a user reports a vuln instead of it going undisclosed/unnoticed in the wild for years, or use a community lib that is always being scrutinized and vetted and has security professionals involved. It's a no brainer for me.
cowsandmilk 2y ago
I would go beyond auth to areas where there historically have been security bugs. Parsing JPEG files for instance, I can write a parser and can probably choose a modern safe language to avoid many classes of bugs, but a popular open source library is more likely to have been fuzzed by dozens of techniques from multiple companies and universities. The code is more resilient because it has been out there to be experimented on in public.
bsder 2y ago
> Auth is a solved problem by minds much smarter than my own. Is it? It seems like all the outsourced auth has lots and lots and lots of problems, issues, etc. along with being remarkably expensive. I'm not saying I, personally, can do better, but it seems like auth is far from a solved problem. Especially since it seems like the difficult part of "auth" is actually "customer support" rather than any technical issue. (I would argue that the lack of an open-source "Auth in a Box" seems to also argue that auth is far from "solved".)
- wutwutwat 2y ago

rramadass 2y ago

> Something that has also worked well for me historically is aggressively updating tool chains as an integral part of the development process. ... I've adopted a policy that for every significant compiler or build system release, we create a branch that verifies a good build. If it doesn't build without warnings or errors, or fails in testing, that is treated the same as a bug in the code to be dealt with ASAP. ... This also makes it much easier to incrementally modernize and refactor the code base with new language features -- the tool chain supports the latest and greatest even if the code base doesn't yet. Good Point.

cpeterso 2y ago

> I've adopted a policy that for every significant compiler or build system release, we create a branch that verifies a good build. Mozilla's Firefox CI builds with multiple compilers, including clang trunk and (IIRC) Rust Nightly compilers to identify code issues that will need to be fixed before Firefox can update to the new compiler versions and to catch regressions in the compilers themselves.

szvsw 2y ago

> I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks. What kind of software development do you do? As someone who is in ML/data science and works primarily in python, occasionally TS/React, at a small company (about 5 FTE, one round of funding), that sentence is so completely foreign to me. Again, this might just be because I am working primarily on ML systems in Python on relatively small teams (also in academia at the same time), where the most basic building blocks like Pandas (or Polars), PyTorch, XGBoost, SKLearn, etc are so essential and so it’s pretty hard to imagine ever being at a scale where they would be feasible or worth it to replace with our own code… and anyways if support for libraries like those collapses, the only explanation is that the entire landscape has shifted so much that it’s hard to imagine the underlying tech of the company not also having naturally shifted with it… Maybe I am misunderstanding how you are using “3rd party,” or maybe you are just operating in a completely different staffing and financial context which I don’t have experience with. Just curious for more details about what kind of world you operate in where that philosophy is viable!

all2 2y ago
I do SW in a hardware world that restricts a lot of dependencies. We vendor some, but mostly roll or own libraries. For time intensive things peripheral to our work we will rely on 3rd party modules (ie, network stack or UI).
crabbone 2y ago
You chose the field with one of the worst rates of churn and brokeness of third-party libraries, and by some kind of magic managed to avoid it? You should be buying lottery tickets by a truckload! All the projects you listed (Pandas, PyTorch, SKLearn) have a huge list of compatibility issues, on top of compatibility issues of Python itself. And I know this because I have to support various research projects, which typically use this stuff. My estimate is that the shelf-life of a project using just these three will not exceed 3 years in >90% of cases. My typical situation with these projects is that I'm called in because some former PhD student, who graduated a year or two ago left a project that now another student is picking up. And nothing works. Results cannot be reproduced. The dependencies cannot be installed. And so on. And then, depending of the amount of effort I'm allowed to put into fixing the project, I end up either crafting some sort of a spec that tries to find a set of versions of used libraries that seem to make the project work again. Or, more commonly, have to vendor some code and add fixes on top to make the packages fit together at least somehow. Sometimes I even end up repackaging someone else's code (a dependency), where the version used before cannot be found anymore, or I cannot figure out the combination of version that could possibly make the program work. I hear that it's worse in JavaScript world. And, my experience with LaTeX packages has also been marred by similar problems (but not anywhere near the extent of how bad it is in Python). On the other hand, I had to build absolutely atrocious (math operations on a mix of signed and unsigned ints, modifying const char etc.) C code (pre-K&R, the kind where they write function argument types after parenthesis), and it was much easier to deal with than any "modern" Python (i.e. 3-4 years old). If anything, Python is the poster child of the problem of unreliability of third-party dependencies.

t-writescode 2y ago

> I also find it difficult to rationalize third-party dependencies these days I would love a deeper insight into what you mean by this. I can't imagine you're rolling your own http server, database communication layer (as in like, postgresql's jdbc driver), json parsing engine, etc.

throwaway290 2y ago
JSON parser, http server are provided by many runtimes and if a runtime does not provide something it provides primitives that make it often trivial to build
- lmz 2y ago
  I'm not saying it's impossible, but writing your own HTTP server ("trivial to build"?) to avoid bugs in other implementations doesn't sound like a good idea to me.
  - throwaway290 2y ago
    Read more carefully. We talk about "third-party dependencies" not about "other implementations". Using HTTP server from your runtime is not against the recommendation of avoiding third-party dependencies. Your runtime is usually not counted as "third-party dependency", normal runtimes don't have breaking changes every other moon (Even if you need to implement something like that from scratch it may be OK, your requirements don't need all of the functionality a third package includes because someone else needed it)
    - lmz 2y ago
      It's the "trivial to build" thing I have issues with - especially for the HTTP server. Not runtime vs third party.
  - rurban 2y ago
    I had to write one recently. One third page of code, multi-threaded. Really trivial, and no security nightmares as with overblown 3rdparty deps.

Lutger 2y ago

Fully agree, though I feel there are some nuances with third party dependencies. Consider the following: Some third party dependencies are so widely adopted and supported, they are on-par with the main framework you are using in terms of support and development. They often don't lag the main framework in supporting new versions for long. I don't think these kinds of dependencies have the same drawbacks. You only have a couple at this level anyway, and should choose them carefully. There are also dependencies that solve one particular problem well and very narrowly couple to a specific feature in your product. If they become problematic over time, it is easy to just replace them or even rewrite it yourself. Specifically, you could select these judging how well you could maintain the sourcecode yourself in such an event. Often, these are easy to extend as the focus is so specific. You can consider just adding it to your codebase and thinking of these dependencies as your code, except that it is written by someone else.

jcarrano 2y ago

I had the same experience with Buildroot. The project was stuck at an older release and it was kind of a time-bomb: as time went on it became harder to update because customizations kept piling on but at the same time there were obsolete packages (some with CVEs) which had to be upgraded ASAP.

vv_ 2y ago

> I also find it difficult to rationalize third-party dependencies these days This depends on what kind of dependency you're talking about. For example using Zephyr RTOS significantly reduces time to market and it brings in a lot of 3rd party libraries that are written specifically for it. In other fields this might differ, but fields like Web development are a nightmare to begin with.

MetaWhirledPeas 2y ago

> I also find it difficult to rationalize third-party dependencies these days, they almost all become disappointments over the long-term. The benefits rarely justify the risks. I think I understand your sentiment here. For me the risk is that you are not just trusting the original author, you're trusting all future maintainers, especially if you follow your first (and good) suggestion of "aggressively updating tool chains". The other risk is what you are mitigating by aggressively updating: things break through decay and obsolescence. That can usually be mitigated through updates, but if you introduce a dependency that is eventually abandoned and not given a quality fork then you are suddenly in need of solving this dependency in an entirely new way.

Olreich 2y ago

Tool-chain dependencies are still dependencies. If you need autoconf, cmake, python, perl, and gcc to build, you’ve made yourself a giant pile of garbage to deal with compared to just having a bash script that you spent two days getting into shape to send incantations to gcc.