3 ms·

I not 100% convinced that profile works either. I tried multiple times to hire for my team which works with a combination of library maintenances build tool dev

I not 100% convinced that profile works either. I tried multiple times to hire for my team which works with a combination of library maintenances build tool development and package management. I even tried a junior developer with an open mind. The skills needed for this job is sadly to broad. One needs to know the platform/language and tools to package software. And don‘t get me started with CI etc. Sysadmins and what most understand of DevOps are not the right kinda people. Not to say there are not people among this group.

devonkim 3y ago
The most appropriate group of folks may be those that already maintain software for the OSS community. The skillset more resembles trades where time and experience matters substantially more than any intrinsic talent or overall intellect to be effective at the job. Honestly, sometimes I wonder if this is perhaps an under-considered area for Generative AI applications given how much drudgery is involved yet precision tends to matter.
- larusso 3y ago
  Maybe… I mean yes at the bottom most package system are a zip with some kind of manifest or a manifest which describes where and how the lib can be build from which sources. If an AI could deal with the fine details of these systems… I mean Debian packages are harder than Arch packages, or at least that is my experience. There packages which are super easy to work with like native python, ruby and Java. But mix it with c extensions and boom. The CI can get complicated.
  - devonkim 3y ago
    If there's a lot of thankless, mundane work for the majority of projects that seems rather routine from what I've seen described thus far across multiple decades, so this seems quite ripe for some form of automation sans projects that seem to keep requiring changes constantly. C extensions, interop across languages, and ABI considerations are certainly much more complicated and require a lot of complicated work but given that 90% of the issues even power users run into are things like "the lxml library I installed won't get detected by package X when I run autoconf, please help." There's nothing sustainable as an industry about something like software dependency management and versioning being so critical for things "just working" while people are sticking their heads in the sand saying "not my problem" as a rule. I'm so tired of solving the same problems that were already solved in the past because of developers wanting to work on "more interesting things" and doing the bare minimum to get stuff compiling and running again regardless of the package format. In fact, Docker may have made this situation even worse with so many Dockerfiles that will basically be unable to work. This is part of why I'm interested in the Nix ecosystem because there's so much work being done to make software installations repeatable and easier to manage. Unfortunately, it's squarely aimed at developers and is way, way too complex for most sysadmins to put the time into understanding, especially when they're so overwhelmed like everyone else with so many tasks.
    - larusso 3y ago
      I‘m 100% with you there. I also think that the problems described are solvable on some levels if not all. I remember the time when brew took of on macOS. I used macports at the time. The reason why people preferred brew was the fact that installs where way faster. The reason was simple: brew based everything on system libs where macports similar to nix maintained packages for all libraries etc. It took way longer to install a package with macports. I‘m running Nixos on my private machine (I need to work with macOS at work as the only other option would be windows) and it is sadly way too complicated. Nix flakes make this harder and easier at the same time since nix channels had a way of breaking as well. I switched to nixos because I had issues with running nix packagemanager on other distros (including macOS). It either behaved wrong when installing gui packages or in some cases with a rust package I maintain had issues with glib-c which nearly drove me nuts. As to docker. Yes I also think it made stuff worse in the majority of cases. What people forget to grasp is that Linux be it a vm or a container is just a the kernel and that we have dependencies for some programs and libraries to the kernel. Reason why we have distros in the first place. So not all containers might run the same on each host because it docker runs on the same kernel as the host machine or the VM which installed (macOS) Issues are rare but can happen. Also the number of base images. „Let’s use alpine because that means I build a small container image“ … No it means you build on top of an alpine release with all the installed system libs etc. And that may or may not work with your software. And stuff like that.
pxc 3y ago
> One needs to know the platform/language and tools to package software How long is acceptable for your packagers to struggle with learning a build toolchain and its quirks? How many language ecosystems are you working with at your company? > And don‘t get me started with CI etc. Respectfully, I'd like to get you started on CI. I kinda feel like the landscape of CI platforms right now sucks, and a lot of my daily work is integrating with them. I'm curious what your gripes are!
- larusso 3y ago
  I meant to not starting with the issues of CI but the complexity of it and how many solve it. There is the simple shell script idea which mostly runs on this one CI system and nowhere else. Or the we use a build system which abstracts all kinds of things away but need a good chunk of knowledge to understand why it’s there in the first place. Developers from camp shellscript with the desire to keep it easy ask: „why do this etc etc“. It’s not that their wrong it’s mainly the ignorance of other things like build reproducible builds and platform and system tools abstractions that make these more complex. To just expect that some developer who never dealed with a multi arch/platform product to just willingly jump into the complexity is unfair. I also don’t like the current CI landscape. I still run a Jenkins at work cause it gives me the most freedom and I know it best (it’s sadly what keeps me off other systems, at least I know what parts not to touch). We use gradle which I really liked 5 years ago. We also use it for other non Java systems since it is pretty adoptable. But the recent version changes bring more and more weird changes which make it harder and harder to maintain custom plugins for it. The rules to code build plugins are more stricter enforced by the compiler which yes is good for gradle but hard to maintain as one needs to read up and understand why one can no longer access a file input outsite of a build execution phase for example even though it worked fine for the last 4 versions. And then there is more.

devonkim 3y ago

The most appropriate group of folks may be those that already maintain software for the OSS community. The skillset more resembles trades where time and experience matters substantially more than any intrinsic talent or overall intellect to be effective at the job. Honestly, sometimes I wonder if this is perhaps an under-considered area for Generative AI applications given how much drudgery is involved yet precision tends to matter.

larusso 3y ago
Maybe… I mean yes at the bottom most package system are a zip with some kind of manifest or a manifest which describes where and how the lib can be build from which sources. If an AI could deal with the fine details of these systems… I mean Debian packages are harder than Arch packages, or at least that is my experience. There packages which are super easy to work with like native python, ruby and Java. But mix it with c extensions and boom. The CI can get complicated.
- devonkim 3y ago
  If there's a lot of thankless, mundane work for the majority of projects that seems rather routine from what I've seen described thus far across multiple decades, so this seems quite ripe for some form of automation sans projects that seem to keep requiring changes constantly. C extensions, interop across languages, and ABI considerations are certainly much more complicated and require a lot of complicated work but given that 90% of the issues even power users run into are things like "the lxml library I installed won't get detected by package X when I run autoconf, please help." There's nothing sustainable as an industry about something like software dependency management and versioning being so critical for things "just working" while people are sticking their heads in the sand saying "not my problem" as a rule. I'm so tired of solving the same problems that were already solved in the past because of developers wanting to work on "more interesting things" and doing the bare minimum to get stuff compiling and running again regardless of the package format. In fact, Docker may have made this situation even worse with so many Dockerfiles that will basically be unable to work. This is part of why I'm interested in the Nix ecosystem because there's so much work being done to make software installations repeatable and easier to manage. Unfortunately, it's squarely aimed at developers and is way, way too complex for most sysadmins to put the time into understanding, especially when they're so overwhelmed like everyone else with so many tasks.
  - larusso 3y ago
    I‘m 100% with you there. I also think that the problems described are solvable on some levels if not all. I remember the time when brew took of on macOS. I used macports at the time. The reason why people preferred brew was the fact that installs where way faster. The reason was simple: brew based everything on system libs where macports similar to nix maintained packages for all libraries etc. It took way longer to install a package with macports. I‘m running Nixos on my private machine (I need to work with macOS at work as the only other option would be windows) and it is sadly way too complicated. Nix flakes make this harder and easier at the same time since nix channels had a way of breaking as well. I switched to nixos because I had issues with running nix packagemanager on other distros (including macOS). It either behaved wrong when installing gui packages or in some cases with a rust package I maintain had issues with glib-c which nearly drove me nuts. As to docker. Yes I also think it made stuff worse in the majority of cases. What people forget to grasp is that Linux be it a vm or a container is just a the kernel and that we have dependencies for some programs and libraries to the kernel. Reason why we have distros in the first place. So not all containers might run the same on each host because it docker runs on the same kernel as the host machine or the VM which installed (macOS) Issues are rare but can happen. Also the number of base images. „Let’s use alpine because that means I build a small container image“ … No it means you build on top of an alpine release with all the installed system libs etc. And that may or may not work with your software. And stuff like that.

pxc 3y ago

> One needs to know the platform/language and tools to package software How long is acceptable for your packagers to struggle with learning a build toolchain and its quirks? How many language ecosystems are you working with at your company? > And don‘t get me started with CI etc. Respectfully, I'd like to get you started on CI. I kinda feel like the landscape of CI platforms right now sucks, and a lot of my daily work is integrating with them. I'm curious what your gripes are!

larusso 3y ago
I meant to not starting with the issues of CI but the complexity of it and how many solve it. There is the simple shell script idea which mostly runs on this one CI system and nowhere else. Or the we use a build system which abstracts all kinds of things away but need a good chunk of knowledge to understand why it’s there in the first place. Developers from camp shellscript with the desire to keep it easy ask: „why do this etc etc“. It’s not that their wrong it’s mainly the ignorance of other things like build reproducible builds and platform and system tools abstractions that make these more complex. To just expect that some developer who never dealed with a multi arch/platform product to just willingly jump into the complexity is unfair. I also don’t like the current CI landscape. I still run a Jenkins at work cause it gives me the most freedom and I know it best (it’s sadly what keeps me off other systems, at least I know what parts not to touch). We use gradle which I really liked 5 years ago. We also use it for other non Java systems since it is pretty adoptable. But the recent version changes bring more and more weird changes which make it harder and harder to maintain custom plugins for it. The rules to code build plugins are more stricter enforced by the compiler which yes is good for gradle but hard to maintain as one needs to read up and understand why one can no longer access a file input outsite of a build execution phase for example even though it worked fine for the last 4 versions. And then there is more.