7 ms·
I'm not going to say where and I'm not asking for interested parties (we're not currently hiring anyway), but where I work (as a kernel/OS developer), package m
by 0x0203 3y ago
I'm not going to say where and I'm not asking for interested parties (we're not currently hiring anyway), but where I work (as a kernel/OS developer), package maintenance and management has been an ever-increasing burden to the point that we've tried hiring somebody specifically for the role. Unfortunately, it seems like this is a very specialized skill set that the vast majority of developers seem to find boring, uninteresting, and drudgerous, and finding someone qualified and interested has been exceedingly difficult. How would someone/a company go about finding people that are actually interested in this kind of work? There are clearly people who take on this task (some may even enjoy it!), and many who even do it for free for various open source projects, but I've no clue how to recruit or connect with any that would be willing to do this for a job. Any tips for finding or connecting with such folks would be much appreciated.
Attending in person won't be possible, but I'll have to keep an eye on this conference and see if they provide a way to post/share such opportunities.
- seabass-labrax 3y agoI think you might have more success advertising to sysadmins or devops engineer rather than developers. It's precisely the negative attitude, or at least apathy, towards packaging that makes it difficult to find employment where packaging is valued. On the other hand, a lot of the packaging related work has traditionally been the remit of the UNIX/Linux system administrator, and that as a profession is slowly moving towards automation as 'devops' - but it's still fundamentally the same concepts. Also, you could sprinkle in some 'optional but desired' skills and experience relating to packaging into your advertisements, anything from CMake and GNU Autotools to Nix and Podman. That would be enough to pique a packager's interest and differentiate it from the 'write-only' coding jobs!
- saurik 3y agoI will second this advice; both of the engineers I have worked with who actively enjoyed being tasked with packaging software identified as systems operations / devops / site reliability engineering.
- 0x0203 3y agoThese are both very useful comments; thank you! In your experience, do the devops/sys ops engineers typically have useful coding experience? In our environment, porting a package to our OS may require a little, or sometimes a fair bit of modification to make it work and we often need to be able to read/understand the upstream code in whatever language it's in and change/add to it. Hence our (or at least my) assumption that we'd need a "developer."
- AaronM 3y agoSome do, maybe more with SRE title's. Better to put that in ajob description and filter for those with development backgrounds.
- mch82 3y agoIn my experience creating packages for an operating system package manager or Docker images, a DevOps or sysadmin mindset is more beneficial than a developer mindset. For those packages, the goal is to follow operating system or deployment best practices. I’ve not created a Python or Rust package or something like that. If you’re creating that type of package, my guess is that it may help a lot to hire a developer who loves to think about the most “idiomatic” way to package your code. Edit: if your package will be published and used by customers, then you might look for DevRel skill as a bonus.
- faeriechangling 3y agoA good sysadmin may not be able to do graphics programming, but they have plenty of practice in the sort of programming used in package management, as well as a fairly good understanding of operating systems. They’re not going to grasp everything as well as somebody with a development background, like learning NEW programming languages on the fly and upstreaming changes, but I don’t think it’s totally outside of what they can learn and hiring a developer would present just as many challenges. I might look more for somebody who bills themselves as “devops” or “sre” than sysadmin though since those job descriptions tend to effectively filter out those who can’t program at all. If anything I see those people as being software developers that specialize in systems administration.
- larusso 3y agoI not 100% convinced that profile works either. I tried multiple times to hire for my team which works with a combination of library maintenances build tool development and package management. I even tried a junior developer with an open mind. The skills needed for this job is sadly to broad. One needs to know the platform/language and tools to package software. And don‘t get me started with CI etc. Sysadmins and what most understand of DevOps are not the right kinda people. Not to say there are not people among this group.
- devonkim 3y agoThe most appropriate group of folks may be those that already maintain software for the OSS community. The skillset more resembles trades where time and experience matters substantially more than any intrinsic talent or overall intellect to be effective at the job. Honestly, sometimes I wonder if this is perhaps an under-considered area for Generative AI applications given how much drudgery is involved yet precision tends to matter.
- larusso 3y agoMaybe… I mean yes at the bottom most package system are a zip with some kind of manifest or a manifest which describes where and how the lib can be build from which sources. If an AI could deal with the fine details of these systems… I mean Debian packages are harder than Arch packages, or at least that is my experience. There packages which are super easy to work with like native python, ruby and Java. But mix it with c extensions and boom. The CI can get complicated.
- devonkim 3y agoIf there's a lot of thankless, mundane work for the majority of projects that seems rather routine from what I've seen described thus far across multiple decades, so this seems quite ripe for some form of automation sans projects that seem to keep requiring changes constantly. C extensions, interop across languages, and ABI considerations are certainly much more complicated and require a lot of complicated work but given that 90% of the issues even power users run into are things like "the lxml library I installed won't get detected by package X when I run autoconf, please help." There's nothing sustainable as an industry about something like software dependency management and versioning being so critical for things "just working" while people are sticking their heads in the sand saying "not my problem" as a rule. I'm so tired of solving the same problems that were already solved in the past because of developers wanting to work on "more interesting things" and doing the bare minimum to get stuff compiling and running again regardless of the package format. In fact, Docker may have made this situation even worse with so many Dockerfiles that will basically be unable to work. This is part of why I'm interested in the Nix ecosystem because there's so much work being done to make software installations repeatable and easier to manage. Unfortunately, it's squarely aimed at developers and is way, way too complex for most sysadmins to put the time into understanding, especially when they're so overwhelmed like everyone else with so many tasks.
- elsonrodriguez 3y agoFind a sysadmin and point them at FPM: https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm
- 0x0203 3y agoI'd not seen that project; definitely something I'll be looking at, thanks. Unfortunately, for what we're doing, it doesn't look like this will work as is... certainly worth some investigation though.
- dima55 3y agoGood lord. I'll never understand why people promote this project, since it's main feature is that he's doing it wrong. The complexity he's trying to avoid exists for a reason, not because people like complexity. OP: post here: https://lists.debian.org/debian-jobs/ https://lists.debian.org/debian-jobs/
- bravetraveler 3y agoPlease heed this - FPM is basically versioned tarballs. It's only useful for the most simple/contrived scenarios Actual package specs require effort for good reason. It's not just an archive... but interdependence, steps to perform on (un)installation, changelogs, and so on. There are some helpers already provided, ie: RPM macros. Sure, they're esoteric, but show me a specialization that isn't. Refer to the Fedora packaging guidelines and enjoy life.
- viraptor 3y agoIt depends on what you want as a result. Do you want the shortest path to "dpkg -i the-result.deb" working on your server, or do you want a package complying with debian standards around behaviour and file locations, with packaging/source changelog split, potentially upstreamable? Because the second one is a lot more work, but few people actually need that.
- dima55 3y ago
- tguvot 3y agoif it's for some kind of internal systems or shippable hardware/appliances, best approach is to rethink approach to packaging
- 8organicbits 3y agoSome large tech companies dedicate multiple teams to package management and installation. It's a unique enough problem that you may not want developers to focus on it, better to roll it up to a dedicated team. They usually work closely with release operations.
- mch82 3y agoMy packaging experience is limited, but I found the docs were quite poor. It took perseverance & researching install error messages to create an RPM that worked. So you might consider hiring for people willing to do that kind of work. The good news is that I learned a tremendous amount about where programs and files should actually be installed through packaging. It was surprising to me how many conventions package managers enforce. I was also surprised how poorly software followed those conventions when installed without package managers. Packaging improves user experience, developer experience, and quality. Can’t recommend it enough!
- pxc 3y ago> It took perseverance & researching install error messages to create an RPM that worked. So you might consider hiring for people willing to do that kind of work. At its toughest, packaging is a long series of exchanges: one build error for another. It is a stubborn, patient, rabbithole diving, well-I'll-be-damned, yak-be-shorn kind of work. > The good news is that I learned a tremendous amount[.] [...] Can’t recommend it enough! And it's totally worth it, both for the packager and their users. :D
- Waterluvian 3y agoIf it requires skill but it’s boring, pay them enough to care.
- scrollaway 3y agoI’m a maintainer at Arch Linux and have been in FOSS for long enough to know the landscape and who’s skilled looking for work. Send me an email with your needs and we can talk. As others have said, what you’re hiring for is more of a specialised sysadmin role. I agree with leaving the software engineering label behind.
- bravetraveler 3y agoI find it's a mildly common skill among system administrators/SREs. I personally picked this up, and developed it, in these circles across many companies Good ones are a bit rare... as this tends to go They tend to write automation/utilities that help keep the lights on - packages are an important part in distributing that.
- chrismarlow9 3y ago> There are clearly people who take on this task (some may even enjoy it!), and many who even do it for free for various open source projects, I feel like I missed something obvious here. Doesn't this answer your own question? I've worked for companies before that found great talent for very unique skills by contributing to similar projects and reaching out to frequent commit authors.
- yjftsjthsd-h 3y agoI'd be interested in the other direction: I accidentally stumbled into a position where I get to do packaging work, which is one of my favorite parts of the job. But like... that was an accident. As much as I would love to do more work in the same vein, I have no clue how I would intentionally seek it out if I were job hunting.
- pxc 3y agoTo corroborate my claims from another comment, I'm calling you out as a recognizable Nix fan on HN. ;) I told you us Nix weirdos enjoy packaging, GP!
- yjftsjthsd-h 3y agoLol:) I mean, I was into packaging before I started using nix... it just happens to be an often nice option:P
- pxc 3y agoI think that's how it usually goes! That's how it was for me, too, at any rate. :D
- pabs3 3y agoThe large open source companies do a lot of packaging, probably some of the smaller open source consultancies too, maybe try some of the employers listed here: https://github.com/fossjobs/fossjobs/wiki/resources https://github.com/fossjobs/fossjobs/wiki/resources
- ZoomerCretin 3y agoThis attitude is exactly why hiring sucks. You and everyone else want to find your purple squirrel, your Python developer with more years of experience in Python than Guido Van Rossum. Why is it so hard to accept that the best candidate you will ever get may be someone with relevant but not _exactly_ the experience that you want?
- 0x0203 3y agoHaving been on both sides of this equation, I don't think I'm being unreasonable, and am well aware of the fact that good candidates and the "right" hire will rarely be the ones that check off a list of random attributes plucked from the thin air between the pointy-haired bosses ears. I'm not looking for a purple squirrel, and I'm not looking for someone with 25 years experience in rust or the like. I'm looking for someone who both has an interest in, and a capacity to do a somewhat unique job we need done. We've found plenty of qualified people, and hired some, many of whom tend to be fresh graduates with zero non-academic experience. What I've had a difficult time finding, even figuring out how to look for, are the people that have the interest or desire to work on this particular type of niche problem. Few developers out there seem to, and we've found even less, so the package porting/maintenance/management part of our job continues to be a burden for the whole development team. Thus my question. And thankfully, some insightful replies suggest that I'm mainly looking in the wrong category of candidates. All of which has nothing to do with matching an overly strict list of experience requirements.
- predictabl3 3y agoLiterally go to nixpkgs/nixos areas and ask for who's looking for a job? Well, those folks are going to want to do Nix packaging, so... ymmv. Also, my other comment about Nix in this thread has gone through a rollercoaster so uh, that's sure interesting. Hard to not raise an eyebrow at.
- hobofan 3y agoIt might be helpful to know that this niche is mostly organizing itself under the "release engineering" term. So explicitly putting up job offers for a release engineer might help.
- pxc 3y ago> package maintenance and management has been an ever-increasing burden to the point that we've tried hiring somebody specifically for the role I do packaging work at my current job, and it's one of my favorite tasks there. I currently maintain a small collection of private Nix packages for macOS and Linux, and last week spun up a private Homebrew tap (for Casks only!) on a lark. I have some experience packaging things in other formats, too, and I'd be happy to pick up a new one if I had a use case. But it has literally never occurred to me to look for a packaging job specifically! I didn't know roles dedicated to packaging were available anywhere. > How would someone/a company go about finding people that are actually interested in this kind of work? Even when I'm happy at work, I'm generally interested in keeping an eye on jobs where Nix skills are desired. 'Nix' itself is hard to search for (because it's ambiguous with *nix as an abbreviation for Unix-likes), so I also search for 'NixOS' and 'Nixpkgs' on job search sites sometimes. I don't think I've ever once seen a result for 'Nixpkgs', so any listing that mentioned that would stand out, and a job explicitly titled something like 'software packager', 'software distribution engineer', 'package management engineer', etc., would definitely catch my eye. Job boards and development mailing lists for large package collections (Linux distros, BSD ports collections, and other collections like Conda) would also likely have some interested people! (In the case of the latter, just make sure that job posts are in line with the norms of the list.) Communities surrounding advanced build systems (Bazel, Buck, Pants, etc.) would also probably be good places to look.
- boris 3y ago> Unfortunately, it seems like this is a very specialized skill set that the vast majority of developers seem to find boring, uninteresting, and drudgerous, and finding someone qualified and interested has been exceedingly difficult. The reason most find it boring, uninteresting, and drudgerous (I would also add "frustrating" to this list) is because one has to build on top of decades of bad decisions and quick hacks, over and over again. Which also suggests a way to make it interesting: make it about fixing the underlying problem (even if in a limited context) instead of just going along with whatever is available. For example, you can attempt to automate producing various packages from some "sane" common metadata. You didn't provide much context on what exactly you are trying to package, but here is one example of what it might look like: https://build2.org/bpkg/doc/bpkg-pkg-bindist.xhtml https://build2.org/bpkg/doc/bpkg-pkg-bindist.xhtml
- mike_hearn 3y agoOr post it as a job to use AI to automate packaging.
- kailiu42 3y agoLike you said there are many people do it for free for open source projects. Why not try to figure out the reason for them to do so and do so for free? With that reasoning, I believe you maybe can find out some changes to the "job" or the "company culture", or the "expectation from the management", or something else that could help you attract more candidates.
- ensignavenger 3y agoAt my last employer, we were called Systems Engineers. I created tooling to switch us off of a custom packaging solution to using Debian packages. The other part of my project was tooling around Ansible for configuration management across a fleet of over 48,000 vms deployed at over 6,000 locations, with different locations requiring slightly different configurations. I also helped build a web based system to track approvals and schedules for all the changes. I think Dev Ops or Systems Engineers would be the most used title for this type of work.
- franciscop 3y agoI've tried the basics of building a package manager before for fun, and found it a much harder problem than I anticipated (as most things! that's why I experiment by trying to build them, to learn! And sometimes actually build them successfully [1]). So I do agree with the sentiment of finding someone qualified to actually build it and not just a generic dev. Depending on your needs I'd first try to not need the expertise in the first place (a generic public package manager is much more complex than say, a tightly controlled plugin system), or try to leverage existing package managers/tooks. If that fails/not possible, and you cannot find a proper fulltime dev, many open source devs working on these topics might be up for consulting, so I'd reach out to them to try to set the general guideliness/direction and then use in-house talent to fill in the gaps. [1] https://umbrellajs.com/ https://umbrellajs.com/ as a clone/alternative to jQuery
- pabs3 3y agoGo to where the packagers are; the open source distros and related companies. Some of the distros like Debian have lists of consultants, or a jobs list: https://www.debian.org/consultants/ https://www.debian.org/consultants/ https://lists.debian.org/debian-jobs/ https://lists.debian.org/debian-jobs/
- pabs3 3y agoGiven an upstream project with good practices, packaging is fairly easy these days, since most of it is automated. Most of the work is around reviewing upstream changes and fixing any issues found.
- specialist 3y agoAges ago, my QA/Test people reveled in managing releases. I wouldn't know how to find those kind of people today. Just something to think about when interviewing those sysadmin types (recommended by others).
- solarengineer 3y agoI can recommend a few people who are passionate about packaging. My email is in my profile.