4 ms·

Back in the day repositories had 'maintainers' who reviewed packages before they became included. I guess no one really cares in the web dev world; it's a free-

by JTbane 2y ago

Back in the day repositories had 'maintainers' who reviewed packages before they became included. I guess no one really cares in the web dev world; it's a free-for-all.

giantg2 2y ago
Web dev has always been a hot mess.
CamJN 2y ago
It’s not just web dev: go, rust, swift, ruby, python none of them do any checking. In fact the only repo I know of doing any checking is Java’s Maven/Sonotype and it’s automated not manual.
- debugnik 2y ago
  OCaml's opam does have a review process, although I'm not sure how exhaustive. It's got a proper maintenance team checking for package compatibility, updating manifests and removing problematic versions. I don't think this would be viable if the OCaml community grew larger though.
- SunlitCat 2y ago
  And people bash c/c++ for not having some kind of central package management system. Hah!
  - carlmr 2y ago
    IMO C/C++ is not much better, sure, no central package management system, but then people rewrite everything because it's too hard to use a dependency. Now if you do want to use one of the 1000 rewrites of a library, you'll have a lot more checking to do, and integration is still painful. Painless package management is a good thing. Central package repositories without any checking isn't. You don't have to throw away the good because of the bad.
    - lolinder 2y ago
      > Painless package management is a good thing. Central package repositories without any checking isn't. There's a reason why these things come hand in hand, though. If the package management is so painless that everyone is creating packages, then who is going to pay for the thoroughly checked central repository? And if you can't fund a central repository, how do you get package management to be painless? The balance that most language ecosystems seem to land on is painless package management by way of free-for-all.
      - carlmr 2y ago
        >And if you can't fund a central repository, how do you get package management to be painless? You could host your own package server with your own packages, and have the painless package manager retrieve these painlessly. Of course we're in this situation because people want to see the painlessness with what other people built. But other people includes malicious actors every once in a while.
    - bluGill 2y ago
      I have that in C++: we wrote our own in house package manager. Painless for any package that has passed our review, but since it is our manager we have enforced rules that you need to pass before you can get a new package in thus ensuring it is hard to use something that hasn't been through review. I'm looking at rust, and that it doesn't work well with our package manager (and our rules for review) is one of the big negatives! Note, if you want to do the above just use Conan. We wrote our package manager before Conan existed, and it isn't worth replacing, but it isn't worth maintaining our own. What is important is that you can enforce your review rules in the package manager not what the package manager is.
  - JTbane 2y ago
    Correct me if I'm wrong but the usual advice in the C/C++ world is just grab the source code of any libraries you want and build them yourself (or use built-in OS libs). This is not great if you have a lot of dependencies.
  - rcxdude 2y ago
    Yeah! nothing like the xz backdoor could happen there! wait...
- bqmjjx0kac 2y ago
  I'm curious how much review happens in Nix packages. It seems like individual packages have maintainers (who are typically not the software authors). I wonder how much latitude they have to add their own patches, change the source repo's URL, or other sneaky things.
  - bamboozled 2y ago
    Not a lot in most cases. You’re still just grabbing a package and blindly building whatever source code you get from the web. Unless the maintainer is doing their due diligence nothing. Goes the same for almost all packages in all distros though. I’d say most of us have some connection to what we’re packaging but there are plenty of hastily approved and merged “bump to version x” commits happening.
  - jowea 2y ago
    Nixpkgs package maintainers don't usually have commit rights. I assume that if one tried to include some weird patch, the reviewer would at least glance at it before committing.
    - c0wb0yc0d3r 2y ago
      I’ve never looked at the process of making a nixpkg, but wouldn’t the review process only catch something malicious if it was added to the packaging process? Anything malicious added to the build process wouldn’t show up correct? At least not unless the package maintainer was familiar and looked themself?
      - jowea 2y ago
        I am not sure I understand the distinction between the packaging and build process, at least in the context of nixpkgs. Packages in nixpkgs are essentially build instructions, which you can either build/compile locally (like Gentoo) but normally you download them from the cache. Official packages for the nixpkgs cache are built/compiled on Nix's own infrastructure, not by the maintainers, so you can't just sneak malicious code in that way without cracking into the server. What package maintainers do is contribute these build instructions, called derivations. Here's an example for a moderately complex one: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/mindforger/default.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio... https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/mindforger/hunspell_pkgconfig.patch https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio... As you can see, you can include a patch to the source files, add custom bash commands to be executed and you can point the source code download link to anywhere you want. You could do something malicious in any of these steps, but I expect the reviewer to at least look at it and build it locally for testing before committing, in addition to any other interested party.
- BrenBarn 2y ago
  Some alternative sources for other languages do it. Conda-forge has a process that involves some amount of human vetting. It's true that it doesn't provide much protection against some kinds of attacks, but it makes it harder to just drop something in and suddenly have a bunch of people using it without anyone ever looking at it.
reaperducer 2y ago
Back in the day repositories had 'maintainers' who reviewed packages before they became included. Then "walled garden" became a pejorative, and well… here we are.
- skeaker 2y ago
  Pardon? Maybe my definition of that term is just different but that seems wholly unrelated.

by JTbane 2y ago

Back in the day repositories had 'maintainers' who reviewed packages before they became included. I guess no one really cares in the web dev world; it's a free-for-all.

giantg2 2y ago

Web dev has always been a hot mess.

CamJN 2y ago

It’s not just web dev: go, rust, swift, ruby, python none of them do any checking. In fact the only repo I know of doing any checking is Java’s Maven/Sonotype and it’s automated not manual.

debugnik 2y ago
OCaml's opam does have a review process, although I'm not sure how exhaustive. It's got a proper maintenance team checking for package compatibility, updating manifests and removing problematic versions. I don't think this would be viable if the OCaml community grew larger though.
SunlitCat 2y ago
And people bash c/c++ for not having some kind of central package management system. Hah!
- carlmr 2y ago
  IMO C/C++ is not much better, sure, no central package management system, but then people rewrite everything because it's too hard to use a dependency. Now if you do want to use one of the 1000 rewrites of a library, you'll have a lot more checking to do, and integration is still painful. Painless package management is a good thing. Central package repositories without any checking isn't. You don't have to throw away the good because of the bad.
  - lolinder 2y ago
    > Painless package management is a good thing. Central package repositories without any checking isn't. There's a reason why these things come hand in hand, though. If the package management is so painless that everyone is creating packages, then who is going to pay for the thoroughly checked central repository? And if you can't fund a central repository, how do you get package management to be painless? The balance that most language ecosystems seem to land on is painless package management by way of free-for-all.
    - carlmr 2y ago
      >And if you can't fund a central repository, how do you get package management to be painless? You could host your own package server with your own packages, and have the painless package manager retrieve these painlessly. Of course we're in this situation because people want to see the painlessness with what other people built. But other people includes malicious actors every once in a while.
  - bluGill 2y ago
    I have that in C++: we wrote our own in house package manager. Painless for any package that has passed our review, but since it is our manager we have enforced rules that you need to pass before you can get a new package in thus ensuring it is hard to use something that hasn't been through review. I'm looking at rust, and that it doesn't work well with our package manager (and our rules for review) is one of the big negatives! Note, if you want to do the above just use Conan. We wrote our package manager before Conan existed, and it isn't worth replacing, but it isn't worth maintaining our own. What is important is that you can enforce your review rules in the package manager not what the package manager is.
- JTbane 2y ago
  Correct me if I'm wrong but the usual advice in the C/C++ world is just grab the source code of any libraries you want and build them yourself (or use built-in OS libs). This is not great if you have a lot of dependencies.
- rcxdude 2y ago
  Yeah! nothing like the xz backdoor could happen there! wait...
bqmjjx0kac 2y ago
I'm curious how much review happens in Nix packages. It seems like individual packages have maintainers (who are typically not the software authors). I wonder how much latitude they have to add their own patches, change the source repo's URL, or other sneaky things.
- bamboozled 2y ago
  Not a lot in most cases. You’re still just grabbing a package and blindly building whatever source code you get from the web. Unless the maintainer is doing their due diligence nothing. Goes the same for almost all packages in all distros though. I’d say most of us have some connection to what we’re packaging but there are plenty of hastily approved and merged “bump to version x” commits happening.
- jowea 2y ago
  Nixpkgs package maintainers don't usually have commit rights. I assume that if one tried to include some weird patch, the reviewer would at least glance at it before committing.
  - c0wb0yc0d3r 2y ago
    I’ve never looked at the process of making a nixpkg, but wouldn’t the review process only catch something malicious if it was added to the packaging process? Anything malicious added to the build process wouldn’t show up correct? At least not unless the package maintainer was familiar and looked themself?
    - jowea 2y ago
      I am not sure I understand the distinction between the packaging and build process, at least in the context of nixpkgs. Packages in nixpkgs are essentially build instructions, which you can either build/compile locally (like Gentoo) but normally you download them from the cache. Official packages for the nixpkgs cache are built/compiled on Nix's own infrastructure, not by the maintainers, so you can't just sneak malicious code in that way without cracking into the server. What package maintainers do is contribute these build instructions, called derivations. Here's an example for a moderately complex one: https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/mindforger/default.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio... https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/mindforger/hunspell_pkgconfig.patch https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio... As you can see, you can include a patch to the source files, add custom bash commands to be executed and you can point the source code download link to anywhere you want. You could do something malicious in any of these steps, but I expect the reviewer to at least look at it and build it locally for testing before committing, in addition to any other interested party.
BrenBarn 2y ago
Some alternative sources for other languages do it. Conda-forge has a process that involves some amount of human vetting. It's true that it doesn't provide much protection against some kinds of attacks, but it makes it harder to just drop something in and suddenly have a bunch of people using it without anyone ever looking at it.

reaperducer 2y ago

Back in the day repositories had 'maintainers' who reviewed packages before they became included. Then "walled garden" became a pejorative, and well… here we are.

skeaker 2y ago
Pardon? Maybe my definition of that term is just different but that seems wholly unrelated.