4 ms·

(2022) I remember this one because of this part: > Unbeknownst to me, even with --dry-run pip will execute arbitrary code found in the package's setup.py. In

by gary_0 2y ago

(2022) I remember this one because of this part: > Unbeknownst to me, even with --dry-run pip will execute arbitrary code found in the package's setup.py. In fact, merely asking pip to download a package can execute arbitrary code (see pip issues 7325 and 1884 for more details)! So when I tried to dry-run install almost 400K Python packages, hilarity ensued. I spent a long time cleaning up the mess, and discovered some pretty poor setup.py practices along the way. But hey, at least I got two free pictures of anime catgirls, deposited directly into my home directory. Convenient!

danwills 2y ago
I don't use pip but tbh this makes me much less likely to even try it. So many things about it sound unsafe. Like dry-run not even working as you'd expect? I wonder if that got fixed after this article?
- cozzyd 2y ago
  Probably the fix will be to rename the option --dank-run
odyssey7 2y ago
Why is this ecosystem considered acceptable? It’s because academia is a sort of anchor that preserves poor practices.
- noahssnark 2y ago
  You'll have to ask every developer why it's acceptable, because they keep using this ecosystem. And then ask their managers. But this is what happens when you don't use package ecosystems with maintainers being run by adults.
  - physicsguy 2y ago
    If I wanted to compile a Go binary using an existing dynamically loaded C library, compiled with -ffast-math, what happens?
- cozzyd 2y ago
  Well it's an ecosystem that is meant to be easy to use by people who don't know what they're doing.
- Doxin 2y ago
  > Why is this ecosystem considered acceptable? It largely isn't, but python is a big ship and it takes time to turn. There's been a lot of movement in python packaging semi-recently and as far as I can tell using setup.py has been considered legacy for a while now. Biggest problem is that the new way to do packaging is not documented very well. It's split over three or four different projects. It mostly amounts to "you need to make a pyproject.toml file" but it's somewhat tricky to find what such a file is even supposed to contain.
  - morkalork 2y ago
    Again, why is this acceptable?
    - Joker_vD 2y ago
      Okay, it's unacceptable. Now what? Sadly, we (you, I, the Python leadership) can't just visit every single Python package publisher/maintainer and ask politely but firmly ask them to fix their packaging issues. And even if we could, at least half of them would either ignore us or would mess up in some new, exciting ways while trying to adapt to the flavour du jour of Python packaging.
      - morkalork 2y ago
        I think the answer is to not have a flavour du jour. Pick one, crown it the winner, document it well and give it some time. If the solution is always changing, nobody is going to want to put the effort in.
        oivey 2y ago
        This is really a great idea. Once we finalize the choice, when can we expect the upload into the hivemind?
        Joker_vD 2y ago
        Right after the maintainers of libtelepathy.so release the version with Discord support.
- physicsguy 2y ago
  The same problems happen in any language that links C and C++ libraries largely. It's less common for other languages because they just don't do it as much.
poikroequ 2y ago
I've had to clean up after npm packages as well. Git hooks I didn't ask for. Shell modifications I didn't ask for. I hope that, one day, these package managers will utilize sandboxing/containerization to avoid messes like this.
- richardfey 2y ago
  Python and node software is something I always try to run in containers for convenience and security reasons
zbentley 2y ago
The "--only-binary=:all:" option should force Pip to only install packages that provide wheels, which doesn't run arbitrary setup.py code (right? I've been assuming it doesn't and really hope that is a valid assumption). That will cause some installations to fail if wheels are not available. However, given that wheels are increasingly common (even for pure-python "source" packages), this can be used as a sort of bisect to enumerate and isolate/audit/file issues on/sandbox/etc. the remaining setup.py-based distributions.
- physicsguy 2y ago
  Depends on the platform you're on, some platforms have very few wheels.
Sakos 2y ago
Naive question, does this have the same effect if I use virtual environments?
- plorkyeran 2y ago
  Yes. Virtual environments aren’t a full sandbox.
- aidenn0 2y ago
  Yes, because the line that does it is literally: os.system('cp -r ./pics/ ~/.local/share/koneko/')
  - cozzyd 2y ago
    that's a feature. You can conveniently do pip install nukemydrive #no, this package doesn't exist (yet) rather than than the harder to remember rm -rf --no-preserve-root /
aidenn0 2y ago
Here's the line that copies those pictures in: https://github.com/akazukin5151/koneko/blob/master/setup.py#L18 https://github.com/akazukin5151/koneko/blob/master/setup.py#...

by gary_0 2y ago

danwills 2y ago

I don't use pip but tbh this makes me much less likely to even try it. So many things about it sound unsafe. Like dry-run not even working as you'd expect? I wonder if that got fixed after this article?

cozzyd 2y ago
Probably the fix will be to rename the option --dank-run

odyssey7 2y ago

Why is this ecosystem considered acceptable? It’s because academia is a sort of anchor that preserves poor practices.

noahssnark 2y ago
You'll have to ask every developer why it's acceptable, because they keep using this ecosystem. And then ask their managers. But this is what happens when you don't use package ecosystems with maintainers being run by adults.
- physicsguy 2y ago
  If I wanted to compile a Go binary using an existing dynamically loaded C library, compiled with -ffast-math, what happens?
cozzyd 2y ago
Well it's an ecosystem that is meant to be easy to use by people who don't know what they're doing.
Doxin 2y ago
> Why is this ecosystem considered acceptable? It largely isn't, but python is a big ship and it takes time to turn. There's been a lot of movement in python packaging semi-recently and as far as I can tell using setup.py has been considered legacy for a while now. Biggest problem is that the new way to do packaging is not documented very well. It's split over three or four different projects. It mostly amounts to "you need to make a pyproject.toml file" but it's somewhat tricky to find what such a file is even supposed to contain.
- morkalork 2y ago
  Again, why is this acceptable?
  - Joker_vD 2y ago
    Okay, it's unacceptable. Now what? Sadly, we (you, I, the Python leadership) can't just visit every single Python package publisher/maintainer and ask politely but firmly ask them to fix their packaging issues. And even if we could, at least half of them would either ignore us or would mess up in some new, exciting ways while trying to adapt to the flavour du jour of Python packaging.
    - morkalork 2y ago
      I think the answer is to not have a flavour du jour. Pick one, crown it the winner, document it well and give it some time. If the solution is always changing, nobody is going to want to put the effort in.
      - oivey 2y ago
        This is really a great idea. Once we finalize the choice, when can we expect the upload into the hivemind?
        Joker_vD 2y ago
        Right after the maintainers of libtelepathy.so release the version with Discord support.
physicsguy 2y ago
The same problems happen in any language that links C and C++ libraries largely. It's less common for other languages because they just don't do it as much.

poikroequ 2y ago

I've had to clean up after npm packages as well. Git hooks I didn't ask for. Shell modifications I didn't ask for. I hope that, one day, these package managers will utilize sandboxing/containerization to avoid messes like this.

richardfey 2y ago
Python and node software is something I always try to run in containers for convenience and security reasons

zbentley 2y ago

The "--only-binary=:all:" option should force Pip to only install packages that provide wheels, which doesn't run arbitrary setup.py code (right? I've been assuming it doesn't and really hope that is a valid assumption). That will cause some installations to fail if wheels are not available. However, given that wheels are increasingly common (even for pure-python "source" packages), this can be used as a sort of bisect to enumerate and isolate/audit/file issues on/sandbox/etc. the remaining setup.py-based distributions.

physicsguy 2y ago
Depends on the platform you're on, some platforms have very few wheels.

Sakos 2y ago

Naive question, does this have the same effect if I use virtual environments?

plorkyeran 2y ago
Yes. Virtual environments aren’t a full sandbox.
aidenn0 2y ago
Yes, because the line that does it is literally: os.system('cp -r ./pics/ ~/.local/share/koneko/')
- cozzyd 2y ago
  that's a feature. You can conveniently do pip install nukemydrive #no, this package doesn't exist (yet) rather than than the harder to remember rm -rf --no-preserve-root /

aidenn0 2y ago

Here's the line that copies those pictures in: https://github.com/akazukin5151/koneko/blob/master/setup.py#L18 https://github.com/akazukin5151/koneko/blob/master/setup.py#...