5 ms·
> It's a bit ridiculous that setup.py allows you to run custom code on install. Agreed! The good news is that it's slowly getting better: just yesterday[1], se
by woodruffw 4y ago
> It's a bit ridiculous that setup.py allows you to run custom code on install.
Agreed! The good news is that it's slowly getting better: just yesterday[1], setuptools added support for PEP 660[2], which removes one of the last remaining reasons to use a `setup.py` instead of a fully declarative `pyproject.toml`.
Edit: Also, just to qualify: this is a consistent problem across packaging ecosystems. Even system package managers (reasonably!) allow custom code during installation, e.g. to add service hooks and update locale files.
[1]: https://twitter.com/juanluisback/status/1557734536586625025 https://twitter.com/juanluisback/status/1557734536586625025
[2]: https://peps.python.org/pep-0660/ https://peps.python.org/pep-0660/
- metadat 4y agoHopefully next they'll incentivize package maintainers to migrate to the declarative way. Perhaps spit a scary sounding security warning for all packages which use a setup.py?
- woodruffw 4y ago> Perhaps spit a scary sounding security warning for all packages which use a setup.py? This would be nice, but there's probably too many practical barriers to this. For example, you can't (easily) replace `setup.py`'s dynamism for packages that contain native packages: they need to be able to spawn compilers, etc. and do so in ways that are hard to describe declaratively. We probably don't want to provide a security warning on every native non-wheel package, so that's a no-go for the time being.
- falcor84 4y agoCould we perhaps sandbox any such compilation steps within a container and then copy out the resulting artifact(s), with a hook that does some security scanning?
- woodruffw 4y agoThis is hard to do in the general case: the compilation steps often need system dependencies or to otherwise query system state to do the right thing, which means that the sandbox has to accurately reflect the host. In terms of hooks, audit hooks and events[1] get us some of the way there. But they can be circumvented as well; I wouldn't rely on them for untrusted code. [1]: https://docs.python.org/3/library/audit_events.html https://docs.python.org/3/library/audit_events.html
- de6u99er 4y agoShould start with a warning, and move after one year to users having confirm that they allow pqckqges to execute setup.py and ultimatrly not allow it any more.
- woodruffw 4y agoThis would probably break too much of the ecosystem, since many top Python packages include native extensions that can only be built with a dynamic `setup.py`. Wheels are solving that problem, but it's likely going to be many years before warning about `setup.py` is reasonable and not disruptive to the majority of legitimate users.
- plonk 4y agoInstead there's a scary deprecation warning when a package uses setup.cfg, which is another declarative replacement for setup.py. And some pyproject.toml features are still experimental per the setuptools docs, so I'm not sure which method to choose for our internal projects. I'm glad the setuptools/pip/Python devs are cleaning up the packaging situation though. pyproject.toml seems perfect, when it becomes ready.
- mmastrac 4y agoI published my first Python code in a while using a pyproject.toml and it was amazingly simple. As someone who was doing Rust for a while I really feel like they're learning from the modern ecosystems very well.
- bbkane 4y agoDo you have a link to a good tutorial for this?
- plonk 4y agoFollow the setuptools doc [1], it's very easy and there's an example you can start from. It links to PEP 621 [2] which seems to be the specification if that can help. [1] https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html https://setuptools.pypa.io/en/latest/userguide/pyproject_con... [2] https://peps.python.org/pep-0621/ https://peps.python.org/pep-0621/
- bjt2n3904 4y agoIs toml still not an included dependency in python?
- woodruffw 4y agoNot in 3.10. It's included in 3.11. (But this doesn't matter for packaging purposes, since setuptools is also not included in the base distribution, but does include TOML support.)
- plonk 4y agoThere will be a `tomllib` in Python 3.11: https://docs.python.org/3.11/library/tomllib.html https://docs.python.org/3.11/library/tomllib.html
- cuteboy19 4y agoWhy the discrepancy between csv having the csv package but toml having the lib suffix
- ViViDboarder 4y agoStill doable with pyproject. I actually wrote a Python tool for downloading binary releases from GitHub and Gitea and wrote a “builder” module that can be used in a pyproject file to tell it to download an arbitrary binary and run some commands. Here’s an example in action: https://github.com/IamTheFij/release-gitter/tree/main/sample_pseudo_bin https://github.com/IamTheFij/release-gitter/tree/main/sample...