3 ms·

Maybe you're right, but I see one possible downside that is quite important. I have encountered the case "the package has an important bugfix but is not yet pu

Maybe you're right, but I see one possible downside that is quite important. I have encountered the case "the package has an important bugfix but is not yet published on PyPI" way more than once or twice. With the intermediate maintainers, that's going to get worse. I believe namespaces and signatures are the way to go. With a special privileged namespace for the curated widely known packages (e.g. SciPy or Django) - a little like it's on the Docker Hub, where curated mainstream images are just "debian" or "python" but anyone can upload e.g. "jdoe/debian" if they need some customization.

nyrikki 9y ago
PyPI should also run a build to audit behavior which would be fairly easy to implement. A submitted package would just fail if it access the network or privileged files during compile unless unique needs are called out in an spec file. I do wish that `--user` was the default for pip. It is also a pity that trivial Debian bugs like this block adoption of non sudo pip installs weren't ignored. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839155 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839155 Although Debian/Ubuntu default to --user on pip people resort to sudo because the current standard user bin directory isn't in the default path due to a regression. I may start a project to create a apparmor/selinux wrapper for pip to audit and restrict access to sensitive resources. I actually have a fairly heavyweight version in place on my build pipeline to detect new dependencies. I add the files/network resources that a build accesses outside of the testing stage to the build artifacts. But it wouldn't be cross platform enough for Windows/Mac.
- drdaeman 9y ago
  1. This would require PyPI to provide computing resources to build the packages (for all OSes, if the package contains native code). And then, malicious packages would just detect the build environment and avoid exhibiting the unwanted behavior. I think I read that it's not unusual for malware to detect being ran in a VM and doesn't do anything suspicious to complicate detection and analysis. 2. I don't think `pip install --user` adds any significant security. A little bit - sure, but not much. A trivial injection into ~/.*shrc or ~/.profile (I don't think anyone would notice the file was changed until it's too late) would result in full system compromise on the next login and sudo invocation. Same goes if you have ~/bin or ~/.local/bin (or anything user-writeable) in $PATH. And even with non-root access, malicious software can do a lot of undesirable things (e.g. send spam or steal user data). --- I believe, signature-based trust (with mandatory code signing) is the way to go. On the first `pip install` ever ask "The package not-expect (1.2.3) is signed by The Spanish Inquisition (key: ...) and was audited by The Python Developers (key: ...). Have you verified the keys and do you trust a) this vendor, b) this auditor or c) both?", then it gets recorded in ~/.pip (for this machine) and ./requirements.txt or ./setup.{py,cfg} (for distribution) so future installations don't ask anything. To get non-interactive mode (for CI or something), on must either pass something like --insecure-skip-signature-checks (so they mean it) or pre-supply all the trusted keys. (Not ideal, of course - just a quick idea. Surely, it has a lot of rough edges to polish.)
  - nyrikki 9y ago
    On point 1) yes so they would have to compile, that is not a huge barrier and it would also result in improved quality of modules just as CI/CD does. On 2) even if you ignore the much larger attack surface due to running ALL installs as the root user, consider the one-shot opportunities by disabling the protections of capabilities(7) # capsh --print -- -c 'pip list > /dev/null' Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read+ep And if --user was the norm, it would be trivial to write an apparmor/selinux policy to protect files like ~/.*shrc or ~/.profile It is just the basic principle of least privilege. Heck if irssi can bother with an apparmor profile, the maintainers of pip or the package should be able to.
- rplnt 9y ago
  With this system, one could pull a Volkswagen.

nyrikki 9y ago

PyPI should also run a build to audit behavior which would be fairly easy to implement. A submitted package would just fail if it access the network or privileged files during compile unless unique needs are called out in an spec file. I do wish that `--user` was the default for pip. It is also a pity that trivial Debian bugs like this block adoption of non sudo pip installs weren't ignored. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839155 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839155 Although Debian/Ubuntu default to --user on pip people resort to sudo because the current standard user bin directory isn't in the default path due to a regression. I may start a project to create a apparmor/selinux wrapper for pip to audit and restrict access to sensitive resources. I actually have a fairly heavyweight version in place on my build pipeline to detect new dependencies. I add the files/network resources that a build accesses outside of the testing stage to the build artifacts. But it wouldn't be cross platform enough for Windows/Mac.

drdaeman 9y ago
1. This would require PyPI to provide computing resources to build the packages (for all OSes, if the package contains native code). And then, malicious packages would just detect the build environment and avoid exhibiting the unwanted behavior. I think I read that it's not unusual for malware to detect being ran in a VM and doesn't do anything suspicious to complicate detection and analysis. 2. I don't think `pip install --user` adds any significant security. A little bit - sure, but not much. A trivial injection into ~/.*shrc or ~/.profile (I don't think anyone would notice the file was changed until it's too late) would result in full system compromise on the next login and sudo invocation. Same goes if you have ~/bin or ~/.local/bin (or anything user-writeable) in $PATH. And even with non-root access, malicious software can do a lot of undesirable things (e.g. send spam or steal user data). --- I believe, signature-based trust (with mandatory code signing) is the way to go. On the first `pip install` ever ask "The package not-expect (1.2.3) is signed by The Spanish Inquisition (key: ...) and was audited by The Python Developers (key: ...). Have you verified the keys and do you trust a) this vendor, b) this auditor or c) both?", then it gets recorded in ~/.pip (for this machine) and ./requirements.txt or ./setup.{py,cfg} (for distribution) so future installations don't ask anything. To get non-interactive mode (for CI or something), on must either pass something like --insecure-skip-signature-checks (so they mean it) or pre-supply all the trusted keys. (Not ideal, of course - just a quick idea. Surely, it has a lot of rough edges to polish.)
- nyrikki 9y ago
  On point 1) yes so they would have to compile, that is not a huge barrier and it would also result in improved quality of modules just as CI/CD does. On 2) even if you ignore the much larger attack surface due to running ALL installs as the root user, consider the one-shot opportunities by disabling the protections of capabilities(7) # capsh --print -- -c 'pip list > /dev/null' Current: = cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read+ep And if --user was the norm, it would be trivial to write an apparmor/selinux policy to protect files like ~/.*shrc or ~/.profile It is just the basic principle of least privilege. Heck if irssi can bother with an apparmor profile, the maintainers of pip or the package should be able to.
rplnt 9y ago
With this system, one could pull a Volkswagen.