5 ms·

Malicious Python libraries found and removed from PyPI

101 points by bobjordan 8y ago

r0f1 8y ago
What was wrong with the library timeit? Isn't this a legitimate one?
- polotics 8y ago
  The real timeit doesn't need to be installed off pypi, it comes with python. Which makes this example the sneakier one, the rest is mostly typo-squatting with diango dajngo etc
  - r0f1 8y ago
    Oh wow.. I didn't know that. I will be more cautious from now on.. Thanks for the info :)
JackC 8y ago
Another pypi issue I've been wondering about recently is binary wheels. We saw an issue a few weeks back where pipenv hashes stopped matching because binary wheels were added for a package years after the last release, and it wasn't immediately obvious how to check if they were legitimate. It would be nice if there was some automated scan to check that binary builds match the source. I think there have been some python package reproducible build efforts at Debian, and there's a standard image for building wheels, so maybe that's somewhat possible?
- ris 8y ago
  Nix manages this. https://nixos.org/ https://nixos.org/
nikonyrh 8y ago
Is there an official policy on what kind of code a package can have? I assumed it is the responsibilty of the end-user to determine if the package has code they are willing to run. I mean the system worked just as intended: the user asked for a piece of code and PyPI delivered it.
- ploggingdev 8y ago
  https://pypi.org/help/#admin-intervention https://pypi.org/help/#admin-intervention
red-tea 8y ago
Things like diango don't really seem like they would target typos as much as malicious copy and paste targets.
- Latteland 8y ago
  The whole world of package maintenance is subject to this, from pip to c# packages, npm, everything. I'm sure we all (developers) have some of these on our systems, because you rely on downloading packages to use that depend on other things. It makes me want to do development in a dummy account, and my 'personal account' with my passwords and ssh credentials is somehow a separate account. I never do that in practice because it would be too much pain. That's why these kinds of package attacks are valuable. So what can we do to address this? This story exactly lines up with my long time worry. I could be careful or lucky and avoid things like this, but the packages I use might not be so careful or lucky.
  - blattimwind 8y ago
    > So what can we do to address this? We use a a web of trust and sign releases using our well-known WoTted keys using GPG. Getting hacked every few years is less painful, though. (Actually Debian does this to some extent by asking upstreams for signed releases using a designated key, so this can be made to work)
    - Latteland 8y ago
      Thats a good start but what would have prevented the owner of "dujangoo" from siging their package? Nothing.
      - toyg 8y ago
        The key would appear untrusted, though. Which, admittedly, is not much of a defense: most developers would just assume it’s a new key and trust it blindly. Still, it would be an additional data point, and possibly another chance to spot the typo (if the UI repeats it).
      - 8y ago
ris 8y ago
Ah, wild west package repositories... Only yesterday the developers of Itsdangerous went and deleted their recently-released 1.0.0 version. That's right. Deleted. Gone. Anyone who made the upgrade and for some insane reason were using pip in their release pipeline suddenly had an un-deployable app. And people complain that they don't want maintainers "getting in the way" between them and developers.
- Terr_ 8y ago
  > itsdangerous Truth in advertising then.
- ryebit 8y ago
  That's one big reason my company deploys everything through a devpi proxy. Not only can I blacklist versions, override with our own patched versions... But can also guarantee if I installed it during testing, it'll be there in production. (Also, updating production isn't affected by pypi / GitHub / etc going down). Devpi has its own design issues, but certainly a reliable and workable solution.
- js2 8y ago
  https://github.com/pallets/itsdangerous/issues/112 https://github.com/pallets/itsdangerous/issues/112 > I’m sorry for the inconvenience caused but I missed that there was a signature change that made it into 1.0. I yanked the release now because this change had some very bad consequences and yanking the release is less risky in comparison. Ugh, why not release 1.0.1 with the change reverted? That would be the right thing to do whether 1.0 were yanked or not.
  - aidos 8y ago
    I sounds like mitsuhiko was between a rock and a hard place and had to do damage control to mitigate the situation. Without looking into the details, reverting wasn’t a straightforward option.
- deleted 8y ago
  [deleted]
Eli_P 8y ago
I've been trying to make some kind of smarter approach to prevent typesquatting in my GUI for package managers called pips[1], by means of fast BK-tree[2] package name comparison with the offline index. There's some different approach I'm still working on to roll into pips, hope I won't be too lazy to make it happen, eventually. [1] https://github.com/ptytb/pips https://github.com/ptytb/pips [2] https://en.wikipedia.org/wiki/BK-tree https://en.wikipedia.org/wiki/BK-tree
Alex3917 8y ago
Maybe PyPi should require uploaders to verify their real identities, or at least flag packages where users haven't done so.
- deleted 8y ago
  [deleted]
- mounds 8y ago
  Whoa whoa whoa! Revolutionary idea - signed packages?! /s In all seriousness though, Python's community is so communally fractured, I don't see one standard winning out. When the community revolts against it's own 'benevolent dictator for life' for wanting to better the language specification... It's most likely headed for doom. Python and PyPi are in a precarious state. They are getting a lot of attention from a lot of people, who may or may not have it's best interest in mind. Previous Python users were using the language out of love for the language and it's facilities... This issue is the tip of a much deeper iceberg lingering in the Python ecosystem: how is Python being intelligibly managed, if at all?
  - Alex3917 8y ago
    > When the community revolts against it's own 'benevolent dictator for life' for wanting to better the language specification... It's most likely headed for doom. That's Python's biggest strength. If you want a bug fixed or feature added to javascript, you need to pay $20M to join ECMA or whatever. If you want the same in Python you just post on the mailing list and argue your case. You might not win, but you're at least guaranteed your day in court. That's the most valuable 'feature' of the Python ecosystem, even if it gets a little heated at times.
- rurban 8y ago
  Wouldn't help because they didn't even block the username or used email address of the uploaded.
closed 8y ago
Were the research / data leakage packages from the student who did their thesis on the security of pypi etc..? https://incolumitas.com/2016/06/08/typosquatting-package-managers/ https://incolumitas.com/2016/06/08/typosquatting-package-man...

101 points by bobjordan 8y ago

r0f1 8y ago

What was wrong with the library timeit? Isn't this a legitimate one?

polotics 8y ago
The real timeit doesn't need to be installed off pypi, it comes with python. Which makes this example the sneakier one, the rest is mostly typo-squatting with diango dajngo etc
- r0f1 8y ago
  Oh wow.. I didn't know that. I will be more cautious from now on.. Thanks for the info :)

JackC 8y ago

Another pypi issue I've been wondering about recently is binary wheels. We saw an issue a few weeks back where pipenv hashes stopped matching because binary wheels were added for a package years after the last release, and it wasn't immediately obvious how to check if they were legitimate. It would be nice if there was some automated scan to check that binary builds match the source. I think there have been some python package reproducible build efforts at Debian, and there's a standard image for building wheels, so maybe that's somewhat possible?

ris 8y ago
Nix manages this. https://nixos.org/ https://nixos.org/

nikonyrh 8y ago

Is there an official policy on what kind of code a package can have? I assumed it is the responsibilty of the end-user to determine if the package has code they are willing to run. I mean the system worked just as intended: the user asked for a piece of code and PyPI delivered it.

ploggingdev 8y ago
https://pypi.org/help/#admin-intervention https://pypi.org/help/#admin-intervention

red-tea 8y ago

Things like diango don't really seem like they would target typos as much as malicious copy and paste targets.

Latteland 8y ago
The whole world of package maintenance is subject to this, from pip to c# packages, npm, everything. I'm sure we all (developers) have some of these on our systems, because you rely on downloading packages to use that depend on other things. It makes me want to do development in a dummy account, and my 'personal account' with my passwords and ssh credentials is somehow a separate account. I never do that in practice because it would be too much pain. That's why these kinds of package attacks are valuable. So what can we do to address this? This story exactly lines up with my long time worry. I could be careful or lucky and avoid things like this, but the packages I use might not be so careful or lucky.
- blattimwind 8y ago
  > So what can we do to address this? We use a a web of trust and sign releases using our well-known WoTted keys using GPG. Getting hacked every few years is less painful, though. (Actually Debian does this to some extent by asking upstreams for signed releases using a designated key, so this can be made to work)
  - Latteland 8y ago
    Thats a good start but what would have prevented the owner of "dujangoo" from siging their package? Nothing.
    - toyg 8y ago
      The key would appear untrusted, though. Which, admittedly, is not much of a defense: most developers would just assume it’s a new key and trust it blindly. Still, it would be an additional data point, and possibly another chance to spot the typo (if the UI repeats it).
    - 8y ago

ris 8y ago

Ah, wild west package repositories... Only yesterday the developers of Itsdangerous went and deleted their recently-released 1.0.0 version. That's right. Deleted. Gone. Anyone who made the upgrade and for some insane reason were using pip in their release pipeline suddenly had an un-deployable app. And people complain that they don't want maintainers "getting in the way" between them and developers.

Terr_ 8y ago
> itsdangerous Truth in advertising then.
ryebit 8y ago
That's one big reason my company deploys everything through a devpi proxy. Not only can I blacklist versions, override with our own patched versions... But can also guarantee if I installed it during testing, it'll be there in production. (Also, updating production isn't affected by pypi / GitHub / etc going down). Devpi has its own design issues, but certainly a reliable and workable solution.
js2 8y ago
https://github.com/pallets/itsdangerous/issues/112 https://github.com/pallets/itsdangerous/issues/112 > I’m sorry for the inconvenience caused but I missed that there was a signature change that made it into 1.0. I yanked the release now because this change had some very bad consequences and yanking the release is less risky in comparison. Ugh, why not release 1.0.1 with the change reverted? That would be the right thing to do whether 1.0 were yanked or not.
- aidos 8y ago
  I sounds like mitsuhiko was between a rock and a hard place and had to do damage control to mitigate the situation. Without looking into the details, reverting wasn’t a straightforward option.
deleted 8y ago
[deleted]

Eli_P 8y ago

I've been trying to make some kind of smarter approach to prevent typesquatting in my GUI for package managers called pips[1], by means of fast BK-tree[2] package name comparison with the offline index. There's some different approach I'm still working on to roll into pips, hope I won't be too lazy to make it happen, eventually. [1] https://github.com/ptytb/pips https://github.com/ptytb/pips [2] https://en.wikipedia.org/wiki/BK-tree https://en.wikipedia.org/wiki/BK-tree

Alex3917 8y ago

Maybe PyPi should require uploaders to verify their real identities, or at least flag packages where users haven't done so.

deleted 8y ago
[deleted]
mounds 8y ago
Whoa whoa whoa! Revolutionary idea - signed packages?! /s In all seriousness though, Python's community is so communally fractured, I don't see one standard winning out. When the community revolts against it's own 'benevolent dictator for life' for wanting to better the language specification... It's most likely headed for doom. Python and PyPi are in a precarious state. They are getting a lot of attention from a lot of people, who may or may not have it's best interest in mind. Previous Python users were using the language out of love for the language and it's facilities... This issue is the tip of a much deeper iceberg lingering in the Python ecosystem: how is Python being intelligibly managed, if at all?
- Alex3917 8y ago
  > When the community revolts against it's own 'benevolent dictator for life' for wanting to better the language specification... It's most likely headed for doom. That's Python's biggest strength. If you want a bug fixed or feature added to javascript, you need to pay $20M to join ECMA or whatever. If you want the same in Python you just post on the mailing list and argue your case. You might not win, but you're at least guaranteed your day in court. That's the most valuable 'feature' of the Python ecosystem, even if it gets a little heated at times.
rurban 8y ago
Wouldn't help because they didn't even block the username or used email address of the uploaded.

closed 8y ago

Were the research / data leakage packages from the student who did their thesis on the security of pypi etc..? https://incolumitas.com/2016/06/08/typosquatting-package-managers/ https://incolumitas.com/2016/06/08/typosquatting-package-man...