11 ms·
Devs unknowingly use “malicious” modules snuck into official Python repository
- swiley 9y agoOne nice thing about languages like C is that a lot of programmers just avoid dependencies because dealing with them kind of sucks. That's one solution to this problem.
- LaneRendell 9y agoHaha, I was just about to come say that.
- alanh 9y agoyou're not wrong. once i wrote a whole blog post about how to properly include Boost into C++ code using a particular IDE.
- seanp2k2 9y agoI put Boost up there with PyQt in terms of my annoyance with it and how hard is had been to get it working correctly for the things which have required it. The XQuartz stuff with macOS is also pretty annoying. Bash an OpenSSL versions would be another.
- Waterluvian 9y agoI love Python for this. Big standard library. I hate JavaScript for this... So many weird legacy issues that generally get resolved with libraries. Though ES6 went a long long way.
- seanp2k2 9y agoES6 does not fix the left-pad issue; there is a propensity to use a package to get one small function vs using a library of common helper functions (which is somewhat what e.g. ES6 helps with...less need for underscore or similar). You pretty much don't need jquery now, but so many things depend on it for convenience / backward compatibility / because it's so insanely battle-tested that you know that it will work or at least have a StackOverflow explaining why a particular thing does a thing in a weird way.
- Ajedi32 9y ago> ES6 does not fix the left-pad issue Actually, for that particular problem... https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/padStart https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
- Can_Not 9y ago> there is a propensity to use a package to get one small function vs using a library of common helper functions Lodash functions are available in both forms.
- freshhawk 9y agoIt is good for security research, since that introduces so many security holes of such wonderful diversity that there is a much larger ecosystem to study.
- sametmax 9y agoYeah this way you reinvent a less unit tested, battle field tested and documented wheel. It's like saying the good thing with not having a cellphone is that you avoid a lot of fight with your girlfriend cause you can't talk as much.
- dom0 9y ago> It's like saying the good thing with not having a cellphone is that you avoid a lot of fight with your girlfriend cause you can't talk as much. I can't quite see what's wrong with that...
- tokenizerrr 9y agoDo you have a cellphone? If so, throw it into the trash right now.
- swiley 9y agoI would if I had a girlfriend and she was ok with it. I feel like it's hard to date without one which really sucks.
- GrumpyNl 9y agoThe node.js community sees that very different and thats one aspect i dont understand, the blinded trust in dependencies. A midsize project will need quickly a 1000+ dependencies and we have to trust them all.
- rajangdavis 9y agoHow do you check if you are affected?
- potus50 9y agoI was wondering the same thing.. and why isn't there a list of package names included in this article?
- jp_sc 9y agoThe list is in the advisory. Check if you have installed any of these: – acqusition (uploaded 2017-06-03 01:58:01, impersonates acquisition) – apidev-coop (uploaded 2017-06-03 05:16:08, impersonates apidev-coop_cms) – bzip (uploaded 2017-06-04 07:08:05, impersonates bz2file) – crypt (uploaded 2017-06-03 08:03:14, impersonates crypto) – django-server (uploaded 2017-06-02 08:22:23, impersonates django-server-guardian-api) – pwd (uploaded 2017-06-02 13:12:33, impersonates pwdhash) – setup-tools (uploaded 2017-06-02 08:54:44, impersonates setuptools) – telnet (uploaded 2017-06-02 15:35:05, impersonates telnetsrvlib) – urlib3 (uploaded 2017-06-02 07:09:29, impersonates urllib3) – urllib (uploaded 2017-06-02 07:03:37, impersonates urllib3)
- rajangdavis 9y agoReally close call (I have the correct version of setuptools installed which is what had me worried).
- lamlam 9y agoThere are many tools that can scan your dependencies and tell you if anything is off. Here's one for reference (cannot vouch for it though as I have not used it): https://github.com/jeremylong/DependencyCheck/blob/master/README.md https://github.com/jeremylong/DependencyCheck/blob/master/RE...
- btown 9y agoThe advisory has a regex, but it's not formatted well for copy-paste (non-ASCII quotes!). Here's a version that works: pip list –format=legacy | egrep ‘^(acqusition|apidev-coop|bzip|crypt|django-server|pwd|setup-tools|telnet|urlib3|urllib) ‘
- js2 9y agoFYI: "Malicious software libraries found in PyPI posing as well known libraries" https://news.ycombinator.com/item?id=15256121 https://news.ycombinator.com/item?id=15256121 (2 days ago, 465 points, 245 comments)
- jlgaddis 9y agoOriginal advisory (including fake packages): http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/ http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/
- zizek23 9y agoThe casual culture of pulling in hundred of dependencies, and mushrooming language specific package managers is ridiculously insecure and has to go. There is no way for anyone to know what all this code is doing, there is little way to verify updates and its simply untenable. If some developers like this sort of unsafe practice it should be strictly limited to their machines and in no way make it across in any form as a deployment artifact. There are already secure distribution package managers with the necessary infrastructure, you are not special, use those. Ruby is already paying a price for imposing dependency hell of users and wasting millions of man hours. Many have suffered and do not even bother with Ruby apps anymore. Node and others who think this is a good model will be next. Users should simply boycott such user hostile developers and languages that encourage this kind of insecurity.
- reificator 9y agoNPM is my primary reason for not using NodeJS. I installed a specific package in an empty project and got over 690 dependencies. For running `npm install {package-name}`. What's worse is that I skimmed the tree to check for anything particularly heinous, but there was nothing that stood out as unneeded. With such a tiny stdlib, especially out of the browser environment, there's not really a better alternative than to make it easy to include dependencies for your dependencies. Without NPM, the Node community would be tiny if not already dead. I don't have a better idea either. For my purposes, it just means I use a different language, but that's not really a solution.
- zizek23 9y agoA package manager must at a minimum address code security, dependency hell and deployment. A json parser that wgets urls into a folder is not a package manager. These things must be thought of upfront and cannot be left vague and open to adhoc practices that leak to end users and deployment creating insecurity. For instance a package should be a package, it should not be a simple class or function that in turn pulls in a hundred dependencies. 10 packages like this can pull in 600 packages all with conflicting versions. This is exactly dependency hell that creates insecure practice as no one can verify the chain and sheer number of packages. Language developers cannot just sit back and let this happen. Eventually you will pay the price for this kind of shoddy engineering. This should be the minimum required from any responsible language. And best then if you don't use the system package manager, do not mix up both to create a frankenstein language that multiplies complexity for everyone. The biggest thing is stop being faddish. Recognize upfront that there are always developers and groups jockeying for influence and creating dependencies on their apps and platforms. Discourage them from hoisting their self serving bad ideas on the ecosystem and multiplying complexity for everyone. Ultimately the language and ecosystem have to develop consensus on robust engineering practices or a wildland which will eventually lose users.
- userbinator 9y agoThe IP address it phones home to, 121.42.217.44, is located in China and visiting it with HTTP just displays this interesting message: Hi bro :) Welcome Here! Leave Messages via HTTP Log Please :) On 2017-09-16: Happy to see somebody find it ! :) Just curious about how long it would take for people to find those 'bad' packages As you see, that's just a toy script, no harm, hope you enjoy it ! It looks like someone (security researcher?) just set up a PoC and didn't intend to actually "weaponise" it.
- sametmax 9y agoIt may be https://pytosquatting.org/ https://pytosquatting.org/. In that case it was indeed a harmless and quite useful PoC.
- hannob 9y agoNo. (Co-operator of pytosquatting here) While our code was similar in nature, it was non-obfuscated and we always threw an exception telling the user that he installed something he shouldn't.
- skrause 9y agoBut you still call an URL first, e.g. https://www.pytosquatting.org/pingback/pypi/urllib2/ https://www.pytosquatting.org/pingback/pypi/urllib2/ for "urllib2" before raising the exception. Though it is not an IP in China and only returns the text "Pingback from package urllib2 in repository pypi".
- buckminster 9y agoWhat would you expect the page to say if they were malicious?
- lifeisstillgood 9y agoI think there is a value in upgrading the cheeseshop to use "verified maintainers" - it should be simple to do a first pass of "the person who signed the hash of this module has also verified they own the domain requests.kreitz.org by publishing that public key at that domains root" Even more useful might be domain/keys/kreitz/publickeylisting The key signing party is much harder to arrange but is easier to be confident about Personally I am surprised the Post Office does not do key signing Edit: I thought pip did do cryptographic checks?!
- ubernostrum 9y agopip allows you to specify, along with package name and version, the expected hash of the downloaded package. If you do that, then pip will download the package, calculate the hash, and check against what you specified. In case of mismatch, pip will refuse to install the package. PyPI also supports adding GPG signatures alongside packages, but with no trust/verification process to assert "this key really is the key of the person who should be releasing this package", the signature is literally worthless; anyone who could put up a fake package could also generate a signature for it, and you'd have no way of knowing that the key which signed that package shouldn't be trusted for that package. It is a very very hard problem, and people need to appreciate that.
- falsedan 9y ago> I think there is a value in upgrading the cheeseshop to use "verified maintainers" - it should be simple to do a first pass of "the person who signed the hash of this module has also verified they own the domain requests.kreitz.org by publishing that public key at that domains root" That sounds useless. An attacker could verify that they own evilattacker.com & publish their malicious packages there. I don't think a web of trust is the answer here, because it doesn't really matter if the attack is anonymous or not, and if only trusted people can publish packages, trust will be given more readily to encourage new programmers to contribute. I think a reputation/review system is better, like docker search's star count, or metacpan's ++ rating.
- julianhouston12 9y agoHi My name is Julian Houston, am from upper island cove , Canada.. I want to Use this opportunity to thank God for using this Great prophet to solve my Marriage problem. This Great Prophet of God brought my husband back to me. 3 years ago, i and my husband has been into one quarrel or the other until he finally left me for one lady. I felt my life was over and my kids thought they would never see their father again. I tried to be strong just for the kids but i could not control the pains that was tormenting my heart. My heart was filled with sorrows and pains, because i was really in love with my husband. Every day and night i think of him and always wish he would come back to me. Until one day i melt a good friend of mine that was once in my situation, but her problem was different a little bite, her ex-boyfriend who she had an unwanted pregnancy for refused to take responsibility and dumped her. She told me that mine issue was a minor case and that i shouldn't worry about it at all. So, i asked her what was the solution to my problems and she gave me this Great Prophet of God phone number and his email address. I was doubting if this Great Prophet of God could actually solve my problem. So, I contacted this Great Prophet of God and he told me what to do and i did it. He told me to wait for just four days and that my husband will come crawling on his kneels just for forgiveness. So, I faithfully did what this Great Prophet of God asked me to do and for sure after four days i heard a knock on the door, in a great Surprise i saw him on his knees and i was speechless, when he saw me, all he did was crying and asking me for forgiveness, from that day, all the pains and sorrows in my heart flew away, since then i and my husband and our lovely kids are happy. That why i want to say thank you to God for using Prophet Ikehedu .to solve my marriage problem. This Great Prophet of God made me to understand that there’s no problem on earth that does not Have solution. So, if you are having same problem, any problem that is Similar, i will advise you to a contact This Great Prophet of God straight At prophetikehedu@gmail.com
- addHocker 9y agoThis is a tough one to avoid. Only way i can imagine to detect malicious packages, is by simulating massive deployment or interesting keywords and watch communications.
- stestagg 9y agoThis is a rehash of: https://hackernoon.com/building-a-botnet-on-pypi-be1ad280b8d6 https://hackernoon.com/building-a-botnet-on-pypi-be1ad280b8d... Interestingly, my fake system packages have been downloaded about 480 000 times so far this year
- scaryclam 9y agoOuch, and thanks for being proactive getting those names squatted
- camus2 9y agoWell pip has the same problem as NPM : no namespaces by default. But NPM is worse with all its dependencies of dependencies. Composer (PHP) got both namespace and dependencies right: flat dependencies, it's up to the developer to resolve conflicts, not to the package manager to create insane dependency trees. It leads to more stable packages and make spotting fakes easier.
- Can_Not 9y agoI'd say pip is definitely worse. Things like local dependencies, requirements.txt, and virtual env feel hacked add-ons to make pip more like npm
- deleted 9y ago[deleted]
- nonbel 9y agoThe other day, I was using rmarkdown and noticed that every time it ran it tried to source this: https://mathjax.rstudio.com/2.7.2/MathJax.js?config=TeX-AMS-MML_HTMLorMML https://mathjax.rstudio.com/2.7.2/MathJax.js?config=TeX-AMS-... This was really disconcerting since I did not expect to be downloading anything, let alone running javascript from there just to make a chart.