4 ms·

Their conclusion, "security issues are common in PyPI packages", doesn't really follow from the results. Their methods will classify any use of a function that

Their conclusion, "security issues are common in PyPI packages", doesn't really follow from the results. Their methods will classify any use of a function that is not cryptographically secure (MD5, random), even if it is not used in a cryptographic setting. Similarly any use of a function that is not safe to use on untrusted input (pickle, yaml.load, subprocess, eval) will be flagged, even if the usage is completely safe.

nonameiguess 5y ago
Yes this, but also you shouldn't do an analysis like this on all of PyPi. Anyone can upload to it. It's full of abandoned experiments, name-squatting, and college students uploading hello world libraries just to learn how to do it. Analyzing those is pointless because nobody is using them and nobody is going to use them. Also listing the subprocess module as a standout because of code injection seems silly. That's the entire point of it existing. You may as well say a shell is insecure because it allows injecting shell commands. Obviously, don't put strings from untrusted sources in there, but Python is largely intended for system administration automation, the first thing to turn to when the shell isn't enough if you don't like Perl. It would be pretty useless if you couldn't actually use it to orchestrate arbitrary shell commands.
- emacsen 5y ago
  I was with you until you said that Python is largely intended for system administration automation. Python is used in a wide variety of application environments, especially the web. For example, early versions of Youtube were nothing more than a Python application. Many websites run Django, or other web frameworks. Python is heavily used in scientific computing environments, in data retrieval and visualization, as well as fintech (ie hedge funds). Its use in system administration/configuration management is real and significant, but represents a smallish proportion of its use. Certainly some static analysis would be beneficial here, especially in areas like science or financial technology, but much of your original point remains completely true- much of concern raised speaks more to system design around passing strings as objects of reference, and security design, rather than language issues.
  - meowface 5y ago
    Instagram still runs on Python as well, I believe.
  - takeda 5y ago
    > For example, early versions of Youtube were nothing more than a Python application I heard about it, but I'm wondering, is that how Python got popular at Google and then become mainstream?
    - tooltower 5y ago
      Python applications were widely used before either Google or YouTube existed
      - takeda 5y ago
        Yes, but Python is actually older than Java, yet the boom started much later for Python.
        rmnclmnt 5y ago
        A good reason is because Java was explicitly engineered and targeted towards enterprise use-cases from the beginning by Sun, whereas Python was created as a mean to teach programming with a simpler language.
        emacsen 5y ago
        There are two reasons I see for this based on my experience. First, as the other commenter mentioned, Java was targeted at large enterprise. I remember hearing about Java Beans, and Java Enterprise this and that. It fit with the model of development many were seeing in the late 90s as well- UML diagrams written by some middle manager. There were even tools like Rational Rose that would transform your UML into Java outlines. Python was an academic language for the most part and so it took time to catch up. This is the same time period, where Linux was dismissed as a toy or hacker's tool. Serious people worked on Sun Microsystems computers, or Windows NT. Sun and NT were "Real tools for business" and Linux was for people like me- kids in their dorm rooms. It took time for the industry to catch up, which it did largely out of necessity. A company like Google could run thousands of computers for a fraction of the cost of doing so with NT or Sun, and that gave them a competitive advantage. With Python, the advantage was different- it wasn't the cost as much as it was the simple convenience and network effect of libraries and a robust community. Sun could throw money towards documentation, education and marketing. Python was a grass roots movement that only later had a foundation and corporate involvement.
- deleted 5y ago
  [deleted]
- pabs3 5y ago
  It is possible to use subprocess safely, you just have to be aware of the sharp edges of it: shell=True, sanitising commands, option injection.

nonameiguess 5y ago

Yes this, but also you shouldn't do an analysis like this on all of PyPi. Anyone can upload to it. It's full of abandoned experiments, name-squatting, and college students uploading hello world libraries just to learn how to do it. Analyzing those is pointless because nobody is using them and nobody is going to use them. Also listing the subprocess module as a standout because of code injection seems silly. That's the entire point of it existing. You may as well say a shell is insecure because it allows injecting shell commands. Obviously, don't put strings from untrusted sources in there, but Python is largely intended for system administration automation, the first thing to turn to when the shell isn't enough if you don't like Perl. It would be pretty useless if you couldn't actually use it to orchestrate arbitrary shell commands.

emacsen 5y ago
I was with you until you said that Python is largely intended for system administration automation. Python is used in a wide variety of application environments, especially the web. For example, early versions of Youtube were nothing more than a Python application. Many websites run Django, or other web frameworks. Python is heavily used in scientific computing environments, in data retrieval and visualization, as well as fintech (ie hedge funds). Its use in system administration/configuration management is real and significant, but represents a smallish proportion of its use. Certainly some static analysis would be beneficial here, especially in areas like science or financial technology, but much of your original point remains completely true- much of concern raised speaks more to system design around passing strings as objects of reference, and security design, rather than language issues.
- meowface 5y ago
  Instagram still runs on Python as well, I believe.
- takeda 5y ago
  > For example, early versions of Youtube were nothing more than a Python application I heard about it, but I'm wondering, is that how Python got popular at Google and then become mainstream?
  - tooltower 5y ago
    Python applications were widely used before either Google or YouTube existed
    - takeda 5y ago
      Yes, but Python is actually older than Java, yet the boom started much later for Python.
      - rmnclmnt 5y ago
        A good reason is because Java was explicitly engineered and targeted towards enterprise use-cases from the beginning by Sun, whereas Python was created as a mean to teach programming with a simpler language.
      - emacsen 5y ago
        There are two reasons I see for this based on my experience. First, as the other commenter mentioned, Java was targeted at large enterprise. I remember hearing about Java Beans, and Java Enterprise this and that. It fit with the model of development many were seeing in the late 90s as well- UML diagrams written by some middle manager. There were even tools like Rational Rose that would transform your UML into Java outlines. Python was an academic language for the most part and so it took time to catch up. This is the same time period, where Linux was dismissed as a toy or hacker's tool. Serious people worked on Sun Microsystems computers, or Windows NT. Sun and NT were "Real tools for business" and Linux was for people like me- kids in their dorm rooms. It took time for the industry to catch up, which it did largely out of necessity. A company like Google could run thousands of computers for a fraction of the cost of doing so with NT or Sun, and that gave them a competitive advantage. With Python, the advantage was different- it wasn't the cost as much as it was the simple convenience and network effect of libraries and a robust community. Sun could throw money towards documentation, education and marketing. Python was a grass roots movement that only later had a foundation and corporate involvement.
deleted 5y ago
[deleted]
pabs3 5y ago
It is possible to use subprocess safely, you just have to be aware of the sharp edges of it: shell=True, sanitising commands, option injection.