4 ms·

Be aware that PyPI isn't a great way to get accurate statistics. You can't just ignore a bunch of statistics because they aren't convenient to you; that's not

Be aware that PyPI isn't a great way to get accurate statistics. You can't just ignore a bunch of statistics because they aren't convenient to you; that's not how maths works. Statistical sampling is based on the premise that your sample is randomized from the total population; I would argue that the stats from pypi represent a reasonable random sample of python users around the world. Certainly, you can argue otherwise... if you care to provide some meaningful alternative statistics?

scrollaway 11y ago
GP has argued otherwise. PyPI sampling is not random. It is heavily influenced by automatic downloads and such and completely ignores that most simple 2.x libs / non-3.3-tagged libs will work just fine on python 3. In other words, it is neither random nor representative of python 3 usage, failing the premise you yourself mentioned. There's no easy way to determine Python 3 usage overall. Your best bet is to go around polling companies of all sizes as well as indie devs. What you will find is that: 1. Some amount of people still use Python 2.x because they don't know any better 2. Some amount of people want to use 3.x, but still use Python 2.x because of incompatible dependencies. 3. Some amount of people (companies mostly) still use Python 2.x because a migration to 3.x is currently too costly. 1 -> This is partly caused by FUD. Hating on Python 3 has become a popular past time of people who don't know how to be productive otherwise. It's also caused by the very high amount of tutorials etc around the web that "require" 2.x because, for example, they have an unparenthesized print call (relevant XKCD: 353). Arguably this is one of the big failures of the 3.x push. 2 -> That number is constantly diminishing as more dependencies become available on 3.x. The most recent big win is Scrapy, a massively popular scraping framework, which just finished its 3.x port a couple weeks ago. It's an ongoing process - we are in a much better state than we used to be, the Py3WOS is looking almost entirely green. 3 -> This number is mostly irrelevant as it tends to be in closed ecosystems. So maybe CloudShareBuzzr Inc. is using Python 2.7 internally, but they don't produce anything open source so it doesn't affect the rest of the ecosystem. As new, major features are constantly added to Python 3.x (venv in 3.3, pip in 3.4, async def in 3.5), eventually the cost of switching will be worth it.
- shadowmint 11y ago
  ...it is neither random nor representative of python 3 usage, failing the premise you yourself mentioned. I'm not arguing the py2 vs py3 band wagon; all I'm saying is that if you choose to dispute the validity of statistics you need to do it properly, not just with 'those stats aren't any good because [arbitrary speculative reason 1] [arbitrary speculative reason 2]'. Can you actually justify these assertions? Or are you just speculating because you don't like the results? Your best bet is to go around polling companies of all sizes as well as indie devs. What you will find is that: Oh what? Come on. Now I can't take you seriously at all. You know what they're going to say already, without even asking? You're just speculating. Stats > guesses. Even if the pypi stats aren't completely reflective of the entire ecosystem, I'm going to argue that they give a pretty decent impression of it, regardless of the mild bias that you've listed above but can't quantify.
  - scrollaway 11y ago
    > You know what they're going to say already, without even asking? Yes: - It has been done before - I've been around the Python ecosystem a long time and I know a lot of its users (a random sample, if you will) - I spent several years freelancing as a Python dev which gave me the occasion to do exactly that. While that is not a representative sample on its own, my data matches up with the rest. But all things aside, you really need to check your damn attitude at the door. I neither like nor dislike the results. The facts that matter are that Python 3 has a great ecosystem today and that is what matters to me; not whether my neighbour uses Python 1, 2, 3 or PHP.
    - shadowmint 11y ago
      I'm not interested in the health of the python3 ecosystem at all. My only point is, was and remains, that statistics are more complicated than simply 'those statistics do not match my personal perception of things'; justify your argument with actual data. I'd would be interested to see any alternative stats on the subject you care to link to.
      - 11y ago
jsmeaton 11y ago
As others have mentioned the PyPI statistics aren't random at all. They're a count of downloads based on user agents. The points I mentioned above were discussions I had with the maintainer of PyPI (Donald Stufft) who also recognises that aggregated counts from PyPI are not an effective measure. There are no good stats that give an accurate representation about usage % of each python version. PyPI is the only proxy we have, and that is skewed by a number of factors (two of which I mentioned). The overall trend is probably accurate, but exact percentages certainly are not. My like or dislike for python 2 or 3 has nothing to do with my comment upthread. I was simply stating a number of facts about the data that people can take or leave as they like. It was not "convenient" for me to share those facts. I simply thought people might want to be aware that there's nothing scientific about the aggregated percentages.

scrollaway 11y ago

GP has argued otherwise. PyPI sampling is not random. It is heavily influenced by automatic downloads and such and completely ignores that most simple 2.x libs / non-3.3-tagged libs will work just fine on python 3. In other words, it is neither random nor representative of python 3 usage, failing the premise you yourself mentioned. There's no easy way to determine Python 3 usage overall. Your best bet is to go around polling companies of all sizes as well as indie devs. What you will find is that: 1. Some amount of people still use Python 2.x because they don't know any better 2. Some amount of people want to use 3.x, but still use Python 2.x because of incompatible dependencies. 3. Some amount of people (companies mostly) still use Python 2.x because a migration to 3.x is currently too costly. 1 -> This is partly caused by FUD. Hating on Python 3 has become a popular past time of people who don't know how to be productive otherwise. It's also caused by the very high amount of tutorials etc around the web that "require" 2.x because, for example, they have an unparenthesized print call (relevant XKCD: 353). Arguably this is one of the big failures of the 3.x push. 2 -> That number is constantly diminishing as more dependencies become available on 3.x. The most recent big win is Scrapy, a massively popular scraping framework, which just finished its 3.x port a couple weeks ago. It's an ongoing process - we are in a much better state than we used to be, the Py3WOS is looking almost entirely green. 3 -> This number is mostly irrelevant as it tends to be in closed ecosystems. So maybe CloudShareBuzzr Inc. is using Python 2.7 internally, but they don't produce anything open source so it doesn't affect the rest of the ecosystem. As new, major features are constantly added to Python 3.x (venv in 3.3, pip in 3.4, async def in 3.5), eventually the cost of switching will be worth it.

shadowmint 11y ago
...it is neither random nor representative of python 3 usage, failing the premise you yourself mentioned. I'm not arguing the py2 vs py3 band wagon; all I'm saying is that if you choose to dispute the validity of statistics you need to do it properly, not just with 'those stats aren't any good because [arbitrary speculative reason 1] [arbitrary speculative reason 2]'. Can you actually justify these assertions? Or are you just speculating because you don't like the results? Your best bet is to go around polling companies of all sizes as well as indie devs. What you will find is that: Oh what? Come on. Now I can't take you seriously at all. You know what they're going to say already, without even asking? You're just speculating. Stats > guesses. Even if the pypi stats aren't completely reflective of the entire ecosystem, I'm going to argue that they give a pretty decent impression of it, regardless of the mild bias that you've listed above but can't quantify.
- scrollaway 11y ago
  > You know what they're going to say already, without even asking? Yes: - It has been done before - I've been around the Python ecosystem a long time and I know a lot of its users (a random sample, if you will) - I spent several years freelancing as a Python dev which gave me the occasion to do exactly that. While that is not a representative sample on its own, my data matches up with the rest. But all things aside, you really need to check your damn attitude at the door. I neither like nor dislike the results. The facts that matter are that Python 3 has a great ecosystem today and that is what matters to me; not whether my neighbour uses Python 1, 2, 3 or PHP.
  - shadowmint 11y ago
    I'm not interested in the health of the python3 ecosystem at all. My only point is, was and remains, that statistics are more complicated than simply 'those statistics do not match my personal perception of things'; justify your argument with actual data. I'd would be interested to see any alternative stats on the subject you care to link to.
    - 11y ago

jsmeaton 11y ago

As others have mentioned the PyPI statistics aren't random at all. They're a count of downloads based on user agents. The points I mentioned above were discussions I had with the maintainer of PyPI (Donald Stufft) who also recognises that aggregated counts from PyPI are not an effective measure. There are no good stats that give an accurate representation about usage % of each python version. PyPI is the only proxy we have, and that is skewed by a number of factors (two of which I mentioned). The overall trend is probably accurate, but exact percentages certainly are not. My like or dislike for python 2 or 3 has nothing to do with my comment upthread. I was simply stating a number of facts about the data that people can take or leave as they like. It was not "convenient" for me to share those facts. I simply thought people might want to be aware that there's nothing scientific about the aggregated percentages.