18 ms·

The Incredible Growth of Python

333 points by kjhughes 9y ago

cutler 9y ago
Searching Indeed.co.uk by title I've noticed Django gradually gain a 10% lead over Rails in the last 12 months. Python:Ruby is currently 2.9 for London and 3.2 nationwide. I think Django's rise is due to the increased adoption of Python generally and also the meteoric rise of Node.js in the same space as Rails.
calsy 9y ago
PHP or GTFO.
solomatov 9y ago
My hypothesis is that all this caused by data science, machine learning, and deep learning. R is quite cryptic and hard to learn. Almost all new libraries for DS are written in Python. Also, in favor of this hypothesis is that rich countries are ahead of the curve in DS adoption. It's much harder to find a DS job in a developing or not so wealthy developed country than in US.
santaclaus 9y ago
Python is such a good get-shit-done language. I've been diving into the Qt bindings at work recently, and I've gotta say, these are hella underrated.
- mherrmann 9y ago
  I'm using PyQt as well for my file manager [1]. You may be interested in a wiki I maintain for things I had to learn "the hard way" [2]. 1: https://fman.io https://fman.io 2: https://github.com/mherrmann/pyqt-resources https://github.com/mherrmann/pyqt-resources
dopeboy 9y ago
Three years ago, I was shutting down my startup which was a webapp built in PHP using my own framework. I decided I was going to consult and I also decided it was time to learn a professional, modern framework. At the time, RoR was hot and I was leaning towards it. Laravel felt good too because it was new and it seemed to do PHP web "right". The last option was Python and Django. What I liked about the latter is that data science seemed (still is?) the lingua franca of data science and I figured I could learn web programming and build up my Python skills that I could one day apply towards data science. I ultimately went with Python and haven't looked back. Django (particularly DRF - shout out to Tom Christie if he's reading) has been an utter joy to work with. Even though we're in the era of fat front-ends, it's nice to have a bullet-proof, well supported backend. Python donation page: https://www.python.org/psf/donations/ https://www.python.org/psf/donations/ Django donation page: https://www.djangoproject.com/fundraising/ https://www.djangoproject.com/fundraising/ Django rest framework donation page: https://fund.django-rest-framework.org/topics/funding/ https://fund.django-rest-framework.org/topics/funding/
kagamine 9y ago
I do my hobby development on Linux, Python 2.7 and 3.x are always installed but 2.7 is the default. What is the future for Linux distros dropping 2.x and moving only to 3.x? My understanding is that Linux requires 2.7 as the default version, but it means development is a minor PITA, I have to set up v3 and use pip3 (iirc) instead of just pip and type python3 in the shell instead of just python. It appears minor, but it can get a bit annoying for a Python newbie.
- maweki 9y ago
  That python points to 2.7 is currently a PEP requirement. Nothing to do with confidence on anybodys part.
- nyrikki 9y ago
  Linux doesn't require 2.7, and will be switching soon. But you can remove most of the pain from a future migration by just adding the following to all your 2.7 code and changing your code. Note that these future modules were all available in 2.6 which was released in 2008. But just like the ruby 2.0 upgrade people procrastinate. But you only have 2 years before python 2.7 is no longer maintained. from __future__ import (absolute_import, division, print_function) __metaclass__ = type
d--b 9y ago
I know I shouldn't feel that way, but as a developer who loathes Python, this is a painful read. I start to feel that if I can't fight the trend then I should join it. But then every time I try Python, I feel such unease because of the low performance and the dynamic typing. It's just something I don't understand... this is really depressing to me.
- eatplayrove 9y ago
  I am not an evangelist of any form, but running python code with pypy isn't that bad in terms of performance (especially given the ease of writing the actual code), and you can use type hints if you like since 3.5 : https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html
- peferron 9y ago
  You're not alone. The only place I like Python is as a replacement for shell scripting.
- dom96 9y ago
  You might find Nim[1] to be a worthy alternative. I primarily see it as a compiled Python with good performance and static typing. 1 - https://nim-lang.org https://nim-lang.org
  - d--b 9y ago
    Oh, usually I am wary of trying new languages but this looks very cool indeed.
- Boothroid 9y ago
  I hate Python also - supposed to be this magic low barrier to entry language, but as soon as I want to do anything of substance I start encountering just as much complexity as any other language, only the documentation is worse, and the libraries I want to use are in one way or another a nightmare. I don't buy the hype, use it reluctantly when I have to.
nstart 9y ago
Predicting it now. A large part of the "why" is going to be because of the jump in data science. Python is an incredibly kind language to new comers in the sense that it allows you to just get started. A lot of people are making bigger jumps into data science (purely anecdotal) in the sense that they are programming more around it. Scraping data, cleaning data, etc is also really easy with Python. Devops stuff is also a great one to pair with Python. Reason I'm predicting this: I've kept an eye on Python+Flask /Django jobs for over a year now, and while I see "Python" show up more frequently now, "Flask" and "Django" are still about as frequent as they were in the past. Looking forward to the "why" post. Glad that they mentioned that they'll be publishing that soon.
- vanderZwan 9y ago
  I work at Karolinska as the sole developer on a data browsing platform, inside a neurobiology research group. The entire department is either using or learning Python. Of course, the work environment itself is a severe form of selection bias, but there definitely has been an uptake in Python among scientists. (Which leaves me kinda sad, because I like Julia more - I'm sure it will keep growing though)
  - nstart 9y ago
    Julia is definitely a step up in data science programming. It's purpose built for data science. I guess that's also the reason why Python has grown more than Julia. The likelihood of someone having worked with Python in an unrelated field means that when starting out in another field, if the Python option exists, they are probably going to take it. Apart from that, if one is just getting started out in data science and has never really handled a programming language before, Julia is very nice really. And their documentation is pretty damn awesome. And once you get past the performance recommendations I think it gets pretty fast (?). I'm out of my depth beyond a certain point since I've only ever experimented with it.
    - roystonvassey 9y ago
      The problem is that data science, often, isn't just data science. It might involve doing a bunch of activities that are strictly not data science - scraping data off the web (Python has requests), visualization for some understanding (Python has dozens of nifty libraries), store data in a small db (Python works nicely with sqllite, among others), convert all of this into an app (Python has Flask, again among others). R's data frame concept was fantastic for those transitioning from 'spreadsheet' analysis but with Pandas, Python has improved and enhanced this concept even further. This is just one example but I think it is this versatility that makes it more attractive than R or Julia.
tankerdude 9y ago
It's multiple reasons why Python is gaining traction again. One of them is definitely due to all the data science that uses it. The second though is that schools are also making the switch from Java to Python as their first language. So the folks learning are searching for a bunch of packages and I suspect a number of the easier questions.
zimablue 9y ago
I moved from my last job (mostly C#/vba, front office finance)to a new role (python, data guy at hedge fund). Doing data stuff and simple web services is absurdly more straight forward in python, the main things I miss (weaknesses of python to C# and I'm guessing java): nice parallel options (I know several options exist but haven't found any of them as easy to get into as C# async/await, GIL is the problem i guess) the django database layer doesn't do smart diffs in the same way as .net db projects (in .net it's smart enough to actually look at your code schema vs the database and work out how to roll forward/back, in django it's just using a combination of your code schema and a table describing what has and hasn't been rolled out yet, making everything a bit scarier and tougher if anything goes wrong. I dunno if SQLAlchemy does this. edit: also I once read a HN comment that a problem at the heart of python is that block-syntax forces you into having only trivial inline lambdas, after writing it for a while I think they might have been right
- mixmastamyk 9y ago
  Trivial lambdas make Python more readable than highly nested languages.
herogreen 9y ago
The French grande école d'ingénieur in CS & applied Maths in which I study made the switch from Ada to Python as the first language teached last year. Before that I saw all the French Higher School Preparatory Classes starting to teach C.S. with Python3 in 2013. So I am happy to see that many of France's higher education establishments made the right move.
jaco8 9y ago
Yes , but the young contenders already exist. If Nim gets more traction , over time it may replace Python.
- analog31 9y ago
  I think about that once in a while, since I've gone all-in on Python, and here's how I've come to grips with it. Even if Python eventually declines (as did Pascal, HyperCard, and Visual Basic), it will still have made me a better programmer, and the continual improvement of languages means that the young contenders have to prove themselves better than Python. So, bring 'em on.
- alcalde 9y ago
  Nim's designer is too enamored with Delphi and has chosen to borrow many of Delphi's bad ideas for nostalgic reasons. This adds more ambiguities, more Perl-like symbol use, more reliance on an intelligent IDE, etc. Python is about the importance of simplicity and readability. I don't see Python devs abandoning Python for Nim any time soon. The ones who would have been tempted by Nim have already been tempted away by Go.
  - dom96 9y ago
    What bad ideas from Delphi did Nim borrow? Presumably you are referring to the way imports are done in Nim, i.e. the fact that all symbols are imported into the current module by default (the equivalent of `from module import *` in Python). Nim could certainly follow Python's lead, but that would limit the language: Nim supports UFCS and operator overloading which requires modules to be imported this way. You can of course feel free to use `from module import nil` in Nim to enforce module name prefixes for each procedure call. But keep in mind that there is no risk of ambiguities, the compiler statically checks everything and refuses to compile your code if there is an ambiguity (at which point you will need to resolve it by prefixing with the module name). I would be interested to hear more about other things that you think are a bad idea. Could you elaborate a bit?
- goatlover 9y ago
  Why would Nim be a better choice over Julia for data science stuff?
frgtpsswrdlame 9y ago
It's just too easy. I never programmed a thing until after I had graduated college and yet when I did start running into times where I wanted a little script or program, it was always less than 100 lines in python to set it up.
jzl 9y ago
TL;DR: Slightly interesting but narrowly focused article with clickbait-ish headline. It charts the growth of Stack Overflow python question views in the last five years. While this is an interesting statistic there is no additional analysis that it correlates to python's usage growth. And no hypothesis is given for the growth in question views, although they tease a future post about that. It would have been far more interesting if they correlated their stats to lines of code in github or something along those lines. To me, if anything I'm curious why python was so low percentage-wise in question views only five years ago. Anecdotally it doesn't seem to me that python usage in "industry" has grown that dramatically in the last five years, but there could be other factors involved such as education. Or my biased gut instinct could be completely off. (Note: been a python user for 20+ years.)
- BeetleB 9y ago
  As others have pointed out, more people coming out of school know it. When I started my job, the company (fairly large one) was mostly Perl. Perl was the big language of a past generation, so when those engineers were hired, Perl was king. I was told by at least two people that having Python on my resume is not a boost for engineering companies (not counting software), as most managers had never heard of it. Not too many years later, a lot of new work in the company is being done in Python. Why? Because that's what the newer employees know and prefer. I think in the software world, Python was big a long time ago (even in the days I joined). But non-software companies were stuck with Tcl and Perl for much longer. Now that they're shifting, you will see some of that growth.
- d0mine 9y ago
  They address this point in an earlier post on Flash (Flash is Dead). See near "Conclusion: do questions really represent the health of a technology?" https://stackoverflow.blog/2017/08/01/flash-dead-technologies-might-next/ https://stackoverflow.blog/2017/08/01/flash-dead-technologie...
- Alex3917 9y ago
  > To me, if anything I'm curious why python was so low percentage-wise in question views only five years ago. The majority of the top CS colleges switched to teaching using Python within the last five years. This provides a big reason for companies to use Python as well, since it drives down the cost of hiring.
andrewstuart 9y ago
Python 2 advocates were quite firm in telling us that Python 3 and its incompatibility with Python 2 had killed Python and people would abandon Python altogether in droves for other languages. This post is suggesting the opposite, that Python is more healthy than ever and growing incredibly fast.
- arikrak 9y ago
  Maybe the confusion over python 2 vs. 3 is why people have so many questions about it.. </snark, mostly>
- blub 9y ago
  Those people were advocating for stability, not necessarily for P2. The 2->3 was a pretty big screw up, and it definitely had an impact on the language and ecosystem. Python seems to have overcome that (keyword: seems), thanks to a strong offering in data science, but 2->3 is still a headache. I never really used Python for application development, but rather for utility/scripts and I am still on 2, because I don't need more. 2 is available out of the box on all OSes and at work we also use 2 for building tools. I have the feeling that Python is still losing people to Golang due to its weak performance and Golang's reputation in that area. So instead of being arrogant about 2->3, I'd say Python devs should be thankful to data science for saving their butts. :)
  - fleurdelotus 9y ago
    >>Python is still losing people to Golang Any source (other than Rob Pike) ?
    - blub 9y ago
      Anecdata I'm afraid, hence "feeling". This can only be solved through an actual market study. The SO post is not the study we need, because it uses a proxy variable (SO tags).
  - goatlover 9y ago
    I doubt Python is losing people to Golang in scientific computing.
agumonkey 9y ago
I'm surprised by the decline of C#. I thought it was better off than java these days. Also PHP is gliding down. They improved the language a lot, but I can't say I will miss the PHP world much. About python, isn't it related to numpy mostly ?
- brianwawok 9y ago
  C# is not that special off Windows. Most shops dont run Windows. Ergo Java wins over c#.
- deleted 9y ago
  [deleted]
- kirse 9y ago
  I'm surprised by the decline of C# The graphs are for # of views to questions on SO involving that language. What it likely means for C# (and PHP/Java, etc) is that the user-base is maturing and stabilizing, and not as many questions need to be asked (or found). For C#, it could also have something to do with the fact that the MSDN .NET docs have vastly improved over the past 3-5 years.
twobyfour 9y ago
The chart there really emphasizes to what degree Java, C++, and to a lesser degree Javascript are used as teaching languages. There's a clear cyclical pattern in their question frequency that presumably corresponds to academic semesters.
- stupidcar 9y ago
  Hmm. Interesting that JS's cycle is the inverse of Java's. People learning Java during term-time, then JS during their holidays?
  - sotojuan 9y ago
    This was the case in college. We had C++ classes but people learned Python and JS during breaks for internships because the are very little C++ jobs for college students.
  - pen2l 9y ago
    In their holidays, people make websites ("The next facebook!"), at term-time they make sodoku solving applications in Java for programming class.
    - falmak 9y ago
      Is "the next Facebook" really what today's students dream of building? It just seems a bit... dull. When I was a student we all wanted to create "the next <popular AAA game>".
      - goatlover 9y ago
        The next VR Snapchat?
amelius 9y ago
What packages do people use mostly on Python? Those packages could explain why Python is so popular.
- goatlover 9y ago
  Numpy undergirds the scientific computing stack in Python.
- FTA 9y ago
  All the scientists I know on Python (N > 30) use matplotlib and (numpy or scipy or pandas). Some niche libraries for certain data formats are also necessary. These libraries are compatible in 2 and 3, and are easy to install with pip or conda, so everyone I've shown them to has made the jump quickly from Matlab, IDL, and others.
- brianwawok 9y ago
  Web people Django or similar and requests to make http... then a database driver. that gets you pretty far.
deleted 9y ago
[deleted]
singularity2001 9y ago
interestingly the curve starts to take off even before the introduction of tensorflow
singularity2001 9y ago
one funny way to interprete the data would be: "the language gets more and more complicated and batteries which used to be included now have to be Inquired on Stack overflow."
granitosaurus 9y ago
You have to give props to Python Foundation and just python's community in general - very well managed on both sides. PEP proposals and the general pythons philosophy is still going strong while new major changes are being introduced in every version without breaking stuff. As someone who has been contributing to python q&a websites and such for quite a while now, I feel like I'm starting to have trouble of keeping up - a lot of new newcommers but even more people willing to help! I'd be really be surprised to see Python on a decline any time soon.
mylh 9y ago
I'm a web developer and I love python because it provides me with the fastest and shortest way from an idea to its implementation. In most cases prototype works forever in production. Rich standard library and clear version support lifecycle are also very important. I developed with many other languages including java, с++ and Haskell and python is my choice among all.
- StavrosK 9y ago
  I agree (Django is life), but I find myself wanting the new optional typing and type checker to mature so I can add them to my set of linters. I'm tired of 1) not knowing whether I made a mistake and passed something of the wrong type and 2) not being able to see at a glance what types each function is supposed to accept and return.
  - SimbaOnSteroids 9y ago
    How large, in your experience, does an app need to get before it outgrows django?
    - StavrosK 9y ago
      I'm not sure, I haven't managed to write an app that large yet. I'm also not sure how you can outgrow Django, since you can always split off parts of your codebase, or not use the parts of Django you don't need. I've recently written a flask application too, and I found myself having to reinvent the wheel at every step, and I cursed a lot. Django's ecosystem, both in libraries and in sheer community documentation, is hard to replicate.
oxplot 9y ago
Does the usage of a site like SO (where you ask for help) necessarily correlate with growth of a language? Or more specifically, does the difference between views of two different languages on SO correlate with real world difference of growth between those two languages? As a personal example, I've searched for answers related to golang far less than I have for python because I tend to find my answers by simply reading golang docs but in case of python, docs do not help me as much so I turn to google and consequently to SO.
- rb666 9y ago
  Not the usage in absolute terms (as it may just indicate the language requires a lot of SO research to do anything useful), but the growth of such usage would seem to correlate with growth of the language.
goodoldboys 9y ago
As someone who works with Python every day, this isn't surprising. I forget where I heard/read it, but someone made the claim that Python is the "second best language for everything" (paraphrasing) and I couldn't agree more. Need to create a web app that interfaces with a data pipeline? Python/Django/Flask is perfect. There are countless other examples too.
- pbreit 9y ago
  I agree that Python works well for just about everything but wish there was a framework between Django & Flask.
  - denisw 9y ago
    The most popular framework that fits this description might be Pyramid [1]. [1]: https://trypyramid.com/ https://trypyramid.com/
  - joaodlf 9y ago
    Agreed! I lean much more towards Flask (I'd rather have too much simplicity than too much complexity), but something in the middle would be fantastic. A simple application structure with loose enough rules and a strong standard library is what I would like. Flask can be too little, Django can be too big.
  - dagw 9y ago
    Have you looked at: http://www.django-rest-framework.org/ http://www.django-rest-framework.org/ and the Lightweight Django book: http://shop.oreilly.com/product/0636920032502.do http://shop.oreilly.com/product/0636920032502.do
  - Walkman 9y ago
    Pyramid is exactly that.
  - abledon 9y ago
    Use Django and strip out the middleware and admin/authentication addons. see https://github.com/lightweightdjango https://github.com/lightweightdjango
torbjorn 9y ago
python has a cool name. as time goes on new kids learn to program. kids choose the programming language with the cool name.
- kaffee 9y ago
  Isn't Java a cool name? How would you operationalize the "coolness" of a name? Maybe you could use pytorch for this? It seems like a supervised learning problem.
  - nol13 9y ago
    no name could make java cool
    - henryscala 9y ago
      :-)
cs702 9y ago
This is probably related to explosion of interest in AI and deep learning (DL), because Python has become THE standard language for AI/DL research. Most AI frameworks support Python out of the box, including TensorFlow, PyTorch, Theano, and CAFFE/CAFFE2, which together currently have the most developer mindshare. Every single TensorFlow, Pytorch, Theano, and CAFFE/CAFFE2 tutorial or example is written in Python. With very few exceptions (e.g., the occasional Lua code coming out of Yann Le Cun's group at FB), new AI papers come with code written in Python. If you want to see Python's utter dominance in this field, just search for "deep learning" or "neural net" on github.
- solomatov 9y ago
  And rich countries are ahead of the curve in data science, AI and DL.
- nl 9y ago
  This is true. Even the Le Cunn group has switched to PyTorch I think. It's also because no one can work out how to use matplotlib properly, so you need to look at about 1000 StackOverflow questions per line of code.
  - photojosh 9y ago
    To be fair, this was also true when I was using Matlab back before switching to Python (2006ish!), except you had to dig through documentation instead of just having everything immediately available on SO. :)
    - princeb 9y ago
      the matlab documentation, imo, is the best technical documentation ever written for anything. if your background was more math-heavy then the docs provided easy to understand translation between recognizable mathematical idioms into code (if matlab wasn't already very math-idiomatic). and back in 2010 or so, python's numerical optimization libraries left a lot to be desired, and totally hopeless compared to matlab. (edit, i don't remember it being nearly as good as octave, which wasn't particularly good either back then).
hackbinary 9y ago
We've standardised on Python and we're migrating all of our systems to Python / Django / Postgres from PHP / Drupal / WordPress / MySQL. We started out with one senior Dev on the Python stack, and simply found that the productivity on Python/Django just too compelling to stay on PHP. The (senior) Dev on Python had a couple of highly successful projects which resulted with him getting some junior Dev support. Then the other Devs started to notice and saw/felt that they were missing out on better and more advanced tooling. They then started to push to move their main environments to Python/Django. This has further been pushed by the business stakeholders who have seen the Python/Django projects move much faster than PHP projects. So for us, that is now 6 Devs on Python/Django plus some consultancy support. So we've move from 2 python Devs to 6 Python devs within the last year.
- eighthnate 9y ago
  > We started out with one senior Dev on the Python stack, and simply found that the productivity on Python/Django just too compelling to stay on PHP. There is nothing inherently "more productive" on one stack over the next. It's just technologies which are fundamentally similar. A competent senior dev on the python stack would be just as productive as a competent senior dev on a php stack and vice versa. It just seems like you found a good python developer who knows what he is doing.
  - tcdent 9y ago
    Absolutely false. While PHP has gotten better recently, the language is not nearly as powerful or complete as almost all of the popular alternatives. Up until very recently, something like the Django ORM was impossible to express in PHP. Take an honest look at the available frameworks (especially in comparison to something archaic like Drupal) and you'll find there's no competition.
    - acomjean 9y ago
      Python certainly has a lot more software written for it. I use it for scientific computing. For building websites, php has plenty of good options too. I do really like the Silex framework for website construction. I backend process with java or python when needed to do any substantial computations. The "standardization" on common libraries from symfony and zend, have made everything a lot better.
antod 9y ago
As someone who has been using Python since 2000 roughly, another minor factor (besides the obvious data science factors) might also be the decline in Ruby. Ruby (and Rails) growth in the 2005-2010 era, seemed to noticeably put the brakes on Python's previous growth up till that point. Back then Ruby had all the boosterism/hype and Python was the supposedly "boring/legacy" option. Even now, Python's growth doesn't seem driven by hype the same way Ruby's was. It always seemed to be a more pragmatic choice rather than a fashionable one. Although somewhat ironically, I don't use much Python any more now that I work at a mostly Ruby/Rails company haha.
- cutler 9y ago
  I don't think hype is the main factor. Ruby's decline is due to its narrow focus on web development and Rails in particular. Whilst there certainly are gems covering other areas of computing Ruby never seemed to diversify the way Python has. With Node.js eating Ruby's lunch in the web development sphere Ruby is left looking like the single-commodity Venezuelan economy. Let me add that I think Rails is a fantastic piece of software. Maybe TruffleRuby and Substrate VM will revive Ruby's fortunes. Matz's promises for Ruby 3x3 are too little too late, I fear.
  - goatlover 9y ago
    I really hate that Ruby became synonymous with Rails. It's such a nice language on it's own.
  - ksec 9y ago
    It is not just Matz but the community or core team as a whole. Look at every Python 3.x release, and then look at the release in Ruby, some of the bigger improvement were more like catch up. And since then there is little changes. It is also rather unfortunate no company wants to invest in Ruby.
  - ngsayjoe 9y ago
    I think besides Rails, Ruby is also heavily used in DevOps, e.g. Chef, Puppet, etc. I personally prefer using Ruby for testing & Selenium as well.
s17n 9y ago
Python is pretty much becoming the de facto language of academia (outside of CS). I wonder how much this is driving overall Python usage?
- ohsonice 9y ago
  Grad student at a math department here- I see python used in tons or research applications with fellow students studying math, engineering, stats, etc. But in my experience Matlab still reigns king in math academia, especially in classroom environments.
valuearb 9y ago
Isn't this just proving Python is the language that people need the most help with?
qu4ku 9y ago
python is beautiful. period.

333 points by kjhughes 9y ago

Searching Indeed.co.uk by title I've noticed Django gradually gain a 10% lead over Rails in the last 12 months. Python:Ruby is currently 2.9 for London and 3.2 nationwide. I think Django's rise is due to the increased adoption of Python generally and also the meteoric rise of Node.js in the same space as Rails.

calsy 9y ago

PHP or GTFO.

solomatov 9y ago

My hypothesis is that all this caused by data science, machine learning, and deep learning. R is quite cryptic and hard to learn. Almost all new libraries for DS are written in Python. Also, in favor of this hypothesis is that rich countries are ahead of the curve in DS adoption. It's much harder to find a DS job in a developing or not so wealthy developed country than in US.

santaclaus 9y ago

Python is such a good get-shit-done language. I've been diving into the Qt bindings at work recently, and I've gotta say, these are hella underrated.

mherrmann 9y ago
I'm using PyQt as well for my file manager [1]. You may be interested in a wiki I maintain for things I had to learn "the hard way" [2]. 1: https://fman.io https://fman.io 2: https://github.com/mherrmann/pyqt-resources https://github.com/mherrmann/pyqt-resources

dopeboy 9y ago

Three years ago, I was shutting down my startup which was a webapp built in PHP using my own framework. I decided I was going to consult and I also decided it was time to learn a professional, modern framework. At the time, RoR was hot and I was leaning towards it. Laravel felt good too because it was new and it seemed to do PHP web "right". The last option was Python and Django. What I liked about the latter is that data science seemed (still is?) the lingua franca of data science and I figured I could learn web programming and build up my Python skills that I could one day apply towards data science. I ultimately went with Python and haven't looked back. Django (particularly DRF - shout out to Tom Christie if he's reading) has been an utter joy to work with. Even though we're in the era of fat front-ends, it's nice to have a bullet-proof, well supported backend. Python donation page: https://www.python.org/psf/donations/ https://www.python.org/psf/donations/ Django donation page: https://www.djangoproject.com/fundraising/ https://www.djangoproject.com/fundraising/ Django rest framework donation page: https://fund.django-rest-framework.org/topics/funding/ https://fund.django-rest-framework.org/topics/funding/

kagamine 9y ago

I do my hobby development on Linux, Python 2.7 and 3.x are always installed but 2.7 is the default. What is the future for Linux distros dropping 2.x and moving only to 3.x? My understanding is that Linux requires 2.7 as the default version, but it means development is a minor PITA, I have to set up v3 and use pip3 (iirc) instead of just pip and type python3 in the shell instead of just python. It appears minor, but it can get a bit annoying for a Python newbie.

maweki 9y ago
That python points to 2.7 is currently a PEP requirement. Nothing to do with confidence on anybodys part.
nyrikki 9y ago
Linux doesn't require 2.7, and will be switching soon. But you can remove most of the pain from a future migration by just adding the following to all your 2.7 code and changing your code. Note that these future modules were all available in 2.6 which was released in 2008. But just like the ruby 2.0 upgrade people procrastinate. But you only have 2 years before python 2.7 is no longer maintained. from __future__ import (absolute_import, division, print_function) __metaclass__ = type

d--b 9y ago

I know I shouldn't feel that way, but as a developer who loathes Python, this is a painful read. I start to feel that if I can't fight the trend then I should join it. But then every time I try Python, I feel such unease because of the low performance and the dynamic typing. It's just something I don't understand... this is really depressing to me.

eatplayrove 9y ago
I am not an evangelist of any form, but running python code with pypy isn't that bad in terms of performance (especially given the ease of writing the actual code), and you can use type hints if you like since 3.5 : https://docs.python.org/3/library/typing.html https://docs.python.org/3/library/typing.html
peferron 9y ago
You're not alone. The only place I like Python is as a replacement for shell scripting.
dom96 9y ago
You might find Nim[1] to be a worthy alternative. I primarily see it as a compiled Python with good performance and static typing. 1 - https://nim-lang.org https://nim-lang.org
- d--b 9y ago
  Oh, usually I am wary of trying new languages but this looks very cool indeed.
Boothroid 9y ago
I hate Python also - supposed to be this magic low barrier to entry language, but as soon as I want to do anything of substance I start encountering just as much complexity as any other language, only the documentation is worse, and the libraries I want to use are in one way or another a nightmare. I don't buy the hype, use it reluctantly when I have to.

nstart 9y ago

Predicting it now. A large part of the "why" is going to be because of the jump in data science. Python is an incredibly kind language to new comers in the sense that it allows you to just get started. A lot of people are making bigger jumps into data science (purely anecdotal) in the sense that they are programming more around it. Scraping data, cleaning data, etc is also really easy with Python. Devops stuff is also a great one to pair with Python. Reason I'm predicting this: I've kept an eye on Python+Flask /Django jobs for over a year now, and while I see "Python" show up more frequently now, "Flask" and "Django" are still about as frequent as they were in the past. Looking forward to the "why" post. Glad that they mentioned that they'll be publishing that soon.

vanderZwan 9y ago
I work at Karolinska as the sole developer on a data browsing platform, inside a neurobiology research group. The entire department is either using or learning Python. Of course, the work environment itself is a severe form of selection bias, but there definitely has been an uptake in Python among scientists. (Which leaves me kinda sad, because I like Julia more - I'm sure it will keep growing though)
- nstart 9y ago
  Julia is definitely a step up in data science programming. It's purpose built for data science. I guess that's also the reason why Python has grown more than Julia. The likelihood of someone having worked with Python in an unrelated field means that when starting out in another field, if the Python option exists, they are probably going to take it. Apart from that, if one is just getting started out in data science and has never really handled a programming language before, Julia is very nice really. And their documentation is pretty damn awesome. And once you get past the performance recommendations I think it gets pretty fast (?). I'm out of my depth beyond a certain point since I've only ever experimented with it.
  - roystonvassey 9y ago
    The problem is that data science, often, isn't just data science. It might involve doing a bunch of activities that are strictly not data science - scraping data off the web (Python has requests), visualization for some understanding (Python has dozens of nifty libraries), store data in a small db (Python works nicely with sqllite, among others), convert all of this into an app (Python has Flask, again among others). R's data frame concept was fantastic for those transitioning from 'spreadsheet' analysis but with Pandas, Python has improved and enhanced this concept even further. This is just one example but I think it is this versatility that makes it more attractive than R or Julia.

tankerdude 9y ago

It's multiple reasons why Python is gaining traction again. One of them is definitely due to all the data science that uses it. The second though is that schools are also making the switch from Java to Python as their first language. So the folks learning are searching for a bunch of packages and I suspect a number of the easier questions.

zimablue 9y ago

I moved from my last job (mostly C#/vba, front office finance)to a new role (python, data guy at hedge fund). Doing data stuff and simple web services is absurdly more straight forward in python, the main things I miss (weaknesses of python to C# and I'm guessing java): nice parallel options (I know several options exist but haven't found any of them as easy to get into as C# async/await, GIL is the problem i guess) the django database layer doesn't do smart diffs in the same way as .net db projects (in .net it's smart enough to actually look at your code schema vs the database and work out how to roll forward/back, in django it's just using a combination of your code schema and a table describing what has and hasn't been rolled out yet, making everything a bit scarier and tougher if anything goes wrong. I dunno if SQLAlchemy does this. edit: also I once read a HN comment that a problem at the heart of python is that block-syntax forces you into having only trivial inline lambdas, after writing it for a while I think they might have been right

mixmastamyk 9y ago
Trivial lambdas make Python more readable than highly nested languages.

herogreen 9y ago

The French grande école d'ingénieur in CS & applied Maths in which I study made the switch from Ada to Python as the first language teached last year. Before that I saw all the French Higher School Preparatory Classes starting to teach C.S. with Python3 in 2013. So I am happy to see that many of France's higher education establishments made the right move.

jaco8 9y ago

Yes , but the young contenders already exist. If Nim gets more traction , over time it may replace Python.

analog31 9y ago
I think about that once in a while, since I've gone all-in on Python, and here's how I've come to grips with it. Even if Python eventually declines (as did Pascal, HyperCard, and Visual Basic), it will still have made me a better programmer, and the continual improvement of languages means that the young contenders have to prove themselves better than Python. So, bring 'em on.
alcalde 9y ago
Nim's designer is too enamored with Delphi and has chosen to borrow many of Delphi's bad ideas for nostalgic reasons. This adds more ambiguities, more Perl-like symbol use, more reliance on an intelligent IDE, etc. Python is about the importance of simplicity and readability. I don't see Python devs abandoning Python for Nim any time soon. The ones who would have been tempted by Nim have already been tempted away by Go.
- dom96 9y ago
  What bad ideas from Delphi did Nim borrow? Presumably you are referring to the way imports are done in Nim, i.e. the fact that all symbols are imported into the current module by default (the equivalent of `from module import *` in Python). Nim could certainly follow Python's lead, but that would limit the language: Nim supports UFCS and operator overloading which requires modules to be imported this way. You can of course feel free to use `from module import nil` in Nim to enforce module name prefixes for each procedure call. But keep in mind that there is no risk of ambiguities, the compiler statically checks everything and refuses to compile your code if there is an ambiguity (at which point you will need to resolve it by prefixing with the module name). I would be interested to hear more about other things that you think are a bad idea. Could you elaborate a bit?
goatlover 9y ago
Why would Nim be a better choice over Julia for data science stuff?

frgtpsswrdlame 9y ago

It's just too easy. I never programmed a thing until after I had graduated college and yet when I did start running into times where I wanted a little script or program, it was always less than 100 lines in python to set it up.

jzl 9y ago

TL;DR: Slightly interesting but narrowly focused article with clickbait-ish headline. It charts the growth of Stack Overflow python question views in the last five years. While this is an interesting statistic there is no additional analysis that it correlates to python's usage growth. And no hypothesis is given for the growth in question views, although they tease a future post about that. It would have been far more interesting if they correlated their stats to lines of code in github or something along those lines. To me, if anything I'm curious why python was so low percentage-wise in question views only five years ago. Anecdotally it doesn't seem to me that python usage in "industry" has grown that dramatically in the last five years, but there could be other factors involved such as education. Or my biased gut instinct could be completely off. (Note: been a python user for 20+ years.)

BeetleB 9y ago
As others have pointed out, more people coming out of school know it. When I started my job, the company (fairly large one) was mostly Perl. Perl was the big language of a past generation, so when those engineers were hired, Perl was king. I was told by at least two people that having Python on my resume is not a boost for engineering companies (not counting software), as most managers had never heard of it. Not too many years later, a lot of new work in the company is being done in Python. Why? Because that's what the newer employees know and prefer. I think in the software world, Python was big a long time ago (even in the days I joined). But non-software companies were stuck with Tcl and Perl for much longer. Now that they're shifting, you will see some of that growth.
d0mine 9y ago
They address this point in an earlier post on Flash (Flash is Dead). See near "Conclusion: do questions really represent the health of a technology?" https://stackoverflow.blog/2017/08/01/flash-dead-technologies-might-next/ https://stackoverflow.blog/2017/08/01/flash-dead-technologie...
Alex3917 9y ago
> To me, if anything I'm curious why python was so low percentage-wise in question views only five years ago. The majority of the top CS colleges switched to teaching using Python within the last five years. This provides a big reason for companies to use Python as well, since it drives down the cost of hiring.

andrewstuart 9y ago

Python 2 advocates were quite firm in telling us that Python 3 and its incompatibility with Python 2 had killed Python and people would abandon Python altogether in droves for other languages. This post is suggesting the opposite, that Python is more healthy than ever and growing incredibly fast.

arikrak 9y ago
Maybe the confusion over python 2 vs. 3 is why people have so many questions about it.. </snark, mostly>
blub 9y ago
Those people were advocating for stability, not necessarily for P2. The 2->3 was a pretty big screw up, and it definitely had an impact on the language and ecosystem. Python seems to have overcome that (keyword: seems), thanks to a strong offering in data science, but 2->3 is still a headache. I never really used Python for application development, but rather for utility/scripts and I am still on 2, because I don't need more. 2 is available out of the box on all OSes and at work we also use 2 for building tools. I have the feeling that Python is still losing people to Golang due to its weak performance and Golang's reputation in that area. So instead of being arrogant about 2->3, I'd say Python devs should be thankful to data science for saving their butts. :)
- fleurdelotus 9y ago
  >>Python is still losing people to Golang Any source (other than Rob Pike) ?
  - blub 9y ago
    Anecdata I'm afraid, hence "feeling". This can only be solved through an actual market study. The SO post is not the study we need, because it uses a proxy variable (SO tags).
- goatlover 9y ago
  I doubt Python is losing people to Golang in scientific computing.

agumonkey 9y ago

I'm surprised by the decline of C#. I thought it was better off than java these days. Also PHP is gliding down. They improved the language a lot, but I can't say I will miss the PHP world much. About python, isn't it related to numpy mostly ?

brianwawok 9y ago
C# is not that special off Windows. Most shops dont run Windows. Ergo Java wins over c#.
deleted 9y ago
[deleted]
kirse 9y ago
I'm surprised by the decline of C# The graphs are for # of views to questions on SO involving that language. What it likely means for C# (and PHP/Java, etc) is that the user-base is maturing and stabilizing, and not as many questions need to be asked (or found). For C#, it could also have something to do with the fact that the MSDN .NET docs have vastly improved over the past 3-5 years.

twobyfour 9y ago

The chart there really emphasizes to what degree Java, C++, and to a lesser degree Javascript are used as teaching languages. There's a clear cyclical pattern in their question frequency that presumably corresponds to academic semesters.

stupidcar 9y ago
Hmm. Interesting that JS's cycle is the inverse of Java's. People learning Java during term-time, then JS during their holidays?
- sotojuan 9y ago
  This was the case in college. We had C++ classes but people learned Python and JS during breaks for internships because the are very little C++ jobs for college students.
- pen2l 9y ago
  In their holidays, people make websites ("The next facebook!"), at term-time they make sodoku solving applications in Java for programming class.
  - falmak 9y ago
    Is "the next Facebook" really what today's students dream of building? It just seems a bit... dull. When I was a student we all wanted to create "the next <popular AAA game>".
    - goatlover 9y ago
      The next VR Snapchat?

amelius 9y ago

What packages do people use mostly on Python? Those packages could explain why Python is so popular.

goatlover 9y ago
Numpy undergirds the scientific computing stack in Python.
FTA 9y ago
All the scientists I know on Python (N > 30) use matplotlib and (numpy or scipy or pandas). Some niche libraries for certain data formats are also necessary. These libraries are compatible in 2 and 3, and are easy to install with pip or conda, so everyone I've shown them to has made the jump quickly from Matlab, IDL, and others.
brianwawok 9y ago
Web people Django or similar and requests to make http... then a database driver. that gets you pretty far.

deleted 9y ago

[deleted]

singularity2001 9y ago

interestingly the curve starts to take off even before the introduction of tensorflow

singularity2001 9y ago

one funny way to interprete the data would be: "the language gets more and more complicated and batteries which used to be included now have to be Inquired on Stack overflow."

granitosaurus 9y ago

You have to give props to Python Foundation and just python's community in general - very well managed on both sides. PEP proposals and the general pythons philosophy is still going strong while new major changes are being introduced in every version without breaking stuff. As someone who has been contributing to python q&a websites and such for quite a while now, I feel like I'm starting to have trouble of keeping up - a lot of new newcommers but even more people willing to help! I'd be really be surprised to see Python on a decline any time soon.

mylh 9y ago

I'm a web developer and I love python because it provides me with the fastest and shortest way from an idea to its implementation. In most cases prototype works forever in production. Rich standard library and clear version support lifecycle are also very important. I developed with many other languages including java, с++ and Haskell and python is my choice among all.

StavrosK 9y ago
I agree (Django is life), but I find myself wanting the new optional typing and type checker to mature so I can add them to my set of linters. I'm tired of 1) not knowing whether I made a mistake and passed something of the wrong type and 2) not being able to see at a glance what types each function is supposed to accept and return.
- SimbaOnSteroids 9y ago
  How large, in your experience, does an app need to get before it outgrows django?
  - StavrosK 9y ago
    I'm not sure, I haven't managed to write an app that large yet. I'm also not sure how you can outgrow Django, since you can always split off parts of your codebase, or not use the parts of Django you don't need. I've recently written a flask application too, and I found myself having to reinvent the wheel at every step, and I cursed a lot. Django's ecosystem, both in libraries and in sheer community documentation, is hard to replicate.

oxplot 9y ago

Does the usage of a site like SO (where you ask for help) necessarily correlate with growth of a language? Or more specifically, does the difference between views of two different languages on SO correlate with real world difference of growth between those two languages? As a personal example, I've searched for answers related to golang far less than I have for python because I tend to find my answers by simply reading golang docs but in case of python, docs do not help me as much so I turn to google and consequently to SO.

rb666 9y ago
Not the usage in absolute terms (as it may just indicate the language requires a lot of SO research to do anything useful), but the growth of such usage would seem to correlate with growth of the language.

goodoldboys 9y ago

As someone who works with Python every day, this isn't surprising. I forget where I heard/read it, but someone made the claim that Python is the "second best language for everything" (paraphrasing) and I couldn't agree more. Need to create a web app that interfaces with a data pipeline? Python/Django/Flask is perfect. There are countless other examples too.

pbreit 9y ago
I agree that Python works well for just about everything but wish there was a framework between Django & Flask.
- denisw 9y ago
  The most popular framework that fits this description might be Pyramid [1]. [1]: https://trypyramid.com/ https://trypyramid.com/
- joaodlf 9y ago
  Agreed! I lean much more towards Flask (I'd rather have too much simplicity than too much complexity), but something in the middle would be fantastic. A simple application structure with loose enough rules and a strong standard library is what I would like. Flask can be too little, Django can be too big.
- dagw 9y ago
  Have you looked at: http://www.django-rest-framework.org/ http://www.django-rest-framework.org/ and the Lightweight Django book: http://shop.oreilly.com/product/0636920032502.do http://shop.oreilly.com/product/0636920032502.do
- Walkman 9y ago
  Pyramid is exactly that.
- abledon 9y ago
  Use Django and strip out the middleware and admin/authentication addons. see https://github.com/lightweightdjango https://github.com/lightweightdjango

torbjorn 9y ago

python has a cool name. as time goes on new kids learn to program. kids choose the programming language with the cool name.

kaffee 9y ago
Isn't Java a cool name? How would you operationalize the "coolness" of a name? Maybe you could use pytorch for this? It seems like a supervised learning problem.
- nol13 9y ago
  no name could make java cool
  - henryscala 9y ago
    :-)

cs702 9y ago

This is probably related to explosion of interest in AI and deep learning (DL), because Python has become THE standard language for AI/DL research. Most AI frameworks support Python out of the box, including TensorFlow, PyTorch, Theano, and CAFFE/CAFFE2, which together currently have the most developer mindshare. Every single TensorFlow, Pytorch, Theano, and CAFFE/CAFFE2 tutorial or example is written in Python. With very few exceptions (e.g., the occasional Lua code coming out of Yann Le Cun's group at FB), new AI papers come with code written in Python. If you want to see Python's utter dominance in this field, just search for "deep learning" or "neural net" on github.

solomatov 9y ago
And rich countries are ahead of the curve in data science, AI and DL.
nl 9y ago
This is true. Even the Le Cunn group has switched to PyTorch I think. It's also because no one can work out how to use matplotlib properly, so you need to look at about 1000 StackOverflow questions per line of code.
- photojosh 9y ago
  To be fair, this was also true when I was using Matlab back before switching to Python (2006ish!), except you had to dig through documentation instead of just having everything immediately available on SO. :)
  - princeb 9y ago
    the matlab documentation, imo, is the best technical documentation ever written for anything. if your background was more math-heavy then the docs provided easy to understand translation between recognizable mathematical idioms into code (if matlab wasn't already very math-idiomatic). and back in 2010 or so, python's numerical optimization libraries left a lot to be desired, and totally hopeless compared to matlab. (edit, i don't remember it being nearly as good as octave, which wasn't particularly good either back then).

hackbinary 9y ago

We've standardised on Python and we're migrating all of our systems to Python / Django / Postgres from PHP / Drupal / WordPress / MySQL. We started out with one senior Dev on the Python stack, and simply found that the productivity on Python/Django just too compelling to stay on PHP. The (senior) Dev on Python had a couple of highly successful projects which resulted with him getting some junior Dev support. Then the other Devs started to notice and saw/felt that they were missing out on better and more advanced tooling. They then started to push to move their main environments to Python/Django. This has further been pushed by the business stakeholders who have seen the Python/Django projects move much faster than PHP projects. So for us, that is now 6 Devs on Python/Django plus some consultancy support. So we've move from 2 python Devs to 6 Python devs within the last year.

eighthnate 9y ago
> We started out with one senior Dev on the Python stack, and simply found that the productivity on Python/Django just too compelling to stay on PHP. There is nothing inherently "more productive" on one stack over the next. It's just technologies which are fundamentally similar. A competent senior dev on the python stack would be just as productive as a competent senior dev on a php stack and vice versa. It just seems like you found a good python developer who knows what he is doing.
- tcdent 9y ago
  Absolutely false. While PHP has gotten better recently, the language is not nearly as powerful or complete as almost all of the popular alternatives. Up until very recently, something like the Django ORM was impossible to express in PHP. Take an honest look at the available frameworks (especially in comparison to something archaic like Drupal) and you'll find there's no competition.
  - acomjean 9y ago
    Python certainly has a lot more software written for it. I use it for scientific computing. For building websites, php has plenty of good options too. I do really like the Silex framework for website construction. I backend process with java or python when needed to do any substantial computations. The "standardization" on common libraries from symfony and zend, have made everything a lot better.

antod 9y ago

As someone who has been using Python since 2000 roughly, another minor factor (besides the obvious data science factors) might also be the decline in Ruby. Ruby (and Rails) growth in the 2005-2010 era, seemed to noticeably put the brakes on Python's previous growth up till that point. Back then Ruby had all the boosterism/hype and Python was the supposedly "boring/legacy" option. Even now, Python's growth doesn't seem driven by hype the same way Ruby's was. It always seemed to be a more pragmatic choice rather than a fashionable one. Although somewhat ironically, I don't use much Python any more now that I work at a mostly Ruby/Rails company haha.

cutler 9y ago
I don't think hype is the main factor. Ruby's decline is due to its narrow focus on web development and Rails in particular. Whilst there certainly are gems covering other areas of computing Ruby never seemed to diversify the way Python has. With Node.js eating Ruby's lunch in the web development sphere Ruby is left looking like the single-commodity Venezuelan economy. Let me add that I think Rails is a fantastic piece of software. Maybe TruffleRuby and Substrate VM will revive Ruby's fortunes. Matz's promises for Ruby 3x3 are too little too late, I fear.
- goatlover 9y ago
  I really hate that Ruby became synonymous with Rails. It's such a nice language on it's own.
- ksec 9y ago
  It is not just Matz but the community or core team as a whole. Look at every Python 3.x release, and then look at the release in Ruby, some of the bigger improvement were more like catch up. And since then there is little changes. It is also rather unfortunate no company wants to invest in Ruby.
- ngsayjoe 9y ago
  I think besides Rails, Ruby is also heavily used in DevOps, e.g. Chef, Puppet, etc. I personally prefer using Ruby for testing & Selenium as well.

s17n 9y ago

Python is pretty much becoming the de facto language of academia (outside of CS). I wonder how much this is driving overall Python usage?

ohsonice 9y ago
Grad student at a math department here- I see python used in tons or research applications with fellow students studying math, engineering, stats, etc. But in my experience Matlab still reigns king in math academia, especially in classroom environments.

valuearb 9y ago

Isn't this just proving Python is the language that people need the most help with?

qu4ku 9y ago

python is beautiful. period.