5 ms·

Predicting it now. A large part of the "why" is going to be because of the jump in data science. Python is an incredibly kind language to new comers in the sens

by nstart 9y ago

Predicting it now. A large part of the "why" is going to be because of the jump in data science. Python is an incredibly kind language to new comers in the sense that it allows you to just get started. A lot of people are making bigger jumps into data science (purely anecdotal) in the sense that they are programming more around it. Scraping data, cleaning data, etc is also really easy with Python. Devops stuff is also a great one to pair with Python. Reason I'm predicting this: I've kept an eye on Python+Flask /Django jobs for over a year now, and while I see "Python" show up more frequently now, "Flask" and "Django" are still about as frequent as they were in the past. Looking forward to the "why" post. Glad that they mentioned that they'll be publishing that soon.

vanderZwan 9y ago
I work at Karolinska as the sole developer on a data browsing platform, inside a neurobiology research group. The entire department is either using or learning Python. Of course, the work environment itself is a severe form of selection bias, but there definitely has been an uptake in Python among scientists. (Which leaves me kinda sad, because I like Julia more - I'm sure it will keep growing though)
- nstart 9y ago
  Julia is definitely a step up in data science programming. It's purpose built for data science. I guess that's also the reason why Python has grown more than Julia. The likelihood of someone having worked with Python in an unrelated field means that when starting out in another field, if the Python option exists, they are probably going to take it. Apart from that, if one is just getting started out in data science and has never really handled a programming language before, Julia is very nice really. And their documentation is pretty damn awesome. And once you get past the performance recommendations I think it gets pretty fast (?). I'm out of my depth beyond a certain point since I've only ever experimented with it.
  - roystonvassey 9y ago
    The problem is that data science, often, isn't just data science. It might involve doing a bunch of activities that are strictly not data science - scraping data off the web (Python has requests), visualization for some understanding (Python has dozens of nifty libraries), store data in a small db (Python works nicely with sqllite, among others), convert all of this into an app (Python has Flask, again among others). R's data frame concept was fantastic for those transitioning from 'spreadsheet' analysis but with Pandas, Python has improved and enhanced this concept even further. This is just one example but I think it is this versatility that makes it more attractive than R or Julia.
    - vanderZwan 9y ago
      Yes, in principle Julia should be able to do all of these things too, but it simply doesn't have the decades-old history of libraries to stand on yet.
      - cfusting 9y ago
        Yeah I love Julia but it's hard to argue with Python's data science stack. Sometimes it's almost to easy to get things done.
  - cossatot 9y ago
    Julia is really fast but the DataFrames/DataTables package was kind of slow and feature-incomplete. Because Pandas is so optimized, Julia wasn't winning out over Python on the speed side of things, and was definitely losing on the ergonimics, so for data science it wasn't the best choice (unlike say numerical simulation). I'm sure this will change if it hasn't already, especially with the better parallelization that Julia is capable of.
nyrikki 9y ago
I agree but wanted to add in pandas and seaborn. I actually keep a jupyter qtconsole open to use them for ad-hoc data visualization. Pandas replaced excel for me a while ago and I cringe every time I need to abandon seaborn for a Tableau workbook these days. GUI Visualization tools like Tableau or PowerBI seem to error towards presentation, while the defaults for seaborn help discover and visualize data while still producing results good enough to make C*O's happy.
uvtc 9y ago
> Looking forward to the "why" post. Glad that they mentioned that they'll be publishing that soon. I think the "why" is because, for many users: * they want a small and easy, dynamically-typed, garbage-collected language with few surprises (see `import this`). Perl misses out here. * they want something built with C so there's easy access to native libraries. Languages on general-purpose VMs miss out here. * I don't think they want something that transpiles/compiles down to some other lower-level language (which I suspect leads to difficult debugging). Maybe Nim misses out here. * they're comfortable with imperative-style programming. Maybe lisps and Haskell miss out here. * Python is fast enough for most things where C-speed isn't needed. Very-high-level languages chasing dreams of high performance mostly miss out here, IMO. * they want a language that helps them catch their mistakes. IMO, maybe Lua misses out here (with default nil everywhere). * Users want a general-purpose language. Again, Lua misses here, though, of course, Lua's goal isn't to be general-purpose anyway. * I think many users also want a language that's licensed GPL-compatible. Clojure misses out here. * they want a nice helpful community * Users want a language with familiar C/C++/JS/Java -ish syntax. Python misses out here! {ducks} Python has its own set of problems, but it hits all those marks except for the last one. So, it's gotten quite popular.
- tim333 9y ago
  A lot of people like the Python syntax. It has to be about the most readable of all computer languages.
  - uvtc 9y ago
    > A lot of people like the Python syntax. Of course. > It has to be about the most readable of all computer languages. Quite readable, sure. Though my point is that curlies and semicolons are more familiar to many (and easier to edit/navigate in my editor, for that matter), and probably would be preferred. My comment was about why I think many (not all by any stretch, of course) choose a language like Python.
- nstart 9y ago
  Thats a pretty awesome list!. I'd love for these to be the reasons but somehow I doubt any of us are that thorough. The only reason I used python is that it was the first result that showed up in a Google search "write program to scrape web page" (ah beautiful soup. How I love thee!) The only reason I used flask was Udacitys course taught their web app course with it. I'll admit it, I've never actually analysed the benefits of a language in that way. My only judgement has been "mmm does it feel productive to use this language for what I'm doing". And "how quickly does the documentation take me to a hello world". Which is terribly subjective I admit. I'm suspecting I'm not alone. Or at least I hope I'm not :D. Yes. The embarrassment I feel is real. PS - it's also how I ended up using nginx over Apache. I couldn't really understand Apache in 30 minutes so I used nginx instead. I even remember an interview where I was asked why I used nginx over Apache and I said "um... because it was easier I guess". Definitely not best practices and I'm trying to be more objective nowadays.
rthomas6 9y ago
One "non-programmer" perspective here: Python is the new Perl, except it's fast to read AND write. In Python it's fast to do ...everything. I'm an electrical engineer by trade, and Python gets used a lot. * If I have a giant CSV with data in it: import csv with open(filename) as csvfile: csvreader = csv.DictReader(csvfile) data = list(csvreader) Bam, an array of dictionaries with the data in them, indexed by column name. With listcomps, whatever I need to do will only take a few more lines. And it's this easy for almost any data format. There will be a widely used library for me to `pip install`. * I needed to plot some I/Q data and superimpose some other data as a heat map over it. Complex numbers are built in. Matplotlib, and bam, graph created. It looks as nice as the graphs my coworkers make using Matlab, and I coded it up just as fast, if not faster. * I wanted to play around with ordering a pizza online automatically. Import requests, and off I go. * The code is so easy to read, I've had coworkers update things I wrote when they have never programmed in Python before. I don't have to do any compile steps. I don't have to use a special IDE. I don't have to think in objects and state-passing and all that BS (but I can if I want to!). I don't have to search for some arcane library. I don't have to know a magic build sequence. I don't have to edit config files. I just make cool shit really fast. It runs slower, but for most things I use it for, I don't care if it takes 10 us or 10 seconds. I just want to make it work.
collinmanderson 9y ago
I think Tenserflow is an especially big driver. It's basically making Python the machine learning language of choice.

by nstart 9y ago

vanderZwan 9y ago

I work at Karolinska as the sole developer on a data browsing platform, inside a neurobiology research group. The entire department is either using or learning Python. Of course, the work environment itself is a severe form of selection bias, but there definitely has been an uptake in Python among scientists. (Which leaves me kinda sad, because I like Julia more - I'm sure it will keep growing though)

nstart 9y ago
Julia is definitely a step up in data science programming. It's purpose built for data science. I guess that's also the reason why Python has grown more than Julia. The likelihood of someone having worked with Python in an unrelated field means that when starting out in another field, if the Python option exists, they are probably going to take it. Apart from that, if one is just getting started out in data science and has never really handled a programming language before, Julia is very nice really. And their documentation is pretty damn awesome. And once you get past the performance recommendations I think it gets pretty fast (?). I'm out of my depth beyond a certain point since I've only ever experimented with it.
- roystonvassey 9y ago
  The problem is that data science, often, isn't just data science. It might involve doing a bunch of activities that are strictly not data science - scraping data off the web (Python has requests), visualization for some understanding (Python has dozens of nifty libraries), store data in a small db (Python works nicely with sqllite, among others), convert all of this into an app (Python has Flask, again among others). R's data frame concept was fantastic for those transitioning from 'spreadsheet' analysis but with Pandas, Python has improved and enhanced this concept even further. This is just one example but I think it is this versatility that makes it more attractive than R or Julia.
  - vanderZwan 9y ago
    Yes, in principle Julia should be able to do all of these things too, but it simply doesn't have the decades-old history of libraries to stand on yet.
    - cfusting 9y ago
      Yeah I love Julia but it's hard to argue with Python's data science stack. Sometimes it's almost to easy to get things done.
- cossatot 9y ago
  Julia is really fast but the DataFrames/DataTables package was kind of slow and feature-incomplete. Because Pandas is so optimized, Julia wasn't winning out over Python on the speed side of things, and was definitely losing on the ergonimics, so for data science it wasn't the best choice (unlike say numerical simulation). I'm sure this will change if it hasn't already, especially with the better parallelization that Julia is capable of.

nyrikki 9y ago

I agree but wanted to add in pandas and seaborn. I actually keep a jupyter qtconsole open to use them for ad-hoc data visualization. Pandas replaced excel for me a while ago and I cringe every time I need to abandon seaborn for a Tableau workbook these days. GUI Visualization tools like Tableau or PowerBI seem to error towards presentation, while the defaults for seaborn help discover and visualize data while still producing results good enough to make C*O's happy.

uvtc 9y ago

> Looking forward to the "why" post. Glad that they mentioned that they'll be publishing that soon. I think the "why" is because, for many users: * they want a small and easy, dynamically-typed, garbage-collected language with few surprises (see `import this`). Perl misses out here. * they want something built with C so there's easy access to native libraries. Languages on general-purpose VMs miss out here. * I don't think they want something that transpiles/compiles down to some other lower-level language (which I suspect leads to difficult debugging). Maybe Nim misses out here. * they're comfortable with imperative-style programming. Maybe lisps and Haskell miss out here. * Python is fast enough for most things where C-speed isn't needed. Very-high-level languages chasing dreams of high performance mostly miss out here, IMO. * they want a language that helps them catch their mistakes. IMO, maybe Lua misses out here (with default nil everywhere). * Users want a general-purpose language. Again, Lua misses here, though, of course, Lua's goal isn't to be general-purpose anyway. * I think many users also want a language that's licensed GPL-compatible. Clojure misses out here. * they want a nice helpful community * Users want a language with familiar C/C++/JS/Java -ish syntax. Python misses out here! {ducks} Python has its own set of problems, but it hits all those marks except for the last one. So, it's gotten quite popular.

tim333 9y ago
A lot of people like the Python syntax. It has to be about the most readable of all computer languages.
- uvtc 9y ago
  > A lot of people like the Python syntax. Of course. > It has to be about the most readable of all computer languages. Quite readable, sure. Though my point is that curlies and semicolons are more familiar to many (and easier to edit/navigate in my editor, for that matter), and probably would be preferred. My comment was about why I think many (not all by any stretch, of course) choose a language like Python.
nstart 9y ago
Thats a pretty awesome list!. I'd love for these to be the reasons but somehow I doubt any of us are that thorough. The only reason I used python is that it was the first result that showed up in a Google search "write program to scrape web page" (ah beautiful soup. How I love thee!) The only reason I used flask was Udacitys course taught their web app course with it. I'll admit it, I've never actually analysed the benefits of a language in that way. My only judgement has been "mmm does it feel productive to use this language for what I'm doing". And "how quickly does the documentation take me to a hello world". Which is terribly subjective I admit. I'm suspecting I'm not alone. Or at least I hope I'm not :D. Yes. The embarrassment I feel is real. PS - it's also how I ended up using nginx over Apache. I couldn't really understand Apache in 30 minutes so I used nginx instead. I even remember an interview where I was asked why I used nginx over Apache and I said "um... because it was easier I guess". Definitely not best practices and I'm trying to be more objective nowadays.

rthomas6 9y ago

One "non-programmer" perspective here: Python is the new Perl, except it's fast to read AND write. In Python it's fast to do ...everything. I'm an electrical engineer by trade, and Python gets used a lot. * If I have a giant CSV with data in it: import csv with open(filename) as csvfile: csvreader = csv.DictReader(csvfile) data = list(csvreader) Bam, an array of dictionaries with the data in them, indexed by column name. With listcomps, whatever I need to do will only take a few more lines. And it's this easy for almost any data format. There will be a widely used library for me to `pip install`. * I needed to plot some I/Q data and superimpose some other data as a heat map over it. Complex numbers are built in. Matplotlib, and bam, graph created. It looks as nice as the graphs my coworkers make using Matlab, and I coded it up just as fast, if not faster. * I wanted to play around with ordering a pizza online automatically. Import requests, and off I go. * The code is so easy to read, I've had coworkers update things I wrote when they have never programmed in Python before. I don't have to do any compile steps. I don't have to use a special IDE. I don't have to think in objects and state-passing and all that BS (but I can if I want to!). I don't have to search for some arcane library. I don't have to know a magic build sequence. I don't have to edit config files. I just make cool shit really fast. It runs slower, but for most things I use it for, I don't care if it takes 10 us or 10 seconds. I just want to make it work.

collinmanderson 9y ago

I think Tenserflow is an especially big driver. It's basically making Python the machine learning language of choice.