6 ms·

I'm glad to see the quick convergence on Python. I've evaluated Python every ~2 years since 2006 for "data science" tasks (machine learning, statistics, data mu

by rm999 10y ago

I'm glad to see the quick convergence on Python. I've evaluated Python every ~2 years since 2006 for "data science" tasks (machine learning, statistics, data munging, and visualization). I'd argue that Python only properly covered this full data science stack 1.5-2 years ago. R covered this stack adequately probably around 2011-2012. Matlab had this before 2005. What makes Python a superior language to Matlab and R is the ease of software development. It's an easy, pleasant language to work in, and I trust it for production tasks (I've written production R code and it's fairly hard to read and fragile). What's even better is data science is moving 100% into Python 3 (from 2) by 2020: http://www.python3statement.org/ http://www.python3statement.org/

nerdponx 10y ago
How do you feel about Julia? Data analysis in Python has plenty of warts.
- acmj 10y ago
  Julia is great, but IMO, it needs to reach 1.0 first before making a splash. It would be frustrating if you found your tools developed on v0.4 stopped working on v0.5 released a year later. API/ABI stability is critical to the adoption of new programming languages/libraries.
  - Buttons840 10y ago
    I've heard they plan to release Julia 1.0 this year. So keep an eye on it.
    - acmj 10y ago
      Yeah, I heard about it from others, too. Look forward to it. Hope the dev team stick to the plan.
    - tnecniv 10y ago
      Rumor is it's going to be this summer at the annual conference.
- fnord123 10y ago
  http://danluu.com/julialang/ http://danluu.com/julialang/
fnord123 10y ago
If you work in a team mixing data engineers and data scientists then python has been a superior choice for a decade as the different team members are all using the same language to build the platform and to use it. We've had a lot of success weaning people of R and MATLAB in finance (so replace "data scientist" with "quant"). Of course if you work in a field that doesn't have the libraries and can't build them in house then your mileage will certainly vary.
- rm999 10y ago
  This is a great point. Python has been a solid data engineer/pipeline language for years now. It makes it easy for data scientists and data engineers (or people who wear both hats) to work in the same environment and codebase.
  - est 10y ago
    The only area of un-touched by Python is web frontends and mobile apps. Hope Web assembly would change that.
- thearn4 10y ago
  Off-topic, but what is a "data engineer"?
  - nerdponx 10y ago
    I'm sure someone can come along with a better description, but it's kinda like devops/sysadmin but specifically for data storage and access. You could also look it up...
    - mehaveaccount 10y ago
      That's a good description for a data engineer from my end.
    - xapata 10y ago
      I'd include database engineer in that bin. We used to call people DBAs but that seems a bit limiting now.
Alex3917 10y ago
> What makes Python a superior language to Matlab and R is the ease of software development. In practice I think the trends we're seeing have more to do with the fact that most universities now teach CS and data science using Python. Given that Python is approximately as good as R, and it's becoming increasingly much easier to find good people to hire, there's very little reason not to be a Python shop.
- chronicx0 10y ago
  MATLAB requires each function to be its own file. R has terrible object oriented support. Do you seriously think teaching MATLAB or R in universities can save these dying languages? The only people clinging to R and MATLAB are statisticians and old researchers.
- nerdponx 10y ago
  That, and people started transitioning into data science from other fields where Python was already more popular.
CloudYeller 10y ago
Another reason why I'm glad python is winning (and a fun read): http://arrgh.tim-smith.us/ http://arrgh.tim-smith.us/
- shanusmagnus 10y ago
  Love! "R is a shockingly dreadful language for an exceptionally useful data analysis environment. The more you learn about the R language, the worse it will feel. The development environment suffers from literally decades of accretion of stupid hacks from a community containing, to a first-order approximation, zero software engineers." Yup.
  - charles-salvia 10y ago
    The first time I tried to read R code I was immediately disgusted with how '.' is a valid variable name character. Not only valid, but inexplicably also in common use.
    - gone35 10y ago
      That's in part because, unaccountably, '_' denotes assignment in S.
      - yuubi 10y ago
        > unaccountably Maybe because in ASCII-63 the _ codepoint meant left-arrow (which was preserved in at least smalltalk systems much past the 60s)?
      - shanusmagnus 10y ago
        My theory is that they might have been trying to protest against the non-mathematical theft of = (equality) to mean assignment, as = is used in most other languages. (Not that that's an excuse for violating the shit out of the Principle of Least Surprise in a trillion different ways.)
- makeset 10y ago
roystonvassey 10y ago
+ The versatility of Python. It is one of the "funnest" languages to work with because I'm always surprised by the number and variety of packages available to do all kinds of tasks.

by rm999 10y ago

nerdponx 10y ago

How do you feel about Julia? Data analysis in Python has plenty of warts.

acmj 10y ago
Julia is great, but IMO, it needs to reach 1.0 first before making a splash. It would be frustrating if you found your tools developed on v0.4 stopped working on v0.5 released a year later. API/ABI stability is critical to the adoption of new programming languages/libraries.
- Buttons840 10y ago
  I've heard they plan to release Julia 1.0 this year. So keep an eye on it.
  - acmj 10y ago
    Yeah, I heard about it from others, too. Look forward to it. Hope the dev team stick to the plan.
  - tnecniv 10y ago
    Rumor is it's going to be this summer at the annual conference.
fnord123 10y ago
http://danluu.com/julialang/ http://danluu.com/julialang/

fnord123 10y ago

If you work in a team mixing data engineers and data scientists then python has been a superior choice for a decade as the different team members are all using the same language to build the platform and to use it. We've had a lot of success weaning people of R and MATLAB in finance (so replace "data scientist" with "quant"). Of course if you work in a field that doesn't have the libraries and can't build them in house then your mileage will certainly vary.

rm999 10y ago
This is a great point. Python has been a solid data engineer/pipeline language for years now. It makes it easy for data scientists and data engineers (or people who wear both hats) to work in the same environment and codebase.
- est 10y ago
  The only area of un-touched by Python is web frontends and mobile apps. Hope Web assembly would change that.
thearn4 10y ago
Off-topic, but what is a "data engineer"?
- nerdponx 10y ago
  I'm sure someone can come along with a better description, but it's kinda like devops/sysadmin but specifically for data storage and access. You could also look it up...
  - mehaveaccount 10y ago
    That's a good description for a data engineer from my end.
  - xapata 10y ago
    I'd include database engineer in that bin. We used to call people DBAs but that seems a bit limiting now.

Alex3917 10y ago

> What makes Python a superior language to Matlab and R is the ease of software development. In practice I think the trends we're seeing have more to do with the fact that most universities now teach CS and data science using Python. Given that Python is approximately as good as R, and it's becoming increasingly much easier to find good people to hire, there's very little reason not to be a Python shop.

chronicx0 10y ago
MATLAB requires each function to be its own file. R has terrible object oriented support. Do you seriously think teaching MATLAB or R in universities can save these dying languages? The only people clinging to R and MATLAB are statisticians and old researchers.
nerdponx 10y ago
That, and people started transitioning into data science from other fields where Python was already more popular.

CloudYeller 10y ago

Another reason why I'm glad python is winning (and a fun read): http://arrgh.tim-smith.us/ http://arrgh.tim-smith.us/

shanusmagnus 10y ago
Love! "R is a shockingly dreadful language for an exceptionally useful data analysis environment. The more you learn about the R language, the worse it will feel. The development environment suffers from literally decades of accretion of stupid hacks from a community containing, to a first-order approximation, zero software engineers." Yup.
- charles-salvia 10y ago
  The first time I tried to read R code I was immediately disgusted with how '.' is a valid variable name character. Not only valid, but inexplicably also in common use.
  - gone35 10y ago
    That's in part because, unaccountably, '_' denotes assignment in S.
    - yuubi 10y ago
      > unaccountably Maybe because in ASCII-63 the _ codepoint meant left-arrow (which was preserved in at least smalltalk systems much past the 60s)?
    - shanusmagnus 10y ago
      My theory is that they might have been trying to protest against the non-mathematical theft of = (equality) to mean assignment, as = is used in most other languages. (Not that that's an excuse for violating the shit out of the Principle of Least Surprise in a trillion different ways.)
makeset 10y ago

roystonvassey 10y ago

+ The versatility of Python. It is one of the "funnest" languages to work with because I'm always surprised by the number and variety of packages available to do all kinds of tasks.