4 ms·

I've come across this notion that nowadays machine learning provides (in some sense) the biggest group of Python users a few times recently. What reason is the

I've come across this notion that nowadays machine learning provides (in some sense) the biggest group of Python users a few times recently. What reason is there to suppose this is true? It seems surprising to me.

claytonjy 4y ago
It's really hard to do much ML in anything _except_ python. Virtually everyone improving the ML ecosystems of other language got their start in Python and are knowingly competing with Python (e.g. R, Julia). If you want to get started in ML today, python is the obvious easiest path forward. So, most ML users are python users. I don't know how that group compares to non-ML python users, but I have a feeling there isn't a flood of eager new Django devs the way there is Pytorch users. Most non-ML things you could do with python can be done similarly well in Go/Rust/Typescript, but there's no other option for most ML stuff.
- mjw1007 4y ago
  I found a recentish (2021) survey at [1] which suggests that in 2021 ML was some way behind web development, sysadmin stuff, and data analysis among Python users (and didn't seem to be on the way up the list). [1] https://lp.jetbrains.com/python-developers-survey-2021/#GeneralPythonUsage https://lp.jetbrains.com/python-developers-survey-2021/#Gene...
  - claytonjy 4y ago
    Great source; looks like I've quite underestimated the python-web-dev crowd's size. I'm curious what the longer-term trends look like; not much change between consecutive years. Data analysis is basically a pre-requisite for ML, so the combined "data stuff" usage is quite a lot bigger than web dev usage!
  - aragilar 4y ago
    The obvious next question being, what's the difference between ML and data analysis from the perspective of the survey participants (is ML a strict subset)? Given the values don't add up to 100%, there's likely lots of overlap and so you could easily have web developers choosing Python for the ML ecosystem.
_visgean 4y ago
> What reason is there to suppose this is true? It seems surprising to me. One reason is its just super easy for input output operations. ML is all about data and getting the data to the right place is really easy in python compared to some other languages..
- still_grokking 4y ago
  Which languages? Python is OOP; but the "classical" data-centric languages are actually all more or less in the FP space. (I count array languages and APL-likes to FP in this case). Just an example: You don't have immutable data types by default in Python. This is actually a pretty bad default for data processing tasks.
  - joaogui1 4y ago
    Python has a huge library ecosystem and the average Machine Learning programmer is not a CS geek and so prefers pythonic quasi-OOO then over FP (one of the hurdles for JAX adoption is it's functional paradigm)
    - still_grokking 4y ago
      The claim was that Python is better suited to writing such libs and frameworks than other languages. You now say it's like that because Python has already quite some libs / frameworks in that direction. This looks like circular reasoning. Also the the "prototypical ML dude" comes form the math department. People with math background have a much easier time to grasp FP than procedural programming. FP is much more "natural" when you're used to math. (Procedural programming says things like `x = x + 1`, but even my grandma would know that "this has no solution", or is likewise "plain wrong" ;-))
- zerodensity 4y ago
  Just because python makes sense for ML does not mean that it's primarily used for ML.
deleted 4y ago
[deleted]
zerodensity 4y ago
I can see that a huge part of the ML space is in python. But is the ML space really such a large community? I mean in the companies I have worked for maybe 1 team in a 100 work with some ML stuff while a large majority of the rest come in contact with web/devops/unix scripting. Granted non ML work takes place in many different languages but python is used alot there.

claytonjy 4y ago

It's really hard to do much ML in anything _except_ python. Virtually everyone improving the ML ecosystems of other language got their start in Python and are knowingly competing with Python (e.g. R, Julia). If you want to get started in ML today, python is the obvious easiest path forward. So, most ML users are python users. I don't know how that group compares to non-ML python users, but I have a feeling there isn't a flood of eager new Django devs the way there is Pytorch users. Most non-ML things you could do with python can be done similarly well in Go/Rust/Typescript, but there's no other option for most ML stuff.

mjw1007 4y ago
I found a recentish (2021) survey at [1] which suggests that in 2021 ML was some way behind web development, sysadmin stuff, and data analysis among Python users (and didn't seem to be on the way up the list). [1] https://lp.jetbrains.com/python-developers-survey-2021/#GeneralPythonUsage https://lp.jetbrains.com/python-developers-survey-2021/#Gene...
- claytonjy 4y ago
  Great source; looks like I've quite underestimated the python-web-dev crowd's size. I'm curious what the longer-term trends look like; not much change between consecutive years. Data analysis is basically a pre-requisite for ML, so the combined "data stuff" usage is quite a lot bigger than web dev usage!
- aragilar 4y ago
  The obvious next question being, what's the difference between ML and data analysis from the perspective of the survey participants (is ML a strict subset)? Given the values don't add up to 100%, there's likely lots of overlap and so you could easily have web developers choosing Python for the ML ecosystem.

_visgean 4y ago

> What reason is there to suppose this is true? It seems surprising to me. One reason is its just super easy for input output operations. ML is all about data and getting the data to the right place is really easy in python compared to some other languages..

still_grokking 4y ago
Which languages? Python is OOP; but the "classical" data-centric languages are actually all more or less in the FP space. (I count array languages and APL-likes to FP in this case). Just an example: You don't have immutable data types by default in Python. This is actually a pretty bad default for data processing tasks.
- joaogui1 4y ago
  Python has a huge library ecosystem and the average Machine Learning programmer is not a CS geek and so prefers pythonic quasi-OOO then over FP (one of the hurdles for JAX adoption is it's functional paradigm)
  - still_grokking 4y ago
    The claim was that Python is better suited to writing such libs and frameworks than other languages. You now say it's like that because Python has already quite some libs / frameworks in that direction. This looks like circular reasoning. Also the the "prototypical ML dude" comes form the math department. People with math background have a much easier time to grasp FP than procedural programming. FP is much more "natural" when you're used to math. (Procedural programming says things like `x = x + 1`, but even my grandma would know that "this has no solution", or is likewise "plain wrong" ;-))
zerodensity 4y ago
Just because python makes sense for ML does not mean that it's primarily used for ML.

deleted 4y ago

[deleted]

zerodensity 4y ago

I can see that a huge part of the ML space is in python. But is the ML space really such a large community? I mean in the companies I have worked for maybe 1 team in a 100 work with some ML stuff while a large majority of the rest come in contact with web/devops/unix scripting. Granted non ML work takes place in many different languages but python is used alot there.