3 ms·

If Python doesn't introduce multithreading before 2030, it will die a slow and painful death

FlyingSnake 4y ago
Python notwithstanding its certain limitations, will remain a mainstay language beyond 2030. It is a lucrative option in writing glue code, scripts, and almost all of data engineering and related fields depend on Python.
- oxfordmale 4y ago
  Java is still a mainstay language, despite its popularity slowly dropping for more than a decade. Python will be around for many years; however, now we have hit the end of Moore's law, the easiest way to speed up code is to multi-thread. I code exclusively in Python, so I can't judge if languages like Julia are serious contenders. I also agree that building up the same ecosystem as in Python will take some time, and therefore I do not predict a rapid decline. However, many of the things I use Python for can be easily done in another scripting language.
skrtskrt 4y ago
if other languages don't have equivalents for the devex/productivity enhancements of: * Django * FastAPI/Flask * Numpy/Pandas * PySpark * Jupyter Notebooks (gross I know but this is what "data analysts" and "ML/data engineers" use at many places) Then Python will stick around for forever. I would love if it the Go community would quit the "you don't need a framework, DUH it's GO" attitude. Make a Django/Rails for Go and there would be 10x the Go jobs.
- robswc 4y ago
  I've gotten into the "debate." People always suggest using anything but python but then provide no alternatives to the frameworks mentioned above. You _could_ certainly write it in Go. I write a lot of stuff in Go and I love it. I just don't want to recreate Django for the 100th time though, so I don't and I just use Python. By all means, if someone wants to write these frameworks, people will use them.
  - skrtskrt 4y ago
    agreed. I write Go, Rust, and Python (now only when I have to), but If I had to stand up a small business with a CRUD app in a couple weeks I'd go with Django no doubt - mainly for the no-worries auth and user management within the same monolith as everything else. I don't care about the ORM, I can quickly map out a relational schema. It's the user stuff that's killer. With more time, yeah I might choose Go or Rust and setup a couple different nodes - an API gateway, a user auth node like KeyCloak, Ory, or SuperTokens, then a Go backend. But it would be so nice to have that all ready in one.
    - robswc 4y ago
      Yep. I do enjoy building things but I learn more and more how important it is not to focus on things that are already "solved" if they aren't part of your core business. Distractions can scale exponentially.
- nerdponx 4y ago
  What's with the scare quotes on "data analysts"? Just because you read on HN about how some such tool is bad, doesn't actually make it bad. Hating on notebooks is like hating on Excel: it's trendy in some circles, but generally only espoused by the ignorant.
  - skrtskrt 4y ago
    I've held jobs where we just productionalized data analyst code from notebooks into data pipelines. There's absolutely nothing wrong with notebooks, but they are a serious PITA to take from being handed basically the musings of a data intern into a a production pipeline. I used scare quotes because unfortunately in many places there are essentially zero qualifications to start running around doing those jobs.
- oxfordmale 4y ago
  Ironically, you mention PySpark. It is a Python wrapper around a Java/Scala code base. Jupyter Notebook supports many different scripting languages, such as Julia and R. I agree that Numpy/Pandas will introduce more migration friction, and that is why I mentioned a slow death, similar to the trajectory Java is currently on. Java is still in the top three. However, its popularity has dropped in recent years. It is worth noting that both Cobol and Fortran are still in the top 30 programming languages.
  - skrtskrt 4y ago
    it is funny that PySpark is just a Java/Scala wrapper, but the combo on PySpark + NumPy + Panda + Jupyter notebooks means that Python is option 1, 2, and 3 for any company that's just like "hmm let's start doing some data analysis/ML/whatever"! Also R is what's most often taught in school in my experience and boy does Python feel like a breath of fresh air when you've been trained in R. When you're coming out of college trained in data analysis but not software engineering per se, you've got no idea about the larger world of what other languages could offer.
- crabbone 4y ago
  All of this exists in, basically, any mainstream language in some capacity. A lot of them offer superior alternatives. There's nothing about Python's quality that's worth keeping. The reason for Python's popularity is its popularity. The reason why it won't die is inertia. But, say, someone creates a "killer app", that runs on hardware different enough from anything we have today, and that someone hates Python (smartphones are the most recent example of such a change), then there'd be a chance to dislodge it. But I struggle to see how Python would "organically" die.
prettyStandard 4y ago
I know python's the future because of just how bad it's performance is. /S
bee_rider 4y ago
It doesn’t seem to have held it back so far.
harveywi 4y ago
Python is a religion. No quantity of technical shortcomings will dislodge its habitat in the minds of its acolytes.
- ReflectedImage 4y ago
  It's got the fastest development speed of the main stream languages. It's got a huge technical advantage.
  - crabbone 4y ago
    This is just false (spoken from a perspective of infra / system / automation person, being in the field for many years). But, this is very typical of religious people: to claim anything that suites their agenda w/o even trying to verify their claims, not even being bothered by whether the claim even makes sense. First of all, there's no such thing as a "development speed of a language", just like there isn't a development speed of ice-cream. It's just kind of a Jabberwocky: it feels like it's in English, but it doesn't really mean anything. Development speed differs by the kind of project (eg. Web site vs filesystem), quality requirements, size of the team working on the project, expertise of the team working on the project... Needless to say that there are areas where Python is entirely not applicable, so, the development speed wouldn't even be a factor. But, even in areas where it's commonly used there are often languages that will compete for this metric, and there are certainly teams using other languages that will beat teams using Python. But, overall, Python projects tend to be easy to start and hard to develop further and to refine. Python projects tend to fare worse in large teams. Python also doesn't attract high-caliber programmers, while also is often the first language a programmer learns, so it tends to be populated by mediocre-bad programmers (similar problem used to exist in Java before it was replaced by Python in intro to CS courses). Finally, a huge portion of development speed rests on company's infrastructure: how quickly and reliably can developers test their code plays a tremendous role in productivity. Ironically, Python tooling is so bad that sometimes it's faster to compile a C++ program of equivalent size than to install a bunch of Python packages implementing the same thing (god forbid you are using Anaconda, because that can take hours and days in the worst case to install a handful of packages).

Python notwithstanding its certain limitations, will remain a mainstay language beyond 2030. It is a lucrative option in writing glue code, scripts, and almost all of data engineering and related fields depend on Python.

oxfordmale 4y ago
Java is still a mainstay language, despite its popularity slowly dropping for more than a decade. Python will be around for many years; however, now we have hit the end of Moore's law, the easiest way to speed up code is to multi-thread. I code exclusively in Python, so I can't judge if languages like Julia are serious contenders. I also agree that building up the same ecosystem as in Python will take some time, and therefore I do not predict a rapid decline. However, many of the things I use Python for can be easily done in another scripting language.

skrtskrt 4y ago

if other languages don't have equivalents for the devex/productivity enhancements of: * Django * FastAPI/Flask * Numpy/Pandas * PySpark * Jupyter Notebooks (gross I know but this is what "data analysts" and "ML/data engineers" use at many places) Then Python will stick around for forever. I would love if it the Go community would quit the "you don't need a framework, DUH it's GO" attitude. Make a Django/Rails for Go and there would be 10x the Go jobs.

robswc 4y ago
I've gotten into the "debate." People always suggest using anything but python but then provide no alternatives to the frameworks mentioned above. You _could_ certainly write it in Go. I write a lot of stuff in Go and I love it. I just don't want to recreate Django for the 100th time though, so I don't and I just use Python. By all means, if someone wants to write these frameworks, people will use them.
- skrtskrt 4y ago
  agreed. I write Go, Rust, and Python (now only when I have to), but If I had to stand up a small business with a CRUD app in a couple weeks I'd go with Django no doubt - mainly for the no-worries auth and user management within the same monolith as everything else. I don't care about the ORM, I can quickly map out a relational schema. It's the user stuff that's killer. With more time, yeah I might choose Go or Rust and setup a couple different nodes - an API gateway, a user auth node like KeyCloak, Ory, or SuperTokens, then a Go backend. But it would be so nice to have that all ready in one.
  - robswc 4y ago
    Yep. I do enjoy building things but I learn more and more how important it is not to focus on things that are already "solved" if they aren't part of your core business. Distractions can scale exponentially.
nerdponx 4y ago
What's with the scare quotes on "data analysts"? Just because you read on HN about how some such tool is bad, doesn't actually make it bad. Hating on notebooks is like hating on Excel: it's trendy in some circles, but generally only espoused by the ignorant.
- skrtskrt 4y ago
  I've held jobs where we just productionalized data analyst code from notebooks into data pipelines. There's absolutely nothing wrong with notebooks, but they are a serious PITA to take from being handed basically the musings of a data intern into a a production pipeline. I used scare quotes because unfortunately in many places there are essentially zero qualifications to start running around doing those jobs.
oxfordmale 4y ago
Ironically, you mention PySpark. It is a Python wrapper around a Java/Scala code base. Jupyter Notebook supports many different scripting languages, such as Julia and R. I agree that Numpy/Pandas will introduce more migration friction, and that is why I mentioned a slow death, similar to the trajectory Java is currently on. Java is still in the top three. However, its popularity has dropped in recent years. It is worth noting that both Cobol and Fortran are still in the top 30 programming languages.
- skrtskrt 4y ago
  it is funny that PySpark is just a Java/Scala wrapper, but the combo on PySpark + NumPy + Panda + Jupyter notebooks means that Python is option 1, 2, and 3 for any company that's just like "hmm let's start doing some data analysis/ML/whatever"! Also R is what's most often taught in school in my experience and boy does Python feel like a breath of fresh air when you've been trained in R. When you're coming out of college trained in data analysis but not software engineering per se, you've got no idea about the larger world of what other languages could offer.
crabbone 4y ago
All of this exists in, basically, any mainstream language in some capacity. A lot of them offer superior alternatives. There's nothing about Python's quality that's worth keeping. The reason for Python's popularity is its popularity. The reason why it won't die is inertia. But, say, someone creates a "killer app", that runs on hardware different enough from anything we have today, and that someone hates Python (smartphones are the most recent example of such a change), then there'd be a chance to dislodge it. But I struggle to see how Python would "organically" die.

prettyStandard 4y ago

I know python's the future because of just how bad it's performance is. /S

bee_rider 4y ago

It doesn’t seem to have held it back so far.

harveywi 4y ago

Python is a religion. No quantity of technical shortcomings will dislodge its habitat in the minds of its acolytes.

ReflectedImage 4y ago
It's got the fastest development speed of the main stream languages. It's got a huge technical advantage.
- crabbone 4y ago
  This is just false (spoken from a perspective of infra / system / automation person, being in the field for many years). But, this is very typical of religious people: to claim anything that suites their agenda w/o even trying to verify their claims, not even being bothered by whether the claim even makes sense. First of all, there's no such thing as a "development speed of a language", just like there isn't a development speed of ice-cream. It's just kind of a Jabberwocky: it feels like it's in English, but it doesn't really mean anything. Development speed differs by the kind of project (eg. Web site vs filesystem), quality requirements, size of the team working on the project, expertise of the team working on the project... Needless to say that there are areas where Python is entirely not applicable, so, the development speed wouldn't even be a factor. But, even in areas where it's commonly used there are often languages that will compete for this metric, and there are certainly teams using other languages that will beat teams using Python. But, overall, Python projects tend to be easy to start and hard to develop further and to refine. Python projects tend to fare worse in large teams. Python also doesn't attract high-caliber programmers, while also is often the first language a programmer learns, so it tends to be populated by mediocre-bad programmers (similar problem used to exist in Java before it was replaced by Python in intro to CS courses). Finally, a huge portion of development speed rests on company's infrastructure: how quickly and reliably can developers test their code plays a tremendous role in productivity. Ironically, Python tooling is so bad that sometimes it's faster to compile a C++ program of equivalent size than to install a bunch of Python packages implementing the same thing (god forbid you are using Anaconda, because that can take hours and days in the worst case to install a handful of packages).