5 ms·

Python is awful. There are so many one offs in libraries, none agree on a style, it’s slow, and it’s way too easy to do the wrong thing. I often work with data

Python is awful. There are so many one offs in libraries, none agree on a style, it’s slow, and it’s way too easy to do the wrong thing. I often work with data scientists and have to productionize their jupyter notebooks which is pure suboptimal hell. I guess it must be a good easy learning curve for research/scratchpad

nextaccountic 2mo ago
Seems like an excellent user for LLMs
mjr00 2mo ago
> Python is awful. > I often work with data scientists and have to productionize their jupyter notebooks I'm not a huge Python fan, despite working with it fulltime, but this feels like mixing correlation and causation. Data scientists would not be writing good, optimized code in any language.
- slashdave 2mo ago
  Well, yeah. You might need to bring an R notebook into production.
ks2048 2mo ago
> it’s slow For little utilities, it’s faster than a lot of alternatives - just start the interpreter, no compilation needed. It’s all relative, but if you view it as replacing bash scripts for renaming files or running other tools, it’s 100x better.
applfanboysbgon 2mo ago
Python is amazing compared to writing bat/sh scripts. Different languages are for different purposes, using eg. Rust to write system scripts would just be mental. Whether people abuse those languages for purposes they were not intended for is another story, but that doesn't mean the language is inherently bad. And I mean, > and it’s way too easy to do the wrong thing is there another programming language where you believe a data scientist is going to have an easier time writing correct code than Python? Do you think C or Rust or JavaScript or C# make it harder to do the wrong thing?
superze 2mo ago
C'mon man, I don't know any mid and above python developer who seriously has ever considered programming in Jupiter Notebooks. Python is not slow, it's you being the issue. If you are an amateur then it's easy to do the wrong thing, that's true.
fultonn 2mo ago
The performance hell thing is also also kind of a virtue, though. The language is awful, so everything that does any amount of compute is FFI'd into third party libraries (numpy, torch, sympy, etc). Those libraries are for the most part pretty well designed... or, at least, keep you in a few pretty well-constrained patterns that are easy enough to translate. If you've ever read through FORTRAN code from a mathematics department or MATLAB/C/C++ from (non-software) engineering disciplines, then you probably understand why productionizing a jupyter notebook is definitely not the worst of all possible worlds.
pseudosavant 2mo ago
I've never become a fan of the language syntax, but otherwise I've become quite smitten with the total Python ecosystem. The Agents/LLMs + uv combo have made Python so useful and productive for me. My CLI tools publish from Github to PyPI so that I can run tools with just `uvx sql-agent-cli` or `uvx dlna-here. Nothing for me to handle downloading (directly myself), no environment to manually setup, portable (Linux, Windows, Mac, ARM, x86). Easy for agents to run from a skill.md file without any other prereq than uv. Really useful library ecosystem to leverage. No more shell scripts, or TS/JS/PHP backend services. I've even used Python on devices I've built around Raspberry Pi Zero 2 boards.
- braiamp 2mo ago
  > No more shell scripts, or TS/JS/PHP backend services You didn't use Perl before? https://xkcd.com/353/ https://xkcd.com/353/
  - pseudosavant 2mo ago
    That is a great xkcd comic! Sums up my feelings well. I honestly can't stand writing Python by hand (whitespace as syntax??) but here I am flying!
qurren 2mo ago
I'm fine with the language. I just hate that you can't do import numpy==1.5.4 and the code gets exactly the version it wants.
- frollogaston 2mo ago
  The imports/packages situation is terrible in general. This was basically broken until uv, and uv is still not the default. And it's weird how you import files. They're dot-separated packages that resemble file structure but not exactly. NodeJS has a self-explanatory require("./foo.js") or "../foo.js". The newer JS `import` syntax is annoyingly different from `require` but not terrible.
- pjjpo 2mo ago
  Relatively new, scripts can define dependency metadata now adays to achieve that to some degree. Any pip style versioning including an exact match works there. https://packaging.python.org/en/latest/specifications/inline-script-metadata/#example https://packaging.python.org/en/latest/specifications/inline...
edparcell 2mo ago
I used to build quant investment notebooks that had to be deployed in production. Lots of problems with that. Mine were: Notebook cells run out of order, so you often have something that works in a session, but not in a fresh run. Developing against limited datasets, so you fail against things you didn’t know to test for. Small adaptions that have to be made every time the notebook is translated into a code file. We streamlined it by making a graph-structured Computation a first class object that tracked staleness as code or data was updated. Then that class could be directly published, and when failures happened in production, the graph could be serialized with the inputs and intermediate calculation data that caused failure, for investigation in a notebook. We open sourced the implementation https://github.com/janushendersonassetallocation/loman https://github.com/janushendersonassetallocation/loman
ahartmetz 2mo ago
Python is a language for "consenting adults". It doesn't try to prevent you from doing awful things so you can do great things. People who can't program well are given plenty of rope to hang themselves. It shares that with Perl and Ruby. That said, I find it the nicest, cleanest option of the three. I still wouldn't use it for large and complex projects. I really like it for stuff where one might otherwise use shellscript. It's way way better than shellscript... except if it's all about files and running external commands.
- jonhohle 2mo ago
  Until someone adds a dependency…
  - ahartmetz 2mo ago
    I generally just "apt install" them. pyenv and pip (just pyenv really) are a little clunky, but work too.
- Revanche1367 2mo ago
  >language for "consenting adults". It doesn't try to prevent you from doing awful things so you can do great things. People who can't program well are given plenty of rope to hang themselves. This is exactly what I remember being said about C (which I agree with) and often given as a reason why higher level languages like Python or Java have so many protections against things C/C++ allowed (memory management being the biggest one of course). Very funny, and I assume not coincidental, to read this about Python in the modern programming landscape.
  - ahartmetz 2mo ago
    Well, there are plenty of safety mechanisms that Python doesn't have and dangerous (usually powerful, occasionally badly designed) mechanisms that it does have.
- ModernMech 2mo ago
  If that's the case maybe we should stop teaching it to kids?
itishappy 2mo ago
Scripting languages are awful. Python is one of the nicest scripting languages.
- UqWBcuFx6NV4r 2mo ago
  Holy 2000s! Are we really still doing “programming” vs “scripting”?
  - bigstrat2003 2mo ago
    Why wouldn't we? It's still a very relevant distinction, even if the terminology is a bit weird (since scripting is by definition programming). A programmer has very different needs when he writes a script to automate some server tasks versus a complex piece of software. It makes perfect sense that different tools will be more or less effective at meeting those different needs.
    - Revanche1367 2mo ago
      There is no reason why automating a server task cannot be a complex piece of software. I think you’re not aware of just what server automation is used for nowadays in countless cases. Also, Python is more popularly used in ML and web-development areas compared to server automation, so that would mean it’s not a scripting language by your logic.
      - mixmastamyk 2mo ago
        There’s still a great difference in requirements between small and large scale development.
        deleted 2mo ago
        [deleted]
  - itishappy 2mo ago
    My distinction is "systems" vs "scripting" but yeah. I'm bringing Y2K back next!
UqWBcuFx6NV4r 2mo ago
You don’t like working with data scientists. The data science Python ecosystem is really a separate beast that’ll have “normal” coders scratching their heads at the best of times, some of the most popular packages do all sorts of metaprogramming, and the standards for code quality are very different. Don’t blame the language. Well, blame it only in that it allows such things in the first place, which does have some very nice precipitations now and again, as well as some very bad ones. In an age where people are still standing by C over memory-safe systems programming languages, I feel quite comfortable depending Python for the great many things that Python is good at.
renegade-otter 2mo ago
How is that a language problem? Data scientists are not engineers. No matter what language you give them, they will hand you something you are going to have to polish for production. The fact that Python has become the language of choice for machine learning and data science is not a language issue.
stouset 2mo ago
I’ll never not be bitter than Python “won” the scripting language war over Ruby, more or less just because someone did a bit of AI work in it first and it took over that space by default. Ruby has such a nice holistic consistency to it. With a few exceptions, it feels like it was conceived of by one person with a core idea in mind. Python feels like a mess.
- Revanche1367 2mo ago
  Long before AI work, numerical data crunching is what Python became popular for among non-computer scientists and this led directly to the AI use cases. The reason was obviously the lower barrier to entry without having a software engineering background. I also share with you that feeling about Ruby in particular.
- frollogaston 2mo ago
  Python's strength is that it's easy to make C libs work in it. That's also why CPython is de facto the only Python implementation and stuff like PyPy never took off.
  - AdieuToLogic 2mo ago
    > Python's strength is that it's easy to make C libs work in it. SWIG[0] makes working with C libraries trivial for over a dozen programming languages; Perl, Python, and Ruby included. 0 - https://www.swig.org/ https://www.swig.org/
    - frollogaston 2mo ago
      There are reasons all those Py libraries with C code didn't just do it in SWIG.
      - AdieuToLogic 2mo ago
        > There are reasons all those Py libraries with C code didn't just do it in SWIG. And those reasons are?
itissid 2mo ago
When one writes jupyter notebooks for DS you are not writing python. If you ask 10 DSs explain to me what python's attribute lookup model is and why is it different from other OO languages like say Java or C++, they would not care about it. The only thing DSs care about is the rich DS Library support and fast speed of protoyping. To a DS using jupyter this is almost the same feedback loop as a type system at compile time. Have you tried using `uv`'s newer tools? They help a lot e.g. with linting speed, lock management, package dependency separation, correct python version mgmt and no need to fudge with venv.
jihadjihad 2mo ago
> I often work with data scientists and have to productionize their jupyter notebooks At least it’s Python/Jupyter and not R, SAS, or MATLAB.
Daishiman 2mo ago
Spoken like someone who hasn't tried to get data scientists to use other languages productively, where they'll be missing half the libraries, will have to triple their dependency count because you can't count on large common libraries and will have to dig to the ends of GitHub to find random functionality etc.

nextaccountic 2mo ago

Seems like an excellent user for LLMs

mjr00 2mo ago

> Python is awful. > I often work with data scientists and have to productionize their jupyter notebooks I'm not a huge Python fan, despite working with it fulltime, but this feels like mixing correlation and causation. Data scientists would not be writing good, optimized code in any language.

slashdave 2mo ago
Well, yeah. You might need to bring an R notebook into production.

ks2048 2mo ago

> it’s slow For little utilities, it’s faster than a lot of alternatives - just start the interpreter, no compilation needed. It’s all relative, but if you view it as replacing bash scripts for renaming files or running other tools, it’s 100x better.

applfanboysbgon 2mo ago

Python is amazing compared to writing bat/sh scripts. Different languages are for different purposes, using eg. Rust to write system scripts would just be mental. Whether people abuse those languages for purposes they were not intended for is another story, but that doesn't mean the language is inherently bad. And I mean, > and it’s way too easy to do the wrong thing is there another programming language where you believe a data scientist is going to have an easier time writing correct code than Python? Do you think C or Rust or JavaScript or C# make it harder to do the wrong thing?

superze 2mo ago

C'mon man, I don't know any mid and above python developer who seriously has ever considered programming in Jupiter Notebooks. Python is not slow, it's you being the issue. If you are an amateur then it's easy to do the wrong thing, that's true.

fultonn 2mo ago

The performance hell thing is also also kind of a virtue, though. The language is awful, so everything that does any amount of compute is FFI'd into third party libraries (numpy, torch, sympy, etc). Those libraries are for the most part pretty well designed... or, at least, keep you in a few pretty well-constrained patterns that are easy enough to translate. If you've ever read through FORTRAN code from a mathematics department or MATLAB/C/C++ from (non-software) engineering disciplines, then you probably understand why productionizing a jupyter notebook is definitely not the worst of all possible worlds.

pseudosavant 2mo ago

I've never become a fan of the language syntax, but otherwise I've become quite smitten with the total Python ecosystem. The Agents/LLMs + uv combo have made Python so useful and productive for me. My CLI tools publish from Github to PyPI so that I can run tools with just `uvx sql-agent-cli` or `uvx dlna-here. Nothing for me to handle downloading (directly myself), no environment to manually setup, portable (Linux, Windows, Mac, ARM, x86). Easy for agents to run from a skill.md file without any other prereq than uv. Really useful library ecosystem to leverage. No more shell scripts, or TS/JS/PHP backend services. I've even used Python on devices I've built around Raspberry Pi Zero 2 boards.

braiamp 2mo ago
> No more shell scripts, or TS/JS/PHP backend services You didn't use Perl before? https://xkcd.com/353/ https://xkcd.com/353/
- pseudosavant 2mo ago
  That is a great xkcd comic! Sums up my feelings well. I honestly can't stand writing Python by hand (whitespace as syntax??) but here I am flying!

qurren 2mo ago

I'm fine with the language. I just hate that you can't do import numpy==1.5.4 and the code gets exactly the version it wants.

frollogaston 2mo ago
The imports/packages situation is terrible in general. This was basically broken until uv, and uv is still not the default. And it's weird how you import files. They're dot-separated packages that resemble file structure but not exactly. NodeJS has a self-explanatory require("./foo.js") or "../foo.js". The newer JS `import` syntax is annoyingly different from `require` but not terrible.
pjjpo 2mo ago
Relatively new, scripts can define dependency metadata now adays to achieve that to some degree. Any pip style versioning including an exact match works there. https://packaging.python.org/en/latest/specifications/inline-script-metadata/#example https://packaging.python.org/en/latest/specifications/inline...

edparcell 2mo ago

I used to build quant investment notebooks that had to be deployed in production. Lots of problems with that. Mine were: Notebook cells run out of order, so you often have something that works in a session, but not in a fresh run. Developing against limited datasets, so you fail against things you didn’t know to test for. Small adaptions that have to be made every time the notebook is translated into a code file. We streamlined it by making a graph-structured Computation a first class object that tracked staleness as code or data was updated. Then that class could be directly published, and when failures happened in production, the graph could be serialized with the inputs and intermediate calculation data that caused failure, for investigation in a notebook. We open sourced the implementation https://github.com/janushendersonassetallocation/loman https://github.com/janushendersonassetallocation/loman

ahartmetz 2mo ago

Python is a language for "consenting adults". It doesn't try to prevent you from doing awful things so you can do great things. People who can't program well are given plenty of rope to hang themselves. It shares that with Perl and Ruby. That said, I find it the nicest, cleanest option of the three. I still wouldn't use it for large and complex projects. I really like it for stuff where one might otherwise use shellscript. It's way way better than shellscript... except if it's all about files and running external commands.

jonhohle 2mo ago
Until someone adds a dependency…
- ahartmetz 2mo ago
  I generally just "apt install" them. pyenv and pip (just pyenv really) are a little clunky, but work too.
Revanche1367 2mo ago
>language for "consenting adults". It doesn't try to prevent you from doing awful things so you can do great things. People who can't program well are given plenty of rope to hang themselves. This is exactly what I remember being said about C (which I agree with) and often given as a reason why higher level languages like Python or Java have so many protections against things C/C++ allowed (memory management being the biggest one of course). Very funny, and I assume not coincidental, to read this about Python in the modern programming landscape.
- ahartmetz 2mo ago
  Well, there are plenty of safety mechanisms that Python doesn't have and dangerous (usually powerful, occasionally badly designed) mechanisms that it does have.
ModernMech 2mo ago
If that's the case maybe we should stop teaching it to kids?

itishappy 2mo ago

Scripting languages are awful. Python is one of the nicest scripting languages.

UqWBcuFx6NV4r 2mo ago
Holy 2000s! Are we really still doing “programming” vs “scripting”?
- bigstrat2003 2mo ago
  Why wouldn't we? It's still a very relevant distinction, even if the terminology is a bit weird (since scripting is by definition programming). A programmer has very different needs when he writes a script to automate some server tasks versus a complex piece of software. It makes perfect sense that different tools will be more or less effective at meeting those different needs.
  - Revanche1367 2mo ago
    There is no reason why automating a server task cannot be a complex piece of software. I think you’re not aware of just what server automation is used for nowadays in countless cases. Also, Python is more popularly used in ML and web-development areas compared to server automation, so that would mean it’s not a scripting language by your logic.
    - mixmastamyk 2mo ago
      There’s still a great difference in requirements between small and large scale development.
      - deleted 2mo ago
        [deleted]
- itishappy 2mo ago
  My distinction is "systems" vs "scripting" but yeah. I'm bringing Y2K back next!

UqWBcuFx6NV4r 2mo ago

You don’t like working with data scientists. The data science Python ecosystem is really a separate beast that’ll have “normal” coders scratching their heads at the best of times, some of the most popular packages do all sorts of metaprogramming, and the standards for code quality are very different. Don’t blame the language. Well, blame it only in that it allows such things in the first place, which does have some very nice precipitations now and again, as well as some very bad ones. In an age where people are still standing by C over memory-safe systems programming languages, I feel quite comfortable depending Python for the great many things that Python is good at.

renegade-otter 2mo ago

How is that a language problem? Data scientists are not engineers. No matter what language you give them, they will hand you something you are going to have to polish for production. The fact that Python has become the language of choice for machine learning and data science is not a language issue.

stouset 2mo ago

I’ll never not be bitter than Python “won” the scripting language war over Ruby, more or less just because someone did a bit of AI work in it first and it took over that space by default. Ruby has such a nice holistic consistency to it. With a few exceptions, it feels like it was conceived of by one person with a core idea in mind. Python feels like a mess.

Revanche1367 2mo ago
Long before AI work, numerical data crunching is what Python became popular for among non-computer scientists and this led directly to the AI use cases. The reason was obviously the lower barrier to entry without having a software engineering background. I also share with you that feeling about Ruby in particular.
frollogaston 2mo ago
Python's strength is that it's easy to make C libs work in it. That's also why CPython is de facto the only Python implementation and stuff like PyPy never took off.
- AdieuToLogic 2mo ago
  > Python's strength is that it's easy to make C libs work in it. SWIG[0] makes working with C libraries trivial for over a dozen programming languages; Perl, Python, and Ruby included. 0 - https://www.swig.org/ https://www.swig.org/
  - frollogaston 2mo ago
    There are reasons all those Py libraries with C code didn't just do it in SWIG.
    - AdieuToLogic 2mo ago
      > There are reasons all those Py libraries with C code didn't just do it in SWIG. And those reasons are?

itissid 2mo ago

When one writes jupyter notebooks for DS you are not writing python. If you ask 10 DSs explain to me what python's attribute lookup model is and why is it different from other OO languages like say Java or C++, they would not care about it. The only thing DSs care about is the rich DS Library support and fast speed of protoyping. To a DS using jupyter this is almost the same feedback loop as a type system at compile time. Have you tried using `uv`'s newer tools? They help a lot e.g. with linting speed, lock management, package dependency separation, correct python version mgmt and no need to fudge with venv.

jihadjihad 2mo ago

> I often work with data scientists and have to productionize their jupyter notebooks At least it’s Python/Jupyter and not R, SAS, or MATLAB.

Daishiman 2mo ago

Spoken like someone who hasn't tried to get data scientists to use other languages productively, where they'll be missing half the libraries, will have to triple their dependency count because you can't count on large common libraries and will have to dig to the ends of GitHub to find random functionality etc.