8 ms·

I want a common language I can work with. Right now, Python is the only tool which fits the bill. A critical thing is Python does numerics very, very well. Wit

by blagie 5y ago

I want a common language I can work with. Right now, Python is the only tool which fits the bill. A critical thing is Python does numerics very, very well. With machine learning data science, and analytics being what they are, there aren't many alternatives. R, Matlab, and Stata won't do web servers. That's not to mention wonderful integrations with OpenCV, torch, etc. Python is also competent at dev-ops, with tools like ansible, fabric, and similar. It does lots of niches well. For example, it talks to hardware. If you've got a quadcopter or some embedded thing, Python is often a go-to. All of these things need to integrate. A system with Ruby+R+Java will be much worse than one which just uses Python. From there, it's network effects. Python isn't the ideal server language, but it beats a language which _just_ does servers. As a footnote, Python does package management much better than alternatives. pip+virtualenv >> npm + (some subset of require.js / rollup.js / ES2015 modules / AMD / CommonJS / etc.) JavaScript has finally gone from a horrible, no-good, bad language to a somewhat competent one with ES2015, but it has at least another 5-10 years before it can start to compete with Python for numerics or hardware. It's a sane choice if you're front-end heavy, or mobile-heavy. If you're back-end heavy (e.g. an ML system) or hardware-heavy (e.g. something which talks to a dozen cameras), Python often is the only sane choice.

DeathArrow 5y ago
>. R, Matlab, and Stata won't do web servers. Not unless they're pushed to, like Python was. >A critical thing is Python does numerics very, very well. That's not Python doing numerical stuff. That's C code, called from Python.
- mrtranscendence 5y ago
  > That's not Python doing numerical stuff. That's C code, called from Python. That's sort of a distinction without a difference, isn't it? Python can be good for numeric code in many instances because someone has gone through the effort of implementing wrappers atop C and Fortran code. But I'd rather be using the Python wrappers than C or especially Fortran directly, so it makes at least a little sense to say that Python "does numerics [...] well". > Not unless they're pushed to, like Python was. R and Matlab, maybe. A web server in Stata would be a horrible beast to behold. I can't imagine what that would look like. Stata is a terrible general purpose language, excelling only at canned econometrics routines and plotting. I had to write nontrivial Stata code in grad school and it was a painful experience I'd just as soon forget.
  - disgruntledphd2 5y ago
    You can do web stuff in R, but it's a lot harder than it needs to be. R sucks for string interpolation, and a lot of web related stuff is string interpolation.
    - mrtranscendence 5y ago
      Yeah, I'm not surprised by that. The extent of my web experience in R is calling rcurl occasionally, so I've never tried and failed to do anything complicated.
- blagie 5y ago
  It's not C code. It calls into a mixture of C, CUDA, Fortran, and a slew of other things. Someone did the work of finding the best library for me, and integrating them. As for me, I write: A * B It multiplies two matrices. C can't do that. In C, I'd have some unreadable matrix64_multiply(a, b). Readability is a big deal. Math should look more-or-less like math. I can handle 2^4, or 2**4, but if you have mpow(2, 4) in the middle of a complex equation, the number of bugs goes way up. I'd also need to allocate and free memory. Data wrangling is also a disaster in C. Format strings were a really good idea in the seventies, and were a huge step up from BASIC or Python. For 2022? And for that A * B? If I change data types, things just work. This means I can make large algorithmic changes painlessly. Oh, and I can develop interactively. ipython and jupyter are great calculators. Once the math is right, I can copy it into my program. I won't even get started on things like help strings and documentation. Or closures. Closures and modern functional programming are huge. Even in the days of C and C++, I'd rather do math in a Lisp (usually, Scheme). I used to do numerics in C++, and in C before that. It's at least a 10x difference in programmer productivity stepping up to Python. Your comment sounds like someone who has never done numerical stuff before, or at least not serious numerical stuff.
  - tomrod 5y ago
    Yet, Python (and most of her programmers including data scientists, of which I am one) stumble with typing. if 0.1 + 0.2 == 0.3: print('Data is handled as expected.') else: print('Ruh roh.') This fails on Python 3.10 because floats are not decimals, even if we really want them to be. So most folks ignore the complexity (due to naivety or convenience) or architect appropriately after seeing weird bugs. But the "Python is easiest and gets it right" notion that I'm often guilty of has some clear edge cases.
    - dullcrisp 5y ago
      Why would you want decimals for numeric computations though? Rationals might be useful for algebraic computations, but that’d be pretty niche. I’d think decimals would only be useful for presentation and maybe accountancy.
- fractalb 5y ago
  > Not unless they're pushed to, like Python was. Readability of code and ease of use is a big thing. It's just not about pushing hard till we make it. edit: formating
- jonnycomputer 5y ago
  I wouldn't want to do a web-server in MATLAB. I like MATLAB, but no, not that.
- brainz456 5y ago
  Or in some cases, FORTRAN code called from Python iirc.
whimsicalism 5y ago
> it has at least another 5-10 years before it can start to compete with Python for numerics or hardware More, given that no language competes at high-level numerics with Python outside of Julia and numerics in general only adds C++.
- Robotbeat 5y ago
  Fortran >:D
  - whimsicalism 5y ago
    For low-level, fair. I only know of people in astronomy academia who actually use it nowadays though.
    - BenjiWiebe 5y ago
      Signal processing stuff is still sometimes written in it.
    - blagie 4y ago
      There are good technical reasons to use Fortran even in 2022. For example, it avoids** is aliasing (pointers / references / etc.). This allows for some kinds of optimizations impossible in most other languages. It's used in a bunch of small niches, but it has users beyond just astronomy. ** A million disclaimers apply.
      - whimsicalism 4y ago
        I am aware that hypothetically fortran code is the fastest possible. That said, I am not sure how great this aliasing difference is in practice. There is a reason why most BLAS implementations have been rewritten into C.
Denvercoder9 5y ago
> As a footnote, Python does package management much better than alternatives. If you use it as a scripting language, that might very well be the case (it's at least simpler). When you're building libraries or applications, no, definitely not. It's a huge mess, and every 3 years or so we get another new tool that promises to solve it, but just ends up creating a bigger mess.
- whimsicalism 5y ago
  I think poetry actually does solve it
  - Thrymr 5y ago
    Oh, there are a half dozen different tools that solve python package management. Unfortunately, they are mutually incompatible and none solve it for all use cases.
  - wardedVibe 5y ago
    https://xkcd.com/927/ https://xkcd.com/927/
    - AlphaSite 4y ago
      It’s somewhat the opposite situation, poetry is a tool that lots of people are adopting rather than one being pushed by a standards committee. I don’t think it set out to unify a dozen standards, only build a good UX and to be reproducible.
    - whimsicalism 4y ago
      Poetry isn't an additional standard, more like an implementation. It is PEP 518 compliant.
- nerdponx 5y ago
  This is an overly-dire assessment in my opinion. Setuptools + Pip has been more or less stable for years, and no changes in dev environment have been needed since stability was reached. There is a lot of new stuff coming out, for people who want new stuff, but there's nothing wrong with the old stuff if the old stuff works for you, which will remain supported for several more years if not forever.
kbenson 5y ago
> As a footnote, Python does package management much better than alternatives No offense meant, but that sounds like the assessment of someone that has only experienced really shitty package management systems. PyPI has had their XMLRPC search interface disabled for months (a year?) now, so you can't even easily figure out what to install from the shell and have to use other tools/a browser to figure it out. Ultimately, I'm moving towards thinking that most scripting languages actually make for fairly poor systems and admin languages. It used to be the ease of development made all the other problems moot, but there's been large advances in compiled language usability. For scripting languages you're either going to follow the path or Perl or the the path of Python, and they both have their problems. For Perl, you get amazing stability at the expense of eventually the language dying out because there's not enough new features to keep people interested. For Python, the new features mean that module writers want to use them, and then they do, and you'll find that the system Python you have can't handle what modules need for things you want to install, and so you're forced to not just have a separate module environment, but fully separate pythons installed on servers so you cane make use of the module ecosystem. For a specific app you're shipping around this is fine, but when maintaining a fleet of servers and trying to provide a consistent environment, this is a big PITA that you don't want to deal with when you've already chosen a major LTS distro to avoid problems like this. Compiling a scripting language usually doesn't help much either, as that usually results in extremely bloated binaries which have their own packaging and consistency problems. This is cyclical problem we've had so far. A language is used for admin and system work, the requirements of administrators grate up against the usage needs of people that use the language for other things, and it fails for non-admin work and loses popularity and gets replaced be something more popular (Perl -> Python) or it fails for admin work because it caters to other uses and eventually gets replaced by something more stable (what I think will happen to Python, what I think somewhat happened to bash earlier for slightly different reasons). I'm not a huge fan of Go, but I can definitely see why people switch to it for systems work. It alleviates a decent chunk of the consistency problems, so it's at least better in that respect.
- jonnycomputer 5y ago
  >No offense meant, but that sounds like the assessment of someone that has only experienced really shitty package management systems. PyPI has had their XMLRPC search interface disabled for months (a year?) now, so you can't even easily figure out what to install from the shell and have to use other tools/a browser to figure it out. Yes, this is, frankly, an absurd situation for python. And then there is the fact that I end up depending on third-party solutions to manage dependencies. Python is big-time now; stop the amateur hour crap.

by blagie 5y ago

DeathArrow 5y ago

>. R, Matlab, and Stata won't do web servers. Not unless they're pushed to, like Python was. >A critical thing is Python does numerics very, very well. That's not Python doing numerical stuff. That's C code, called from Python.

mrtranscendence 5y ago
> That's not Python doing numerical stuff. That's C code, called from Python. That's sort of a distinction without a difference, isn't it? Python can be good for numeric code in many instances because someone has gone through the effort of implementing wrappers atop C and Fortran code. But I'd rather be using the Python wrappers than C or especially Fortran directly, so it makes at least a little sense to say that Python "does numerics [...] well". > Not unless they're pushed to, like Python was. R and Matlab, maybe. A web server in Stata would be a horrible beast to behold. I can't imagine what that would look like. Stata is a terrible general purpose language, excelling only at canned econometrics routines and plotting. I had to write nontrivial Stata code in grad school and it was a painful experience I'd just as soon forget.
- disgruntledphd2 5y ago
  You can do web stuff in R, but it's a lot harder than it needs to be. R sucks for string interpolation, and a lot of web related stuff is string interpolation.
  - mrtranscendence 5y ago
    Yeah, I'm not surprised by that. The extent of my web experience in R is calling rcurl occasionally, so I've never tried and failed to do anything complicated.
blagie 5y ago
It's not C code. It calls into a mixture of C, CUDA, Fortran, and a slew of other things. Someone did the work of finding the best library for me, and integrating them. As for me, I write: A * B It multiplies two matrices. C can't do that. In C, I'd have some unreadable matrix64_multiply(a, b). Readability is a big deal. Math should look more-or-less like math. I can handle 2^4, or 2**4, but if you have mpow(2, 4) in the middle of a complex equation, the number of bugs goes way up. I'd also need to allocate and free memory. Data wrangling is also a disaster in C. Format strings were a really good idea in the seventies, and were a huge step up from BASIC or Python. For 2022? And for that A * B? If I change data types, things just work. This means I can make large algorithmic changes painlessly. Oh, and I can develop interactively. ipython and jupyter are great calculators. Once the math is right, I can copy it into my program. I won't even get started on things like help strings and documentation. Or closures. Closures and modern functional programming are huge. Even in the days of C and C++, I'd rather do math in a Lisp (usually, Scheme). I used to do numerics in C++, and in C before that. It's at least a 10x difference in programmer productivity stepping up to Python. Your comment sounds like someone who has never done numerical stuff before, or at least not serious numerical stuff.
- tomrod 5y ago
  Yet, Python (and most of her programmers including data scientists, of which I am one) stumble with typing. if 0.1 + 0.2 == 0.3: print('Data is handled as expected.') else: print('Ruh roh.') This fails on Python 3.10 because floats are not decimals, even if we really want them to be. So most folks ignore the complexity (due to naivety or convenience) or architect appropriately after seeing weird bugs. But the "Python is easiest and gets it right" notion that I'm often guilty of has some clear edge cases.
  - dullcrisp 5y ago
    Why would you want decimals for numeric computations though? Rationals might be useful for algebraic computations, but that’d be pretty niche. I’d think decimals would only be useful for presentation and maybe accountancy.
fractalb 5y ago
> Not unless they're pushed to, like Python was. Readability of code and ease of use is a big thing. It's just not about pushing hard till we make it. edit: formating
jonnycomputer 5y ago
I wouldn't want to do a web-server in MATLAB. I like MATLAB, but no, not that.
brainz456 5y ago
Or in some cases, FORTRAN code called from Python iirc.

whimsicalism 5y ago

> it has at least another 5-10 years before it can start to compete with Python for numerics or hardware More, given that no language competes at high-level numerics with Python outside of Julia and numerics in general only adds C++.

Robotbeat 5y ago
Fortran >:D
- whimsicalism 5y ago
  For low-level, fair. I only know of people in astronomy academia who actually use it nowadays though.
  - BenjiWiebe 5y ago
    Signal processing stuff is still sometimes written in it.
  - blagie 4y ago
    There are good technical reasons to use Fortran even in 2022. For example, it avoids** is aliasing (pointers / references / etc.). This allows for some kinds of optimizations impossible in most other languages. It's used in a bunch of small niches, but it has users beyond just astronomy. ** A million disclaimers apply.
    - whimsicalism 4y ago
      I am aware that hypothetically fortran code is the fastest possible. That said, I am not sure how great this aliasing difference is in practice. There is a reason why most BLAS implementations have been rewritten into C.

Denvercoder9 5y ago

> As a footnote, Python does package management much better than alternatives. If you use it as a scripting language, that might very well be the case (it's at least simpler). When you're building libraries or applications, no, definitely not. It's a huge mess, and every 3 years or so we get another new tool that promises to solve it, but just ends up creating a bigger mess.

whimsicalism 5y ago
I think poetry actually does solve it
- Thrymr 5y ago
  Oh, there are a half dozen different tools that solve python package management. Unfortunately, they are mutually incompatible and none solve it for all use cases.
- wardedVibe 5y ago
  https://xkcd.com/927/ https://xkcd.com/927/
  - AlphaSite 4y ago
    It’s somewhat the opposite situation, poetry is a tool that lots of people are adopting rather than one being pushed by a standards committee. I don’t think it set out to unify a dozen standards, only build a good UX and to be reproducible.
  - whimsicalism 4y ago
    Poetry isn't an additional standard, more like an implementation. It is PEP 518 compliant.
nerdponx 5y ago
This is an overly-dire assessment in my opinion. Setuptools + Pip has been more or less stable for years, and no changes in dev environment have been needed since stability was reached. There is a lot of new stuff coming out, for people who want new stuff, but there's nothing wrong with the old stuff if the old stuff works for you, which will remain supported for several more years if not forever.

kbenson 5y ago

> As a footnote, Python does package management much better than alternatives No offense meant, but that sounds like the assessment of someone that has only experienced really shitty package management systems. PyPI has had their XMLRPC search interface disabled for months (a year?) now, so you can't even easily figure out what to install from the shell and have to use other tools/a browser to figure it out. Ultimately, I'm moving towards thinking that most scripting languages actually make for fairly poor systems and admin languages. It used to be the ease of development made all the other problems moot, but there's been large advances in compiled language usability. For scripting languages you're either going to follow the path or Perl or the the path of Python, and they both have their problems. For Perl, you get amazing stability at the expense of eventually the language dying out because there's not enough new features to keep people interested. For Python, the new features mean that module writers want to use them, and then they do, and you'll find that the system Python you have can't handle what modules need for things you want to install, and so you're forced to not just have a separate module environment, but fully separate pythons installed on servers so you cane make use of the module ecosystem. For a specific app you're shipping around this is fine, but when maintaining a fleet of servers and trying to provide a consistent environment, this is a big PITA that you don't want to deal with when you've already chosen a major LTS distro to avoid problems like this. Compiling a scripting language usually doesn't help much either, as that usually results in extremely bloated binaries which have their own packaging and consistency problems. This is cyclical problem we've had so far. A language is used for admin and system work, the requirements of administrators grate up against the usage needs of people that use the language for other things, and it fails for non-admin work and loses popularity and gets replaced be something more popular (Perl -> Python) or it fails for admin work because it caters to other uses and eventually gets replaced by something more stable (what I think will happen to Python, what I think somewhat happened to bash earlier for slightly different reasons). I'm not a huge fan of Go, but I can definitely see why people switch to it for systems work. It alleviates a decent chunk of the consistency problems, so it's at least better in that respect.

jonnycomputer 5y ago
>No offense meant, but that sounds like the assessment of someone that has only experienced really shitty package management systems. PyPI has had their XMLRPC search interface disabled for months (a year?) now, so you can't even easily figure out what to install from the shell and have to use other tools/a browser to figure it out. Yes, this is, frankly, an absurd situation for python. And then there is the fact that I end up depending on third-party solutions to manage dependencies. Python is big-time now; stop the amateur hour crap.