4 ms·

Perhaps there should be apostasy punishments for Python defectors. /s Python's typing story is far from perfect and rather annoying compared to other languages

Perhaps there should be apostasy punishments for Python defectors. /s Python's typing story is far from perfect and rather annoying compared to other languages. Python is good for interacting with the operating system, networking and other things. But wanting a proper compiler is an honest motivation.

typon 8y ago
Yet the entire data science world is built on Python. I don't understand this blatant disregard of reality.
- Annatar 8y ago
  Wrong: the data science world is built on R, which was designed from the ground up to be the freeware alternative to SAS and Mathematica.
  - acomjean 8y ago
    Oddly in biology I’m seeing an uptick in R use as it’s easier to install and run tools in R for non programmers. The python 2/3 confusion (why doesn’t this google result work?), and various install packages (conda/pip...) seem to have made R more popular. This is despite biopython which is quite good. The newest single sell rna seq analysis tools are in R. R is a strang thing to me, though it has its moments (graphing). I still think pandas which gives python R like data frames is one of the great tools out there.
    - snackematician 8y ago
      pandas gives base R a run for its money, but I find the multi-indexing really confusing, and R tidyverse is just way way nicer than pandas. The sentiment that R is easier than python to install/manage is common among some R users but I disagree. With R I'm constantly facing dependency hell problems -- one package wants an old version of R, while another needs the newest. Conda/venv solves this problem very nicely in python. Recently lI've been using Julia more and really like it. One nonstandard case where I've found it really shines is parsing large bioinformatics data, such as pileup files. Python is just so slowww here, but neither do I want to write a C program to do the text parsing. Julia is perfect in this case.
- shadowmint 8y ago
  no, the trivial frontend is built in python; the real code is usually c++ or c. Ignore that reality if you want to, but it is a fact. Big complicated python projects are seldom pure python, they are usually a friendly python frontend to a serious application written in something else. It seems in no way remarkable that someone wanting to build a serious backend type piece of functionality would pick another language that was, just for example, multithreaded.
  - mlthoughts2018 8y ago
    The CPython interpreter itself is a C program. Acting like extension modules “aren’t Python” is highly disingenuous.
    - shadowmint 8y ago
      Oh please, go read the source code for tensorflow and then come back and we can have a real conversation.
  - typon 8y ago
    I don't understand this logic. The users are learning Python, not C++ when they're trying to learn data science or implement a machine learning model. Should I say Tensor flow isn't written in C++ but CUDA or OpenCL? Any self respecting researcher is training their models on GPU or FPGA, not CPU. The point is that Python is the entry point for large majority of data scientists currently and its absolutely disingenuous to try to dispute that reality.
    - _underfl0w_ 8y ago
      > The users are learning Python, not C++ The key word here is "users". Python is fantastic for users. It allows users to get things done without having to worry about types or memory or the underlying hardware in any way.
      - wwright 8y ago
- mlthoughts2018 8y ago
  Not just data science: the entire machine learning and scientific computing world, if we’re honest. Need to do huge numerical linear algebra operations very fast with great memory safety and ease of use? Python.
  - spaced-out 8y ago
    Technically, you'd use a C/C++ library with a Python wrapper.
    - mlthoughts2018 8y ago
      Yes, that is called Python.
      - throwawaymath 8y ago
        No it isn't? The substantive portions of the software aren't written in Python. Python is a thin interface to the actual library. Users of the library who need to rapidly prototype can leverage the library using its Python frontend, but the authors of the library did not write the library in Python.
        mlthoughts2018 8y ago
        To program in CPython is to use thin wrappers around C-level implemented extension types and modules. For example, all major built-in container types. If you program in Python for performance-critical applications, as I do, then you will frequently hand-write C-level extension modules, write extension modules in Cython, use JIT tools like numba. All of this stuff “is Python.”
        wrmsr 8y ago
        even for the crud apps so frequently referred to in this thread the equivalent 'real code' is the sql rdbms written in c or c++ no matter what the webapp language is. i don't advocate using python in million line monoliths despite it being my favorite lang in general at the moment but the volume of cognitive dissonance and false dichotomies in this thread is crushing.
- sfifs 8y ago
  Python is ideally suited for data science and machine learning because 80% of the work tends to be data exploration, transformations and feature engineering and 19% of the work is iterative development of models. In such workflows the lack of type checking & simple REPL is a huge feature. Most of the underlying code for ML is anyway written in Fortran or C and Python becomes a very convenient front end glue For the 1% work that is actually engineering the model's deployment in production, it's not uncommon to use faster languages with the exported model. If what you're doing is writing a vanilla CRUD server that gets thousands or hundreds of thousands of hits a second, Python is probably a bad choice.
  - makapuf 8y ago
    Uh why ? Thousands of hits per second will generally be IO or DB bound and will be done using clear code and a minimum of boilerplate.
  - vonseel 8y ago
    I’m not sure how many people will ever even work on CRUD apps of that scale. Even a mere 1,000 req/sec adds up to 86.4M per day. That sounds like something at least as popular as Stack Overflow, for example. How many web apps are there, really, that have millions of users?
    - pimeys 8y ago
      Or almost any advertisement tracker server.
      - vonseel 8y ago
        Isn't that market dominated by a few, or at the max, no more than 10-15 companies? My point is just that are there really more than a few dozen or maybe a hundred companies that see more than 1000 req/sec (on web apps)?
        tdfx 8y ago
        Depends if you're talking maximum or average. Getting to the front page of HN could easily give you a burst of 1000 req/sec. I would agree there's very few CRUD apps that deal with 1000 req/sec on average. On the other hands, there's various APIs and tracking type of services that probably do handle well in excess of that, being run by not-large companies none of us have ever heard of.
- phkahler 8y ago
  None of the libraries that do the computations are written in python. None of the GUIs or matplotlib are written in python. If you want to build large projects or need performance, python is not the right language. I do like python a lot - for the things it's good at.
- ulucs 8y ago
  I think Python is at a weird place regarding data science. If you need easy-to-use tools, R libraries cover significantly more ground than Python. You don't have to worry about 2/3 compatibility of niche libraries. Plus the functional language forces the libraries to have similar API's. You also get to use tidyverse and ggplot2 in all of their majestic beauty. At most you have to write a "%>% do() %>%" to get around weird library API choices. However if you are in the frontier, you could avoid so much awkward notation with an array-first language like Matlab or Julia. You have "^" instead of "np.linalg.matrix_power", "/","\" instead of "np.linalg.lstsq", the . operator and so much more life-saving tools.
  - orangejewce 8y ago
    R has so many problems itself! Attempting to run it at any sort of scale is basically impossible. Its dependency management is atrocious. You can't thread or multi-process effectively. The amount of overhead to even install R on a system is incredible, and amazingly its documentation is even worse than Python. We use R as our foundational Data science language and it has caused nothing but immense pain for everyone attempting to wrap the packages in scalable software.
bayesian_horse 8y ago
There are type checkers now. Personally I find type systems a lot more annoying because they lead to more verbose code and more cognitive load. There are studies that show bugs being proportional to the number of lines of code, regardless of language. And that effect seems to be more significant than the difference among dynamic and static type systems. Defensive programming can be achieved through various means. Type checking is one of them and often can't replace Unit and Integration testing. Validation of Inputs or other data is another issue.
- melling 8y ago
  I don’t think Haskell or OCaml are more verbose than Python. Other traditional languages are adopting type inference, etc. We could be near an inflection point where we get statically type languages that are as convenient as Python.
  - bayesian_horse 8y ago
    Oh sure, but Haskell and OCaml are much less popular than Python, with consequences for community building, recruiting, reusing existing code bases etc. The popular statically typed languages which are actually Python's competitors are for example C++, C#, Java and the like. Sure as hell they are more verbose. Even Typescript was extremely intimidating to me when I started to try and understand Typescript code bases. And I am NOT going to try and teach Haskell or Ocaml as a first programming language...
  - lerax 8y ago
    > I don’t think Haskell or OCaml are more verbose than Python. I love Haskell computing model, but holy hell, why is so difficult using things like State Monad or even a trivial thing like a global counter [1] ? Why something so simple should be so difficult? Don't say it's because functional programming, Scheme/Racket is not that difficult! Haskell and OCaml has their own problems, I agreed with you that both languages are more expressive but the code density is not necessary a good thing neither. [1]: https://github.com/scotty-web/scotty/blob/master/examples/globalstate.hs#L68 https://github.com/scotty-web/scotty/blob/master/examples/gl...
kristianov 8y ago
Wrestling with borrow checker is enough punishment already. /s
vonseel 8y ago
If one were to write perfectly type-annotated Python, and a hypothetical perfectly annotated standard library existed, what’s preventing a faster runtime being developed with performance similar to C#, for example? That will never happen, so maybe “what if” isn’t even worth asking
- catern 8y ago
  It's already being done: https://github.com/mypyc/mypyc https://github.com/mypyc/mypyc

typon 8y ago

Yet the entire data science world is built on Python. I don't understand this blatant disregard of reality.

Annatar 8y ago
Wrong: the data science world is built on R, which was designed from the ground up to be the freeware alternative to SAS and Mathematica.
- acomjean 8y ago
  Oddly in biology I’m seeing an uptick in R use as it’s easier to install and run tools in R for non programmers. The python 2/3 confusion (why doesn’t this google result work?), and various install packages (conda/pip...) seem to have made R more popular. This is despite biopython which is quite good. The newest single sell rna seq analysis tools are in R. R is a strang thing to me, though it has its moments (graphing). I still think pandas which gives python R like data frames is one of the great tools out there.
  - snackematician 8y ago
    pandas gives base R a run for its money, but I find the multi-indexing really confusing, and R tidyverse is just way way nicer than pandas. The sentiment that R is easier than python to install/manage is common among some R users but I disagree. With R I'm constantly facing dependency hell problems -- one package wants an old version of R, while another needs the newest. Conda/venv solves this problem very nicely in python. Recently lI've been using Julia more and really like it. One nonstandard case where I've found it really shines is parsing large bioinformatics data, such as pileup files. Python is just so slowww here, but neither do I want to write a C program to do the text parsing. Julia is perfect in this case.
shadowmint 8y ago
no, the trivial frontend is built in python; the real code is usually c++ or c. Ignore that reality if you want to, but it is a fact. Big complicated python projects are seldom pure python, they are usually a friendly python frontend to a serious application written in something else. It seems in no way remarkable that someone wanting to build a serious backend type piece of functionality would pick another language that was, just for example, multithreaded.
- mlthoughts2018 8y ago
  The CPython interpreter itself is a C program. Acting like extension modules “aren’t Python” is highly disingenuous.
  - shadowmint 8y ago
    Oh please, go read the source code for tensorflow and then come back and we can have a real conversation.
- typon 8y ago
  I don't understand this logic. The users are learning Python, not C++ when they're trying to learn data science or implement a machine learning model. Should I say Tensor flow isn't written in C++ but CUDA or OpenCL? Any self respecting researcher is training their models on GPU or FPGA, not CPU. The point is that Python is the entry point for large majority of data scientists currently and its absolutely disingenuous to try to dispute that reality.
  - _underfl0w_ 8y ago
    > The users are learning Python, not C++ The key word here is "users". Python is fantastic for users. It allows users to get things done without having to worry about types or memory or the underlying hardware in any way.
    - wwright 8y ago
mlthoughts2018 8y ago
Not just data science: the entire machine learning and scientific computing world, if we’re honest. Need to do huge numerical linear algebra operations very fast with great memory safety and ease of use? Python.
- spaced-out 8y ago
  Technically, you'd use a C/C++ library with a Python wrapper.
  - mlthoughts2018 8y ago
    Yes, that is called Python.
    - throwawaymath 8y ago
      No it isn't? The substantive portions of the software aren't written in Python. Python is a thin interface to the actual library. Users of the library who need to rapidly prototype can leverage the library using its Python frontend, but the authors of the library did not write the library in Python.
      - mlthoughts2018 8y ago
        To program in CPython is to use thin wrappers around C-level implemented extension types and modules. For example, all major built-in container types. If you program in Python for performance-critical applications, as I do, then you will frequently hand-write C-level extension modules, write extension modules in Cython, use JIT tools like numba. All of this stuff “is Python.”
        wrmsr 8y ago
        even for the crud apps so frequently referred to in this thread the equivalent 'real code' is the sql rdbms written in c or c++ no matter what the webapp language is. i don't advocate using python in million line monoliths despite it being my favorite lang in general at the moment but the volume of cognitive dissonance and false dichotomies in this thread is crushing.
sfifs 8y ago
Python is ideally suited for data science and machine learning because 80% of the work tends to be data exploration, transformations and feature engineering and 19% of the work is iterative development of models. In such workflows the lack of type checking & simple REPL is a huge feature. Most of the underlying code for ML is anyway written in Fortran or C and Python becomes a very convenient front end glue For the 1% work that is actually engineering the model's deployment in production, it's not uncommon to use faster languages with the exported model. If what you're doing is writing a vanilla CRUD server that gets thousands or hundreds of thousands of hits a second, Python is probably a bad choice.
- makapuf 8y ago
  Uh why ? Thousands of hits per second will generally be IO or DB bound and will be done using clear code and a minimum of boilerplate.
- vonseel 8y ago
  I’m not sure how many people will ever even work on CRUD apps of that scale. Even a mere 1,000 req/sec adds up to 86.4M per day. That sounds like something at least as popular as Stack Overflow, for example. How many web apps are there, really, that have millions of users?
  - pimeys 8y ago
    Or almost any advertisement tracker server.
    - vonseel 8y ago
      Isn't that market dominated by a few, or at the max, no more than 10-15 companies? My point is just that are there really more than a few dozen or maybe a hundred companies that see more than 1000 req/sec (on web apps)?
      - tdfx 8y ago
        Depends if you're talking maximum or average. Getting to the front page of HN could easily give you a burst of 1000 req/sec. I would agree there's very few CRUD apps that deal with 1000 req/sec on average. On the other hands, there's various APIs and tracking type of services that probably do handle well in excess of that, being run by not-large companies none of us have ever heard of.
phkahler 8y ago
None of the libraries that do the computations are written in python. None of the GUIs or matplotlib are written in python. If you want to build large projects or need performance, python is not the right language. I do like python a lot - for the things it's good at.
ulucs 8y ago
I think Python is at a weird place regarding data science. If you need easy-to-use tools, R libraries cover significantly more ground than Python. You don't have to worry about 2/3 compatibility of niche libraries. Plus the functional language forces the libraries to have similar API's. You also get to use tidyverse and ggplot2 in all of their majestic beauty. At most you have to write a "%>% do() %>%" to get around weird library API choices. However if you are in the frontier, you could avoid so much awkward notation with an array-first language like Matlab or Julia. You have "^" instead of "np.linalg.matrix_power", "/","\" instead of "np.linalg.lstsq", the . operator and so much more life-saving tools.
- orangejewce 8y ago
  R has so many problems itself! Attempting to run it at any sort of scale is basically impossible. Its dependency management is atrocious. You can't thread or multi-process effectively. The amount of overhead to even install R on a system is incredible, and amazingly its documentation is even worse than Python. We use R as our foundational Data science language and it has caused nothing but immense pain for everyone attempting to wrap the packages in scalable software.

bayesian_horse 8y ago

There are type checkers now. Personally I find type systems a lot more annoying because they lead to more verbose code and more cognitive load. There are studies that show bugs being proportional to the number of lines of code, regardless of language. And that effect seems to be more significant than the difference among dynamic and static type systems. Defensive programming can be achieved through various means. Type checking is one of them and often can't replace Unit and Integration testing. Validation of Inputs or other data is another issue.

melling 8y ago
I don’t think Haskell or OCaml are more verbose than Python. Other traditional languages are adopting type inference, etc. We could be near an inflection point where we get statically type languages that are as convenient as Python.
- bayesian_horse 8y ago
  Oh sure, but Haskell and OCaml are much less popular than Python, with consequences for community building, recruiting, reusing existing code bases etc. The popular statically typed languages which are actually Python's competitors are for example C++, C#, Java and the like. Sure as hell they are more verbose. Even Typescript was extremely intimidating to me when I started to try and understand Typescript code bases. And I am NOT going to try and teach Haskell or Ocaml as a first programming language...
- lerax 8y ago
  > I don’t think Haskell or OCaml are more verbose than Python. I love Haskell computing model, but holy hell, why is so difficult using things like State Monad or even a trivial thing like a global counter [1] ? Why something so simple should be so difficult? Don't say it's because functional programming, Scheme/Racket is not that difficult! Haskell and OCaml has their own problems, I agreed with you that both languages are more expressive but the code density is not necessary a good thing neither. [1]: https://github.com/scotty-web/scotty/blob/master/examples/globalstate.hs#L68 https://github.com/scotty-web/scotty/blob/master/examples/gl...

kristianov 8y ago

Wrestling with borrow checker is enough punishment already. /s

vonseel 8y ago

If one were to write perfectly type-annotated Python, and a hypothetical perfectly annotated standard library existed, what’s preventing a faster runtime being developed with performance similar to C#, for example? That will never happen, so maybe “what if” isn’t even worth asking

catern 8y ago
It's already being done: https://github.com/mypyc/mypyc https://github.com/mypyc/mypyc