10 ms·

Ugh... The type hierarchy of Python is painfully intertwined with the implementation of its type system. That implementation varies by Python implementation, an

Ugh... The type hierarchy of Python is painfully intertwined with the implementation of its type system. That implementation varies by Python implementation, and since the Python language is as old as Visual Basic, Borland Pascal, Werther's caramels, and trilobites, it varies dramatically over time. Add to that the fact that Python has a philosophy of wearing leaky abstractions proudly on its sleeve and barfing implementation-level details at its users. Python really is one of the most complex, arbitrary, and poorly abstracted languages in popular use. I have no idea how anyone lives with it for anything except writing extremely repetitive, formulaic, and superficial code, because having to dig into its guts feels like a nightmare. Even Java's type system, as antiquated and simplistic as it is, is easier to understand and work with by comparison.

orf 6y ago
Providing some specific details would make your argument a lot more convincing.
- legerdemain 6y ago
  Sure! In Python 2.x, "class" was an instance of "type." In Python 3.x, "type" is an instance of "class." (This is, of course, an over-simplification of a much more complicated reality.)
  - chrismorgan 6y ago
    Could you expand on that? I’m not sure what exactly you’re saying, because you’re not using standard or unambiguous terminology. (`type` is a thing, but “class” isn’t a thing, but rather a category of things, a syntactic construct.) Python 2: >>> class OldStyleClass: pass >>> isinstance(OldStyleClass, type) False >>> class NewStyleClass(object): pass >>> isinstance(NewStyleClass, type) True Python 3: >>> class Class: pass >>> isinstance(Class, type) True Old-style classes were kind of their own thing in their own space, but with new-style classes, classes are to `type` as instances are to `object`, and I don’t recall there being any major 2/3 difference beyond the simple removal of old-style classes.
    - legerdemain 6y ago
      Python2: >>> type(type) <type 'type'> Python3: >>> type(type) <class 'type'>
      - chrismorgan 6y ago
        Ah, gotcha. But I don’t think that that’s actually a difference—it’s just a slight terminology change around “type” and “class”, which I think may have been related to tidying up old-style classes. (That is: yeah, if you’re comparing old-style classes, it may have been a difference (I’m not certain), but I believe it’s completely superficial once you’re comparing the recommended form of classes for the last quite a few years of Python 2.)
        eesmith 6y ago
        I agree. I'm pretty sure the word "class" here comes from the repr for types, in Objects/typeobject.c : static PyObject * type_repr(PyTypeObject *type) { ... if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins)) rtn = PyUnicode_FromFormat("<class '%U.%U'>", mod, name); else rtn = PyUnicode_FromFormat("<class '%s'>", type->tp_name); ... which is used later on when defining the "type" type: PyTypeObject PyType_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "type", /* tp_name */ ... (reprfunc)type_repr, /* tp_repr */ That is, Python 3 has no actual "class" type, so it's not correct to say that '"type" is an instance of "class."' (I could be wrong - I'm no expert at the Python internals.)
      - siebenmann 6y ago
        The 'class' versus 'type' distinction here is actually arbitrary. The same C level code is responsible for the message that you get from 'type(whatever)' in both Python 2 and Python 3, but in Python 2 it drew a distinction between heap-allocated things (which were reported as 'class') and things that were not heap allocated (which were reported as 'type'). Non heap allocated things had to be created in C; heap allocated things were usually implemented in Python and were usually made with 'class X(base): ...'. (This change was introduced in Python 3.0a5, bug #2565. Looking at the bug, this is a followup of making the type() of new style classes be reported as 'class ...', but preserving old behavior of type() for built-ins, done in 2001.)
pbourke 6y ago
> I have no idea how anyone lives with it for anything except writing extremely repetitive, formulaic, and superficial code, because having to dig into its guts feels like a nightmare. It’s interesting that Python powers an outsized chunk of contemporary scientific research. Perhaps its fitness for purpose is a result of its messiness, or perhaps that messiness maps better to the way that many people think? Would a more rigorously-typed alternate-universe Python have been as successful?
- wyldfire 6y ago
  > Would a more rigorously-typed alternate-universe Python have been as successful? Yes, perhaps even moreso. It certainly would've been easier for alternative implementations like PyPy. Maybe even easier to make the transition to Python3 if the interface had been better designed. But, hey, with all its faults I still think it's really great and deserves a lot of credit. That said, it could be a good time for Nim or some other new language to shine.
- throwaway189262 6y ago
  IMO it's because researchers like it, because you can slap together some super sloppy hacky mess prototype and it will usually work. More germain languages stop you from from generating pure chaos. Researchers also love excel, Fortran, and Matlab. Let that sink in for a bit. Bring the downvotes hahaha
  - Demiurge 6y ago
    So, which is it, "will usually work" or "pure chaos"?
  - rednafi 6y ago
    This is true. I primarily write research code as well as some code that goes to prod. I love writing Python for exactly the reason you've mentioned. If something that needs to be re-written in a "proper" language, we've guys that can write that thing in them quickly and I don't need to worry about them that much. Did I mention those guys usually gets paid half as much as I do? As long as things work man.
mattbillenstein 6y ago
For all its criticisms, it's an unusually good language for just getting shit done. Sure, you can build whatever you want in almost any language you want, but 90% of those will be more verbose, harder to read, require compile steps, and be 10x the LOC, and take 4x longer to write... You can have it; I'll keep writing Python.
- FartyMcFarter 6y ago
  > unusually good language for just getting shit done. I agree, but only for very specific/small values of "shit". A codebase with thousands of lines of Python is IMHO already stretching the limits of readability; when you get to the tens/hundreds of thousands of lines, it's definitely past.
  - strokirk 6y ago
    What do you use instead? I've tried and used 10-15 programming languages in my days, and while I haven't used them all professionally Python is still by far my preferred language out of all that I've tried. TypeScript comes in at a close second, although it depends a lot on what you use it for of course. I've managed to write both buggy and working code in all languages, and can't say that Python is more buggy than any other. Slower at runtime, for sure. But not less correct.
    - lmm 6y ago
      Not the person you replied to, but I was a big Python fan until I found Scala. I try to write code that looks like Python (which means avoiding some libraries that rely too heavily on symbol-heavy names), but the type system can help me check that I really know the things I think I know, and even in small Python scripts there are conveniences that I miss (e.g. case classes - though nowadays attrs is more or less equivalent).
    - FartyMcFarter 6y ago
      Correctness is not necessarily the issue. The hard thing with large Python codebases is that the lack of types and data definitions makes it hard to understand what data any piece of code is operating on. This may not be an issue for a project written/read by a single person, but when working on big team projects it's a different story. Any statically-typed language will avoid this issue.
ketralnis 6y ago
Shrug, so don't use it. Nobody's making you
geofft 6y ago
> I have no idea how anyone lives with it for anything except writing extremely repetitive, formulaic, and superficial code, because having to dig into its guts feels like a nightmare. This is sort of why I like Python: it prioritizes getting something working today and trades off being able to build a complex system over several years. Java is a great language if you're going to build a large-scale application and want a bunch of developers across time and space to productively contribute to it. But business requirements change, and if you can get what you want done by using a series of applications that live for a few weeks or months, why wouldn't you do that? Another commenter mentioned scientific computing, which is a good example. When you're evaluating hypotheses, it's important to be able to write new code with high throughput. That means you need enough power of abstraction to be able to have libraries like NumPy etc., but you don't want so much structure that it's hard to write 10 lines of code and validate or falsify your gut feeling and then move on. Python was built around a REPL; Java only grew JShell fairly recently, and Jupyter was built for Python, not Java. And even when you're trying to reproduce old work, it's a lot more scientifically valuable to have someone be able to look at the paper (the spec) and write a new implementation than to say "Start with this 100 kloc Java codebase which we wrote for Java 1.2 but still works today." There are of course useful applications for that capability and it's great that Java supports it, but this isn't one of those applications. I don't do scientific Python myself, but I support users who do and I do a lot of infrastructure work. For that, it's still much more valuable to be able to write new code and get rid of it than to work within an old and well-built system. If I want to answer questions of, say, what our storage use patterns look like so the business can figure out what to invest in, it's a lot more valuable to be able to take that problem statement and produce an answer quickly than to build a long-lasting application that can keep producing answers to variants of that question, because the next question isn't likely to look too similar in terms of implementation. Or maybe put it this way - if you're the IRS, and you're writing code to process and verify people's tax returns, Java sounds great. You're going to want to keep the tax code from N years ago still working, and you're going to want this application to live many years. But for me, I have a little program to do the math for me to fill out my taxes, and that program is in Python, because I have no need to keep maintaining the script when the tax code changes (or when I move states), and I'm the only author. I make a copy of the Python file each year and I drop anything irrelevant. The ability for me to read the entire script top to bottom and be convinced "OK, this solves this one task accurately" is far more valuable than the program being able to solve hundreds of tasks it needed to solve in the past.
isbjorn16 6y ago
I always love coming into these Python threads. It's extremely cathartic to know there are enough people who hate the language as much as I do, even if we are stuck with it for any number of good reasons. All of the reasons we're stuck with it related to datascience and scientific computing are the exact reasons why we hate it. It's exceptionally fast to write code that does a very specific task without sparing a single solitary thought for any sort of software development best practices. It's incredibly approachable, which means it can be used by any keen scientific or mathematical minds who spend more time reading and writing whitepapers than they do writing code. The code, as an entity in and of itself, clearly shows that. But the _output_ of that code also shows that, which is why it isn't going to go away or be swapped out for anything else. This is like the ideal breeding ground for further scientific or academic pursuits, and since everyone is standing on the shoulders of numpy/scipy/sklearn/pytorch-esquian giants, you're stuck in the scenario of having to replicate all of these libraries in any other language. I can't honestly say it's worth the effort to replace the entire stack, and it's also not going to help when these same researchers continue to use python going forward and start using some other library that you didn't port yet. And so, they will continue pumping out new papers and prototypes-masquerading-as-quality-libraries, and we'll continue roping them in and duct taping fix after fix after fix on this kaleidoscope of horrors and we'll continue self medicating to dull the daily frustration of python while also accepting that it's the price we're going to need to pay if we want to work with these brilliant minds with all their neat new ideas pushing forth such a breadth of new understanding from the data _we already have_. Or at least that's what we say when we try to ignore the growing bald spot and bursts of incandescent rage every time we get a "quick script that needs a little polish" that clocks in at 2000 loc and literally doesn't have a single function definition in it and variable names that identify up to 14 different types depending on some global state and... You know, I'm going to stop typing this. It's christmas eve and I wasn't planning on being this angry today Edit: Some people weren't liking this, it may be because they think I'm being sarcastic when I talk about these brilliant people. I want to just clarify - I am not being sarcastic. They ARE brilliant. They're far more intelligent than I, and they're really pushing the boundaries of human understanding in mathematics and sciences. I acknowledge they're bring a whole hell more good to the table than they are bad, and there's a damn good reason why I am going to be doing this thing I'm doing in perpetuity, or until I can't handle it anymore. But that doesn't mean their coding practices aren't often _abysmal_, and I won't apologize for my frustrations in this and the language that just _lets_ them do it.
marmaduke 6y ago
The same arguments I used to make against MATLAB are used against Python on HN. Yet both are as or more more successful than your favorite language (from where I’m sitting).
brundolf 6y ago
I think to understand its success you have to compare it not to Java, but to Bash. Or maybe C.
zelly 6y ago
Python is basically pseudocode so if you don't like Python then you don't like programming.

orf 6y ago

Providing some specific details would make your argument a lot more convincing.

legerdemain 6y ago
Sure! In Python 2.x, "class" was an instance of "type." In Python 3.x, "type" is an instance of "class." (This is, of course, an over-simplification of a much more complicated reality.)
- chrismorgan 6y ago
  Could you expand on that? I’m not sure what exactly you’re saying, because you’re not using standard or unambiguous terminology. (`type` is a thing, but “class” isn’t a thing, but rather a category of things, a syntactic construct.) Python 2: >>> class OldStyleClass: pass >>> isinstance(OldStyleClass, type) False >>> class NewStyleClass(object): pass >>> isinstance(NewStyleClass, type) True Python 3: >>> class Class: pass >>> isinstance(Class, type) True Old-style classes were kind of their own thing in their own space, but with new-style classes, classes are to `type` as instances are to `object`, and I don’t recall there being any major 2/3 difference beyond the simple removal of old-style classes.
  - legerdemain 6y ago
    Python2: >>> type(type) <type 'type'> Python3: >>> type(type) <class 'type'>
    - chrismorgan 6y ago
      Ah, gotcha. But I don’t think that that’s actually a difference—it’s just a slight terminology change around “type” and “class”, which I think may have been related to tidying up old-style classes. (That is: yeah, if you’re comparing old-style classes, it may have been a difference (I’m not certain), but I believe it’s completely superficial once you’re comparing the recommended form of classes for the last quite a few years of Python 2.)
      - eesmith 6y ago
        I agree. I'm pretty sure the word "class" here comes from the repr for types, in Objects/typeobject.c : static PyObject * type_repr(PyTypeObject *type) { ... if (mod != NULL && !_PyUnicode_EqualToASCIIId(mod, &PyId_builtins)) rtn = PyUnicode_FromFormat("<class '%U.%U'>", mod, name); else rtn = PyUnicode_FromFormat("<class '%s'>", type->tp_name); ... which is used later on when defining the "type" type: PyTypeObject PyType_Type = { PyVarObject_HEAD_INIT(&PyType_Type, 0) "type", /* tp_name */ ... (reprfunc)type_repr, /* tp_repr */ That is, Python 3 has no actual "class" type, so it's not correct to say that '"type" is an instance of "class."' (I could be wrong - I'm no expert at the Python internals.)
    - siebenmann 6y ago
      The 'class' versus 'type' distinction here is actually arbitrary. The same C level code is responsible for the message that you get from 'type(whatever)' in both Python 2 and Python 3, but in Python 2 it drew a distinction between heap-allocated things (which were reported as 'class') and things that were not heap allocated (which were reported as 'type'). Non heap allocated things had to be created in C; heap allocated things were usually implemented in Python and were usually made with 'class X(base): ...'. (This change was introduced in Python 3.0a5, bug #2565. Looking at the bug, this is a followup of making the type() of new style classes be reported as 'class ...', but preserving old behavior of type() for built-ins, done in 2001.)

pbourke 6y ago

> I have no idea how anyone lives with it for anything except writing extremely repetitive, formulaic, and superficial code, because having to dig into its guts feels like a nightmare. It’s interesting that Python powers an outsized chunk of contemporary scientific research. Perhaps its fitness for purpose is a result of its messiness, or perhaps that messiness maps better to the way that many people think? Would a more rigorously-typed alternate-universe Python have been as successful?

wyldfire 6y ago
> Would a more rigorously-typed alternate-universe Python have been as successful? Yes, perhaps even moreso. It certainly would've been easier for alternative implementations like PyPy. Maybe even easier to make the transition to Python3 if the interface had been better designed. But, hey, with all its faults I still think it's really great and deserves a lot of credit. That said, it could be a good time for Nim or some other new language to shine.
throwaway189262 6y ago
IMO it's because researchers like it, because you can slap together some super sloppy hacky mess prototype and it will usually work. More germain languages stop you from from generating pure chaos. Researchers also love excel, Fortran, and Matlab. Let that sink in for a bit. Bring the downvotes hahaha
- Demiurge 6y ago
  So, which is it, "will usually work" or "pure chaos"?
- rednafi 6y ago
  This is true. I primarily write research code as well as some code that goes to prod. I love writing Python for exactly the reason you've mentioned. If something that needs to be re-written in a "proper" language, we've guys that can write that thing in them quickly and I don't need to worry about them that much. Did I mention those guys usually gets paid half as much as I do? As long as things work man.

mattbillenstein 6y ago

For all its criticisms, it's an unusually good language for just getting shit done. Sure, you can build whatever you want in almost any language you want, but 90% of those will be more verbose, harder to read, require compile steps, and be 10x the LOC, and take 4x longer to write... You can have it; I'll keep writing Python.

FartyMcFarter 6y ago
> unusually good language for just getting shit done. I agree, but only for very specific/small values of "shit". A codebase with thousands of lines of Python is IMHO already stretching the limits of readability; when you get to the tens/hundreds of thousands of lines, it's definitely past.
- strokirk 6y ago
  What do you use instead? I've tried and used 10-15 programming languages in my days, and while I haven't used them all professionally Python is still by far my preferred language out of all that I've tried. TypeScript comes in at a close second, although it depends a lot on what you use it for of course. I've managed to write both buggy and working code in all languages, and can't say that Python is more buggy than any other. Slower at runtime, for sure. But not less correct.
  - lmm 6y ago
    Not the person you replied to, but I was a big Python fan until I found Scala. I try to write code that looks like Python (which means avoiding some libraries that rely too heavily on symbol-heavy names), but the type system can help me check that I really know the things I think I know, and even in small Python scripts there are conveniences that I miss (e.g. case classes - though nowadays attrs is more or less equivalent).
  - FartyMcFarter 6y ago
    Correctness is not necessarily the issue. The hard thing with large Python codebases is that the lack of types and data definitions makes it hard to understand what data any piece of code is operating on. This may not be an issue for a project written/read by a single person, but when working on big team projects it's a different story. Any statically-typed language will avoid this issue.

ketralnis 6y ago

Shrug, so don't use it. Nobody's making you

geofft 6y ago

> I have no idea how anyone lives with it for anything except writing extremely repetitive, formulaic, and superficial code, because having to dig into its guts feels like a nightmare. This is sort of why I like Python: it prioritizes getting something working today and trades off being able to build a complex system over several years. Java is a great language if you're going to build a large-scale application and want a bunch of developers across time and space to productively contribute to it. But business requirements change, and if you can get what you want done by using a series of applications that live for a few weeks or months, why wouldn't you do that? Another commenter mentioned scientific computing, which is a good example. When you're evaluating hypotheses, it's important to be able to write new code with high throughput. That means you need enough power of abstraction to be able to have libraries like NumPy etc., but you don't want so much structure that it's hard to write 10 lines of code and validate or falsify your gut feeling and then move on. Python was built around a REPL; Java only grew JShell fairly recently, and Jupyter was built for Python, not Java. And even when you're trying to reproduce old work, it's a lot more scientifically valuable to have someone be able to look at the paper (the spec) and write a new implementation than to say "Start with this 100 kloc Java codebase which we wrote for Java 1.2 but still works today." There are of course useful applications for that capability and it's great that Java supports it, but this isn't one of those applications. I don't do scientific Python myself, but I support users who do and I do a lot of infrastructure work. For that, it's still much more valuable to be able to write new code and get rid of it than to work within an old and well-built system. If I want to answer questions of, say, what our storage use patterns look like so the business can figure out what to invest in, it's a lot more valuable to be able to take that problem statement and produce an answer quickly than to build a long-lasting application that can keep producing answers to variants of that question, because the next question isn't likely to look too similar in terms of implementation. Or maybe put it this way - if you're the IRS, and you're writing code to process and verify people's tax returns, Java sounds great. You're going to want to keep the tax code from N years ago still working, and you're going to want this application to live many years. But for me, I have a little program to do the math for me to fill out my taxes, and that program is in Python, because I have no need to keep maintaining the script when the tax code changes (or when I move states), and I'm the only author. I make a copy of the Python file each year and I drop anything irrelevant. The ability for me to read the entire script top to bottom and be convinced "OK, this solves this one task accurately" is far more valuable than the program being able to solve hundreds of tasks it needed to solve in the past.

isbjorn16 6y ago

I always love coming into these Python threads. It's extremely cathartic to know there are enough people who hate the language as much as I do, even if we are stuck with it for any number of good reasons. All of the reasons we're stuck with it related to datascience and scientific computing are the exact reasons why we hate it. It's exceptionally fast to write code that does a very specific task without sparing a single solitary thought for any sort of software development best practices. It's incredibly approachable, which means it can be used by any keen scientific or mathematical minds who spend more time reading and writing whitepapers than they do writing code. The code, as an entity in and of itself, clearly shows that. But the _output_ of that code also shows that, which is why it isn't going to go away or be swapped out for anything else. This is like the ideal breeding ground for further scientific or academic pursuits, and since everyone is standing on the shoulders of numpy/scipy/sklearn/pytorch-esquian giants, you're stuck in the scenario of having to replicate all of these libraries in any other language. I can't honestly say it's worth the effort to replace the entire stack, and it's also not going to help when these same researchers continue to use python going forward and start using some other library that you didn't port yet. And so, they will continue pumping out new papers and prototypes-masquerading-as-quality-libraries, and we'll continue roping them in and duct taping fix after fix after fix on this kaleidoscope of horrors and we'll continue self medicating to dull the daily frustration of python while also accepting that it's the price we're going to need to pay if we want to work with these brilliant minds with all their neat new ideas pushing forth such a breadth of new understanding from the data _we already have_. Or at least that's what we say when we try to ignore the growing bald spot and bursts of incandescent rage every time we get a "quick script that needs a little polish" that clocks in at 2000 loc and literally doesn't have a single function definition in it and variable names that identify up to 14 different types depending on some global state and... You know, I'm going to stop typing this. It's christmas eve and I wasn't planning on being this angry today Edit: Some people weren't liking this, it may be because they think I'm being sarcastic when I talk about these brilliant people. I want to just clarify - I am not being sarcastic. They ARE brilliant. They're far more intelligent than I, and they're really pushing the boundaries of human understanding in mathematics and sciences. I acknowledge they're bring a whole hell more good to the table than they are bad, and there's a damn good reason why I am going to be doing this thing I'm doing in perpetuity, or until I can't handle it anymore. But that doesn't mean their coding practices aren't often _abysmal_, and I won't apologize for my frustrations in this and the language that just _lets_ them do it.

marmaduke 6y ago

The same arguments I used to make against MATLAB are used against Python on HN. Yet both are as or more more successful than your favorite language (from where I’m sitting).

brundolf 6y ago

I think to understand its success you have to compare it not to Java, but to Bash. Or maybe C.

zelly 6y ago

Python is basically pseudocode so if you don't like Python then you don't like programming.