4 ms·

It fundamentally can't be improved in a significant enough way and still be Python. The more realistic options are to be okay with Python's type system (I perso

It fundamentally can't be improved in a significant enough way and still be Python. The more realistic options are to be okay with Python's type system (I personally am) or to look at different languages if you really want something more robust.

6gvONxR4sf7o 6y ago
I don't think that's accurate. Python's type system is largely defined by mypy + the annotations, in addition to runtime python. In that regard, python's type system has been evolving really quickly, and it's becoming something quite interesting. I'm a fanboy of static type systems, and in the last bunch of years, python's type system has gone from just bearable to really useful. I'm excited to see where it goes from here.
- skohan 6y ago
  But to some extent isn't it always going to be bolted-on and dependent on library authors to which extent it's consistent?
  - 6gvONxR4sf7o 6y ago
    The dependence on library authors is always a challenge in any language. You might have one author using `[a]` where another uses `PositiveNumeric a, Fin n => NonEmptyList n a` for the same thing. You can always just annotate whatever the library author used (e.g. they return a list of strings, so you use List[str]). There are some interesting further add ons that seem very python, allowing you to go further. For example, with a pandas dataframe you can just say your type is a dataframe which isn't so useful, but it's possible to hack your own types onto it in the vein of https://github.com/CedricFR/dataenforce https://github.com/CedricFR/dataenforce, or use things like https://smarie.github.io/python-vtypes/ https://smarie.github.io/python-vtypes/ to get smarter typing on things the authors didn't type. I expect that trend will continue. What fascinates me about python's types is actually the very fact that they are bolted on. You have a language that lets you do crazy things and a type system trying to catch up and make it convenient to verify those crazy things. It's a nice complement to the usual developments of verifying all of the things and slowly extending the set of things you can do.
    - lmm 6y ago
      > You might have one author using `[a]` where another uses `PositiveNumeric a, Fin n => NonEmptyList n a` for the same thing. You can always just annotate whatever the library author used (e.g. they return a list of strings, so you use List[str]). The difference between these two cases is that a function that returns [a] will always return [a] and if you convert it to `NonEmptyList n a` you will be nudged towards handling the case where it didn't actually return n elements, whereas if you annotate someone else's list as returning List[str] then it may well silently not do some of the time, and you won't get an error until a long way away.
      - 6gvONxR4sf7o 6y ago
        I was unclear. I meant the other way around. In the case that the third party code does in fact always return no empty lists of positive numbers, the author might just type it as a list. A third party author in even a language like Haskell won’t necessarily use the bells and whistles of giving very specific types. And by the List[str] bit, I mean you can annotate it as doing whatever it in fact does. You aren’t protected if you get that wrong, of course, and that’s where things like stubs come in to help.
        lmm 6y ago
        > In the case that the third party code does in fact always return no empty lists of positive numbers, the author might just type it as a list. Yes - but as I said, you can be confident that the list type is accurate, and if you want to convert that list to a specific-length list type then you're nudged towards doing it in a way where you handle the case where the list isn't actually the length you specified. > And by the List[str] bit, I mean you can annotate it as doing whatever it in fact does. You aren’t protected if you get that wrong, of course Not "of course"; the whole point of a type system is that it's automatically checked and therefore you can have reasonable confidence in the types you're given.
        6gvONxR4sf7o 6y ago
        > You aren’t protected if you get that wrong, of course, and that’s where things like stubs come in to help.
  - girvo 6y ago
    For what it’s worth, TypeScript had somewhat of the same problem, and yet these days it’s rare for me to find a well known library that doesn’t include type definitions!
    - FridgeSeal 6y ago
      Yeah the JS community’s adoption of typing seems to be far more enthusiastic than the Python communitys’
simias 6y ago
I agree with you, although I must say that as an occasional Python user the way type hints are implemented absolutely baffles me. In particular the fact that they are completely ignored by the standard implementation. They're glorified comments. Given how opinionated the Python maintainers can be, it baffles me that they accepted to get these optional, noisy, half backed type hints into the core language. In my experience given that they're optional and you'll almost never get 100% of your code and its dependencies with correct and up to date signatures it's just a nuisance. Maybe in the right projects if all the devs are very thorough with them it can be helpful, but that's really not my experience. If at least it triggered an assertion at runtime when the type doesn't match it would be massively more useful. And even then, if you're so thorough with your typing, why not just use a proper statically typed language? I really don't get why it's even there to be honest.
- mumblemumble 6y ago
  > the fact that they are completely ignored by the standard implementation. They're glorified comments. Not quite. You can reflect on and interact with them at run-time, too. This does make it possible to implement run-time checking as a library solution. I can't speak to why run-time checking wasn't built into the language as a default behavior, and the PEP doesn't explain why (though perhaps the answer is in the mailing list), but one possibility is that it would be wasteful. Most functions that are only prepared to accept certain types already have some form of run-time type check baked in, either explicitly or incidentally, so adding a second check would likely introduce overhead without producing a whole lot more type safety.
  - uryga 6y ago
    > I can't speak to why run-time checking wasn't built into the language as a default behavior tldr: you other get either huge overhead or need to type annotate everything like it's Java, and it'll still be pretty limited/not super useful --- there's a bunch of libraries that do this, and they're all fundamentally limited [though i guess some of these may just be failures of imagination on my part]: - hard to check generics: if it's a builtin, you have to traverse the whole structure and check every element. and for user-defined collections, the programmer has to to manually instantiate every generic like: CoolSequence[int]([1,2,3]) (which sucks) and every method call/property access needs to be checked - if your function takes a callback, there's no way to typecheck that callback at runtime (without calling it, at which point it's not terribly useful. which extends to all functions i guess) - not much you can do with iterators/generators. can't check if something really is an `Iterator[int]` without exhausting it, and e.g. type-erroring after processing 20 elements because it yielded not-an-int isn't very useful so in the end, even if you wrap Every Damn Thing in a type assertion, you still get the errors very late, not much sooner than your usual "AttributeError: object None has no property named 'foobar'". kind of a worst-of-both-worlds situation
dragonwriter 6y ago
> It fundamentally can't be improved in a significant enough way and still be Python. Why not? Static enforcement of invariants that doesn't change the space of what is possible in the language when you aren't asserting conflicting invariants doesn't make Python any less Python.

6gvONxR4sf7o 6y ago

I don't think that's accurate. Python's type system is largely defined by mypy + the annotations, in addition to runtime python. In that regard, python's type system has been evolving really quickly, and it's becoming something quite interesting. I'm a fanboy of static type systems, and in the last bunch of years, python's type system has gone from just bearable to really useful. I'm excited to see where it goes from here.

skohan 6y ago
But to some extent isn't it always going to be bolted-on and dependent on library authors to which extent it's consistent?
- 6gvONxR4sf7o 6y ago
  The dependence on library authors is always a challenge in any language. You might have one author using `[a]` where another uses `PositiveNumeric a, Fin n => NonEmptyList n a` for the same thing. You can always just annotate whatever the library author used (e.g. they return a list of strings, so you use List[str]). There are some interesting further add ons that seem very python, allowing you to go further. For example, with a pandas dataframe you can just say your type is a dataframe which isn't so useful, but it's possible to hack your own types onto it in the vein of https://github.com/CedricFR/dataenforce https://github.com/CedricFR/dataenforce, or use things like https://smarie.github.io/python-vtypes/ https://smarie.github.io/python-vtypes/ to get smarter typing on things the authors didn't type. I expect that trend will continue. What fascinates me about python's types is actually the very fact that they are bolted on. You have a language that lets you do crazy things and a type system trying to catch up and make it convenient to verify those crazy things. It's a nice complement to the usual developments of verifying all of the things and slowly extending the set of things you can do.
  - lmm 6y ago
    > You might have one author using `[a]` where another uses `PositiveNumeric a, Fin n => NonEmptyList n a` for the same thing. You can always just annotate whatever the library author used (e.g. they return a list of strings, so you use List[str]). The difference between these two cases is that a function that returns [a] will always return [a] and if you convert it to `NonEmptyList n a` you will be nudged towards handling the case where it didn't actually return n elements, whereas if you annotate someone else's list as returning List[str] then it may well silently not do some of the time, and you won't get an error until a long way away.
    - 6gvONxR4sf7o 6y ago
      I was unclear. I meant the other way around. In the case that the third party code does in fact always return no empty lists of positive numbers, the author might just type it as a list. A third party author in even a language like Haskell won’t necessarily use the bells and whistles of giving very specific types. And by the List[str] bit, I mean you can annotate it as doing whatever it in fact does. You aren’t protected if you get that wrong, of course, and that’s where things like stubs come in to help.
      - lmm 6y ago
        > In the case that the third party code does in fact always return no empty lists of positive numbers, the author might just type it as a list. Yes - but as I said, you can be confident that the list type is accurate, and if you want to convert that list to a specific-length list type then you're nudged towards doing it in a way where you handle the case where the list isn't actually the length you specified. > And by the List[str] bit, I mean you can annotate it as doing whatever it in fact does. You aren’t protected if you get that wrong, of course Not "of course"; the whole point of a type system is that it's automatically checked and therefore you can have reasonable confidence in the types you're given.
        6gvONxR4sf7o 6y ago
        > You aren’t protected if you get that wrong, of course, and that’s where things like stubs come in to help.
- girvo 6y ago
  For what it’s worth, TypeScript had somewhat of the same problem, and yet these days it’s rare for me to find a well known library that doesn’t include type definitions!
  - FridgeSeal 6y ago
    Yeah the JS community’s adoption of typing seems to be far more enthusiastic than the Python communitys’

simias 6y ago

I agree with you, although I must say that as an occasional Python user the way type hints are implemented absolutely baffles me. In particular the fact that they are completely ignored by the standard implementation. They're glorified comments. Given how opinionated the Python maintainers can be, it baffles me that they accepted to get these optional, noisy, half backed type hints into the core language. In my experience given that they're optional and you'll almost never get 100% of your code and its dependencies with correct and up to date signatures it's just a nuisance. Maybe in the right projects if all the devs are very thorough with them it can be helpful, but that's really not my experience. If at least it triggered an assertion at runtime when the type doesn't match it would be massively more useful. And even then, if you're so thorough with your typing, why not just use a proper statically typed language? I really don't get why it's even there to be honest.

mumblemumble 6y ago
> the fact that they are completely ignored by the standard implementation. They're glorified comments. Not quite. You can reflect on and interact with them at run-time, too. This does make it possible to implement run-time checking as a library solution. I can't speak to why run-time checking wasn't built into the language as a default behavior, and the PEP doesn't explain why (though perhaps the answer is in the mailing list), but one possibility is that it would be wasteful. Most functions that are only prepared to accept certain types already have some form of run-time type check baked in, either explicitly or incidentally, so adding a second check would likely introduce overhead without producing a whole lot more type safety.
- uryga 6y ago
  > I can't speak to why run-time checking wasn't built into the language as a default behavior tldr: you other get either huge overhead or need to type annotate everything like it's Java, and it'll still be pretty limited/not super useful --- there's a bunch of libraries that do this, and they're all fundamentally limited [though i guess some of these may just be failures of imagination on my part]: - hard to check generics: if it's a builtin, you have to traverse the whole structure and check every element. and for user-defined collections, the programmer has to to manually instantiate every generic like: CoolSequence[int]([1,2,3]) (which sucks) and every method call/property access needs to be checked - if your function takes a callback, there's no way to typecheck that callback at runtime (without calling it, at which point it's not terribly useful. which extends to all functions i guess) - not much you can do with iterators/generators. can't check if something really is an `Iterator[int]` without exhausting it, and e.g. type-erroring after processing 20 elements because it yielded not-an-int isn't very useful so in the end, even if you wrap Every Damn Thing in a type assertion, you still get the errors very late, not much sooner than your usual "AttributeError: object None has no property named 'foobar'". kind of a worst-of-both-worlds situation

dragonwriter 6y ago

> It fundamentally can't be improved in a significant enough way and still be Python. Why not? Static enforcement of invariants that doesn't change the space of what is possible in the language when you aren't asserting conflicting invariants doesn't make Python any less Python.