4 ms·

I feel like this needs to be mentioned more: If you don't have some kind of system to enforce types, then TypedDict does nothing at all. You can store a float

I feel like this needs to be mentioned more: If you don't have some kind of system to enforce types, then TypedDict does nothing at all. You can store a float in an attribute annotated as a string, and default Python will not stop you, or display any kind of warning. The typing is purely for development, and does nothing once compiled. If you want typing to actually be enforced, you need to use something like Pydantic.

MrJohz 2y ago
Counter argument: this is actually a myth that gets repeated far too often, and misses the point of what static types do. If your input data is the correct type (big if #1), and if your types are sounds and consistent with each other (big if #2), then you don't need something like Pydantic, because the types guarantee that there will never be a float stored somewhere that's annotated as a string. It cannot happen, because as soon as you try to store a float in a string attribute, your type checker (Pyright, Mypy, etc) will complain. And if you're consistently running that type checker over your code, e.g. in a CI job that runs over your codebase for every push, then you can never have checked-in code where the types are incorrect. There are the two big caveats above, but these turn out not to be such big deals. Caveat #1 is that you can't rely on external data to match the types you've described internally. This is an ideal use case for Pydantic, like you say: you check external types once, at the boundaries of your program, and then internally you can be confident that the types will always be correct. Caveat #2 is that you don't break the type checker's limits. This generally means avoiding things like `Any`, and ensuring that your typed code always calls other typed code, and never untyped, un-annotated code. The easiest way to do this is to start by typing the leaf files in your program (the ones that only get imported, and never import anything in turn). These can't import untyped code, so if they're fully typed and the type checker passes, then you can be confident that their types are correct. Now sure, a bad caller could try and call one of these functions with invalid input, but as they say, Python is for adults, and you're free to call a function with the wrong arguments, and you're free to deal with the results. Now, you can add types to code that only imports typed code, and because you've checked the imported code you know it's correct, so if you also check the importer and it's also correct, then you can be confident at that level too. You can keep going until you've added types to the entire codebase, and now you can be confident that the entire codebase always passes the correct types, and therefore that no runtime enforcement is necessary. There is a third caveat, which is that this relies on having a powerful enough type system to model all the things that you were originally doing in Python's dynamic type system. This is hard, because you can do some pretty wild things in Python, and my own experience with Python type checkers has been a bit disappointing - they handle more boilerplate-heavy code fairly well, but seem to struggle on more idiomatic code that uses Python's dynamic features. However, that seems to be improving all the time, and Typescript demonstrates that it's certainly possible to model a dynamic language with static types to a very reasonable accuracy.
- vrighter 2y ago
  the lsp will complain about i~. The language happily does it anyway.
  - MrJohz 2y ago
    Which is why you use a type checker to ensure that it never happens in the codebase. If there is no line of code that ever does something like int_field: int = "string" or self.int_field = "string" Then you can be confident that the language will never store a string value in that integer field. And if you expand this to cover all fields and variables in your codebase, you can see that, if you use the type system correctly, the language won't violate those constraints. There are some exceptions to this, like I said in the previous comment, but I think when getting started with typed Python, it's easier to assume that the types will be valid, because that's the case almost all of the time. Usually, overriding type assertions requires some kind of explicit cast or statement, and this is usually documented.
    - vrighter 2y ago
      another way to accomplish this, and also eliminate any and all possibilities of human error, is to make the type checker part of the language. You're still doing the same thing, just with extra steps and implicit rules
- calibas 2y ago
  What exactly is the "myth" that I'm promoting? Type annotations do nothing in default Python, they require some other system to enforce them.

MrJohz 2y ago

Counter argument: this is actually a myth that gets repeated far too often, and misses the point of what static types do. If your input data is the correct type (big if #1), and if your types are sounds and consistent with each other (big if #2), then you don't need something like Pydantic, because the types guarantee that there will never be a float stored somewhere that's annotated as a string. It cannot happen, because as soon as you try to store a float in a string attribute, your type checker (Pyright, Mypy, etc) will complain. And if you're consistently running that type checker over your code, e.g. in a CI job that runs over your codebase for every push, then you can never have checked-in code where the types are incorrect. There are the two big caveats above, but these turn out not to be such big deals. Caveat #1 is that you can't rely on external data to match the types you've described internally. This is an ideal use case for Pydantic, like you say: you check external types once, at the boundaries of your program, and then internally you can be confident that the types will always be correct. Caveat #2 is that you don't break the type checker's limits. This generally means avoiding things like `Any`, and ensuring that your typed code always calls other typed code, and never untyped, un-annotated code. The easiest way to do this is to start by typing the leaf files in your program (the ones that only get imported, and never import anything in turn). These can't import untyped code, so if they're fully typed and the type checker passes, then you can be confident that their types are correct. Now sure, a bad caller could try and call one of these functions with invalid input, but as they say, Python is for adults, and you're free to call a function with the wrong arguments, and you're free to deal with the results. Now, you can add types to code that only imports typed code, and because you've checked the imported code you know it's correct, so if you also check the importer and it's also correct, then you can be confident at that level too. You can keep going until you've added types to the entire codebase, and now you can be confident that the entire codebase always passes the correct types, and therefore that no runtime enforcement is necessary. There is a third caveat, which is that this relies on having a powerful enough type system to model all the things that you were originally doing in Python's dynamic type system. This is hard, because you can do some pretty wild things in Python, and my own experience with Python type checkers has been a bit disappointing - they handle more boilerplate-heavy code fairly well, but seem to struggle on more idiomatic code that uses Python's dynamic features. However, that seems to be improving all the time, and Typescript demonstrates that it's certainly possible to model a dynamic language with static types to a very reasonable accuracy.

vrighter 2y ago
the lsp will complain about i~. The language happily does it anyway.
- MrJohz 2y ago
  Which is why you use a type checker to ensure that it never happens in the codebase. If there is no line of code that ever does something like int_field: int = "string" or self.int_field = "string" Then you can be confident that the language will never store a string value in that integer field. And if you expand this to cover all fields and variables in your codebase, you can see that, if you use the type system correctly, the language won't violate those constraints. There are some exceptions to this, like I said in the previous comment, but I think when getting started with typed Python, it's easier to assume that the types will be valid, because that's the case almost all of the time. Usually, overriding type assertions requires some kind of explicit cast or statement, and this is usually documented.
  - vrighter 2y ago
    another way to accomplish this, and also eliminate any and all possibilities of human error, is to make the type checker part of the language. You're still doing the same thing, just with extra steps and implicit rules
calibas 2y ago
What exactly is the "myth" that I'm promoting? Type annotations do nothing in default Python, they require some other system to enforce them.