5 ms·

Python type annotations are more useful for a human as a form of documentation, than they ever will be to a machine. I think Python completely ignores them at r

Python type annotations are more useful for a human as a form of documentation, than they ever will be to a machine. I think Python completely ignores them at runtime, and tools like pytype or mypy will never capture all the complicated hacks possible in the Python type system (as evidenced by only 15% of repos passing mypy)

IshKebab 5y ago
This is only because the Python type hinting ecosystem is so bad. Tons of projects don't have type hints, there are multiple conflicting interpretations of the types, MyPy is very buggy (Pyright is much much better but few people use it). Compare it to Typescript which is a similar sort of tacked on type hinting system - Typescript is basically always machine checked.
goodoldneon 5y ago
Python does ignore them at runtime, but as long as you're validating your inputs (e.g. using Pydantic for HTTP request validation) then you can trust that the static types are truthful
- musingsole 5y ago
  > trust that the static types are truthful That's often a mistake even in typed languages like typescript. Satisfying a static type analyzer isn't enough. Real data is whatever it is going to be. If your types are meaningless at runtime, you can easily encounter type mismatches. Static typing as implemented in typescript and python type hints is a tool for engineers instead of a tool for systems.
  - roywiggins 5y ago
    The types are usually ignored, but not entirely erased- Pydantic uses type hints to do validation at runtime, so if you construct a Pydantic object with invalid data it will throw an error, so eg foo=PydanticFoo(**request.json()) Will try and validate a json request against the implied schema provided by the type hints on PydanticFoo while constructing it and will throw if it fails.
  - goodoldneon 5y ago
    You missed my point. If you validate input, then the static types are representative of runtime types
    - musingsole 5y ago
      Ack, so I did :P
- lelandbatey 5y ago
  I want to emphasize for folks reading: if you want actual type enforcement and you want that type enforcement with minimum boilerplate, Pydantic is a fantastic option (I'm sure there are others as well). It's a great library that's totally use-agnostic (it's not HTTP specific, you can use it for whatever) and quite tight. Caveats are that it uses runtime validation to enforce types so there's a performance cost, but it's a performance hit that is mostly eclipsed by all the other huge performance hits that litter your typical Python implementation. Pydantic is amazing!
  - mixmastamyk 5y ago
    IIRC, pydantic was quite big and I chose typeguard instead for a smaller project. Might be an option.
deltaonefour 5y ago
If you make it part of the deploy / build process and force programers to only use the subset of features supported by a type checker then you can benefit.
fantod 5y ago
Exactly. When I initially started using typed Python, I would write code as though I were using a statically-typed language. In order to appease mypy, I found myself increasingly needing to either rewrite my code in awkward ways or add explicit ignore comments. What happened is I started losing out on the advantages that Python provides as a dynamically-typed language without truly gaining those of statically-typed ones. Then I realized, type hints are just that: hints. They're a form of documentation. In some cases, they're very helpful, but in other cases, it doesn't really make sense to use them. I don't need to appease mypy. The type hints are their for the benefit of myself and other developers.
- throwra620 5y ago
  > rewrite my code in awkward ways or add explicit ignore comments. Probably just used to writing poor quality (immediately unintelligible to someone other than the author but functional) code then.
  - ReflectedImage 5y ago
    Using typing in Python is a code smell. Lack of understanding of what the language is for and how to use it.
- 4kelly 5y ago
  There is definitely truth to this statement. However often when I write something that’s hard to type, it’s a code smell. Ex. A complex nested map should probably be represented by a few concrete classes/abstractions. (Similar to when you something you wrote is hard to test, it’s probably just poorly abstracted code.)
  - fantod 5y ago
    I agree and I think a good thing about the type hints is they help uncover these kinds of code smells. But there are definitely cases where it's definitely not worth the effort to fight mypy. One example is that variable re-assignment can cause type errors in mypy but re-assigning a variable to a value of a different type is an extremely common and reasonable thing to do in Python. Another situation where strict typing can be borderline hopeless is when parsing e.g. very dynamic content like json. Sure you could encode every possible situation using types but this would basically defeat the purpose of using Python for such a task.
goodside 5y ago
Type annotations are used at runtime in some cases, most notably when defining dataclasses.
- musingsole 5y ago
  I find this a bit of a wart on the dataclass implementation. Type hints (to me) are more like a comment than a statement. So, from my perspective, dataclasses feel like they're pulling data out of nearby comments. Ick. But it's a wart worth the conciseness, at least, until a better idea comes along. Maybe if type hints could be optionally enforced as a language feature, I'd feel it was more integrated.
mic47 5y ago
I've used python typing heavily at work and was really useful to detect bugs and helped with developement. > as evidenced by only 15% of repos passing mypy That is no evidence. Of course, if you don't have mypy in CI (or pre-commit hooks or whatever), your repo will not pass mypy. But if you have it in CI, then you can rely on them being correct. > tools like pytype or mypy will never capture all the complicated hacks possible But they can either enforce everything being correctly typed (if you go to really strict settings), or yes, you will have some code untyped, so you will be careful around that part. It's not like all or nothing.
whimsicalism 5y ago
Most static typed compiled languages also ignore types at runtime :)
- munificent 5y ago
  Not true. Java, C#, Scala, Kotlin, Swift, Dart, and others all keep around type information at runtime so that casts can throw exceptions at runtime when the value isn't of the right type. Otherwise, soundness is lost. In practice, most modern statically typed languages maintain soundness through a combination of compile time and runtime checking.
  - whimsicalism 5y ago
    > most modern statically typed languages maintain soundness through a combination of compile time and runtime checking. Rust, C, C++ - all do not keep around this information. You're right that the JVM languages (and thus, naturally, C#) do it differently. I think it is a bit silly to name out each of the JVM languages separately.
    - pharmakom 5y ago
      C++ community is split by the RTTI compiler flags
    - zozbot234 5y ago
      Rust standard library includes RTTI support, though it's very rarely used.
    - munificent 5y ago
      Haskell, Go, and I think OCaml also rely on some amount of runtime checking for soundness. I think basically every managed statically typed language ends up carrying around runtime representation and uses some amount of runtime checks.
      - hither_shores 5y ago
throwaway894345 5y ago
> Python type annotations are more useful for a human as a form of documentation, than they ever will be to a machine I mean, machines serve people (for now!). So in addition to type documentation, type annotations are useful for things like "jump to definition" and "find references" and automatic refactoring. Moreover, the utility of annotations for type documentation depends on the correctness of those type annotations--if you don't have something like Mypy, then those inevitably annotations grow outdated over time and incorrect type annotations are even worse than none at all. > tools like pytype or mypy will never capture all the complicated hacks possible in the Python type system Technically you can represent any complicated hack if only via `any`, but that's just pedantry and we all know what you mean. Even still, it's not like "well you can't represent everything, so there's no point in trying to appease mypy" (which may not be your intended implication--I can't tell). The benefits of appeasing mypy are proportional to the amount of your code that is properly annotated--this is the thesis of gradual typing. > as evidenced by only 15% of repos passing mypy This very likely indicates that 85% of repos aren't running mypy regularly, or they're in some transition state. Certainly more than 15% of real-world Python can be described via Python type annotations.

IshKebab 5y ago

This is only because the Python type hinting ecosystem is so bad. Tons of projects don't have type hints, there are multiple conflicting interpretations of the types, MyPy is very buggy (Pyright is much much better but few people use it). Compare it to Typescript which is a similar sort of tacked on type hinting system - Typescript is basically always machine checked.

goodoldneon 5y ago

Python does ignore them at runtime, but as long as you're validating your inputs (e.g. using Pydantic for HTTP request validation) then you can trust that the static types are truthful

musingsole 5y ago
> trust that the static types are truthful That's often a mistake even in typed languages like typescript. Satisfying a static type analyzer isn't enough. Real data is whatever it is going to be. If your types are meaningless at runtime, you can easily encounter type mismatches. Static typing as implemented in typescript and python type hints is a tool for engineers instead of a tool for systems.
- roywiggins 5y ago
  The types are usually ignored, but not entirely erased- Pydantic uses type hints to do validation at runtime, so if you construct a Pydantic object with invalid data it will throw an error, so eg foo=PydanticFoo(**request.json()) Will try and validate a json request against the implied schema provided by the type hints on PydanticFoo while constructing it and will throw if it fails.
- goodoldneon 5y ago
  You missed my point. If you validate input, then the static types are representative of runtime types
  - musingsole 5y ago
    Ack, so I did :P
lelandbatey 5y ago
I want to emphasize for folks reading: if you want actual type enforcement and you want that type enforcement with minimum boilerplate, Pydantic is a fantastic option (I'm sure there are others as well). It's a great library that's totally use-agnostic (it's not HTTP specific, you can use it for whatever) and quite tight. Caveats are that it uses runtime validation to enforce types so there's a performance cost, but it's a performance hit that is mostly eclipsed by all the other huge performance hits that litter your typical Python implementation. Pydantic is amazing!
- mixmastamyk 5y ago
  IIRC, pydantic was quite big and I chose typeguard instead for a smaller project. Might be an option.

deltaonefour 5y ago

If you make it part of the deploy / build process and force programers to only use the subset of features supported by a type checker then you can benefit.

fantod 5y ago

Exactly. When I initially started using typed Python, I would write code as though I were using a statically-typed language. In order to appease mypy, I found myself increasingly needing to either rewrite my code in awkward ways or add explicit ignore comments. What happened is I started losing out on the advantages that Python provides as a dynamically-typed language without truly gaining those of statically-typed ones. Then I realized, type hints are just that: hints. They're a form of documentation. In some cases, they're very helpful, but in other cases, it doesn't really make sense to use them. I don't need to appease mypy. The type hints are their for the benefit of myself and other developers.

throwra620 5y ago
> rewrite my code in awkward ways or add explicit ignore comments. Probably just used to writing poor quality (immediately unintelligible to someone other than the author but functional) code then.
- ReflectedImage 5y ago
  Using typing in Python is a code smell. Lack of understanding of what the language is for and how to use it.
4kelly 5y ago
There is definitely truth to this statement. However often when I write something that’s hard to type, it’s a code smell. Ex. A complex nested map should probably be represented by a few concrete classes/abstractions. (Similar to when you something you wrote is hard to test, it’s probably just poorly abstracted code.)
- fantod 5y ago
  I agree and I think a good thing about the type hints is they help uncover these kinds of code smells. But there are definitely cases where it's definitely not worth the effort to fight mypy. One example is that variable re-assignment can cause type errors in mypy but re-assigning a variable to a value of a different type is an extremely common and reasonable thing to do in Python. Another situation where strict typing can be borderline hopeless is when parsing e.g. very dynamic content like json. Sure you could encode every possible situation using types but this would basically defeat the purpose of using Python for such a task.

goodside 5y ago

Type annotations are used at runtime in some cases, most notably when defining dataclasses.

musingsole 5y ago
I find this a bit of a wart on the dataclass implementation. Type hints (to me) are more like a comment than a statement. So, from my perspective, dataclasses feel like they're pulling data out of nearby comments. Ick. But it's a wart worth the conciseness, at least, until a better idea comes along. Maybe if type hints could be optionally enforced as a language feature, I'd feel it was more integrated.

mic47 5y ago

I've used python typing heavily at work and was really useful to detect bugs and helped with developement. > as evidenced by only 15% of repos passing mypy That is no evidence. Of course, if you don't have mypy in CI (or pre-commit hooks or whatever), your repo will not pass mypy. But if you have it in CI, then you can rely on them being correct. > tools like pytype or mypy will never capture all the complicated hacks possible But they can either enforce everything being correctly typed (if you go to really strict settings), or yes, you will have some code untyped, so you will be careful around that part. It's not like all or nothing.

whimsicalism 5y ago

Most static typed compiled languages also ignore types at runtime :)

munificent 5y ago
Not true. Java, C#, Scala, Kotlin, Swift, Dart, and others all keep around type information at runtime so that casts can throw exceptions at runtime when the value isn't of the right type. Otherwise, soundness is lost. In practice, most modern statically typed languages maintain soundness through a combination of compile time and runtime checking.
- whimsicalism 5y ago
  > most modern statically typed languages maintain soundness through a combination of compile time and runtime checking. Rust, C, C++ - all do not keep around this information. You're right that the JVM languages (and thus, naturally, C#) do it differently. I think it is a bit silly to name out each of the JVM languages separately.
  - pharmakom 5y ago
    C++ community is split by the RTTI compiler flags
  - zozbot234 5y ago
    Rust standard library includes RTTI support, though it's very rarely used.
  - munificent 5y ago
    Haskell, Go, and I think OCaml also rely on some amount of runtime checking for soundness. I think basically every managed statically typed language ends up carrying around runtime representation and uses some amount of runtime checks.
    - hither_shores 5y ago

throwaway894345 5y ago

> Python type annotations are more useful for a human as a form of documentation, than they ever will be to a machine I mean, machines serve people (for now!). So in addition to type documentation, type annotations are useful for things like "jump to definition" and "find references" and automatic refactoring. Moreover, the utility of annotations for type documentation depends on the correctness of those type annotations--if you don't have something like Mypy, then those inevitably annotations grow outdated over time and incorrect type annotations are even worse than none at all. > tools like pytype or mypy will never capture all the complicated hacks possible in the Python type system Technically you can represent any complicated hack if only via `any`, but that's just pedantry and we all know what you mean. Even still, it's not like "well you can't represent everything, so there's no point in trying to appease mypy" (which may not be your intended implication--I can't tell). The benefits of appeasing mypy are proportional to the amount of your code that is properly annotated--this is the thesis of gradual typing. > as evidenced by only 15% of repos passing mypy This very likely indicates that 85% of repos aren't running mypy regularly, or they're in some transition state. Certainly more than 15% of real-world Python can be described via Python type annotations.