6 ms·

> Type hinting (3.5+) > Static vs dynamic typing is a spicy topic in software engineering and almost everyone has an opinion on it. I will let the reader decid

> Type hinting (3.5+) > Static vs dynamic typing is a spicy topic in software engineering and almost everyone has an opinion on it. I will let the reader decide when they should write types, but I think you should at least know that Python 3 supports type hints. I think this gives readers the impression that type hints have anything to do with dynamic vs. static type systems, which isn't true. These are merely annotations that are attached to values. In fact, it's legal to use full strings as type annotations — they have no inherent semantic value. You can gain semantic value by using a tool like MyPy or an IDE like PyCharm, but the type annotations do not in themselves do anything. I think it could be worth clarifying this for readers who are unaware.

xjlin0 7y ago
How to do Type hinting with argument default values?
- DonaldPShimoda 7y ago
  Pretty straightforwardly: x: int = 0 This makes sense when you see that the variable production in the Python grammar [0] has been updated to essentially read "either a lone variable or a variable with a type annotation". (I highlighted the relevant line of the grammar for you in the link.) [0] https://github.com/python/cpython/blob/master/Grammar/Grammar#L53 https://github.com/python/cpython/blob/master/Grammar/Gramma...
- deleted 7y ago
  [deleted]
xtf 7y ago
Except further optimisation in Cython like http://docs.cython.org/en/latest/src/tutorial/pure.html#static-typing http://docs.cython.org/en/latest/src/tutorial/pure.html#stat...
- DonaldPShimoda 7y ago
  I think this falls under the realm of "gain[ing] semantic value by using a tool like MyPy". The implementation of static typing by way of processing type annotations is an implementation choice, and not a part of the Python language specification.
  - Animats 7y ago
    The implementation of static typing by way of processing type annotations is an implementation choice, and not a part of the Python language specification. That comes close to being the worst idea in the history of programming languages. Types aren't checked, you can't trust the annotations, and you give up a big optimization opportunity. But Guido's naive interpreter will still work. If you say something is a float, the compiler needs to both enforce that and use that. Then maybe the Python crowd wouldn't need to call out to C code whenever they needed to do some number crunching.
    - orf 7y ago
      Why is calling out to C code via something like numpy a bad thing? Python is never going to be as fast as C. Ever. Even if you somehow managed to decouple numeric types from PyObject in a backwards compatible way it would not be as fast. Type Hints can’t be used as an optimization in the interpreter due to the way they are resolved. And in any case, it would not be safe to do so and there are very few cases where optimizations make sense.
    - coldtea 7y ago
      >That comes close to being the worst idea in the history of programming languages. Types aren't checked, you can't trust the annotations As for types not being checked, that's not a problem. JITs do extra specialization on assumed types even without annotations all the time, and just drop them at runtime when they are not valid anymore (e.g. a variable that only pointed to integers now stores a string). Still, this runtime overhead of dropping specialized code aside, those optimizations make e.g. v8 hella fast. (And of course you could also just enable optimizations after you've statically checked the whole program types with something like mypy -- this is even easier and less dynamic than what v8 does). >and you give up a big optimization opportunity You don't give anything up, since an implementation can take advantage of this "big optimization opportunity" if it wants and has the manpower to add it. >But Guido's naive interpreter will still work. It's also less work, which unless we have a volunteer here, was and remains the intention.
ehsankia 7y ago
Yeah, I'm pretty sure Python itself explicitly uses the term "annotations" everywhere instead of "type hints". The point is that they can be used for anything, typing being one such use. As for libraries, other than MyPy, there's also Pyre by Facebook and Pytype by Google. I'm definitely excited to see more advanced static analysis being implemented beyond simple type checking. Some of these libraries are starting to explore that.
- Sharlin 7y ago
  Permitting arbitrary annotations is a rather nice feature that, besides static analysis, opens the door for expressive runtime pre/postcondition checking functionality like the Clojure spec library[1]. Does Python have something similar? [1] https://clojure.org/guides/spec https://clojure.org/guides/spec
  - jacobush 7y ago
    https://andreacensi.github.io/contracts/tour.html#quick-tour https://andreacensi.github.io/contracts/tour.html#quick-tour looks to be a stable solution for that.
  - auscompgeek 7y ago
    Unfortunately PEP 563 [1] means that any non-type-hint usage of annotations is considered deprecated. You could still do things with decorators (like mentioned in a sibling comment), although I suppose that doesn't offer such an elegant syntax. [1]: https://www.python.org/dev/peps/pep-0563/#non-typing-usage-of-annotations https://www.python.org/dev/peps/pep-0563/#non-typing-usage-o...
  - DonaldPShimoda 7y ago
    Although not really what they were intended for, somebody used annotations to provide a macro system [0]. [0] https://tinkering.xyz/abusing-type-annotations/ https://tinkering.xyz/abusing-type-annotations/
    - z_mitchell 7y ago
BlackFly 7y ago
They are available in the runtime, so you can do runtime reflection on the annotations and perform dependency injection if that is your thing. In that sense they do something: they provide information to the runtime which you can use if you want.
- DonaldPShimoda 7y ago
  I think this is just a semantic argument at this point, but to me what you mention isn't evidence of the annotations doing anything. They make it possible for things to be done, just as I said. The distinction I'm drawing is to statically-typed languages where simply adding annotations will actually change semantics of the program (e.g., by forcing a downcast or something). But you're absolutely right that annotations can be useful for many things!
BerislavLopac 7y ago
The most underrated part of type annotations are so-called stub files [0], which allow you to keep your annotations separate from the actual code, thus reducing its size while still benefiting from the static analysis with mypy. Unfortunately the IDE support for those files (.pyi extension) is quite limited -- I have even suggested to the PyCharm team to implement an option to automatically save all annotations to stub files, while still displaying them alongside the code in the editor. [0] https://www.python.org/dev/peps/pep-0484/#stub-files https://www.python.org/dev/peps/pep-0484/#stub-files
- NelsonMinar 7y ago
  I'm sort of sad that Python has reinvented .h files.
  - ASalazarMX 7y ago
    Same here. Fortunately those files are entirely optional and apparently seldomly used, judging by the limited IDE support after 5 years.
- DonaldPShimoda 7y ago
  Whoa, that's neat! I had no idea! Does PyCharm support these? Like if I write the stub files manually, will type hints and autocompletion work correctly in the main file? This could be a great way to reduce the amount of syntactic overhead, at least in function definitions and top-level code.
  - BerislavLopac 7y ago
    At the moment, you can click on a (say) class definition, and, if it has the corresponding entry in the pyi file, it will open it. But I would like it to be a bit more integrated. Another advantage is that you can use pyi files for static checking, but remove them when deploying the application.
- tlavoie 7y ago
  Sounds sort of like the .mli files in Ocaml. As I understand it, the compiler uses (and enforces) the types found therein, though can infer a lot from the code in .ml files as well.
mrfusion 7y ago
Are there any tools that can use the annotations and watch a running script and warn you if they’re violated? Maybe during your test suite. Or linters could do some basic analysis.
- uryga 7y ago
  off the top of my head there's pytypes¹ and typeguard² – both look at type annotations and enforce them at runtime. ¹ https://github.com/Stewori/pytypes/ https://github.com/Stewori/pytypes/ ² https://github.com/agronholm/typeguard https://github.com/agronholm/typeguard
deleted 7y ago
[deleted]
bitcurious 7y ago
What's the best practice for type hinting a class that's not being imported? I.e. A module contains a function that accepts a pandas.DataFrame. This module doesn't import pandas.
- jimbo1qaz 7y ago
  from typing import TYPE_CHECKING if TYPE_CHECKING: import whatever def foo(param: 'whatever'): ...
  - bitcurious 7y ago
    Thanks! :)

xjlin0 7y ago

How to do Type hinting with argument default values?

DonaldPShimoda 7y ago
Pretty straightforwardly: x: int = 0 This makes sense when you see that the variable production in the Python grammar [0] has been updated to essentially read "either a lone variable or a variable with a type annotation". (I highlighted the relevant line of the grammar for you in the link.) [0] https://github.com/python/cpython/blob/master/Grammar/Grammar#L53 https://github.com/python/cpython/blob/master/Grammar/Gramma...
deleted 7y ago
[deleted]

xtf 7y ago

Except further optimisation in Cython like http://docs.cython.org/en/latest/src/tutorial/pure.html#static-typing http://docs.cython.org/en/latest/src/tutorial/pure.html#stat...

DonaldPShimoda 7y ago
I think this falls under the realm of "gain[ing] semantic value by using a tool like MyPy". The implementation of static typing by way of processing type annotations is an implementation choice, and not a part of the Python language specification.
- Animats 7y ago
  The implementation of static typing by way of processing type annotations is an implementation choice, and not a part of the Python language specification. That comes close to being the worst idea in the history of programming languages. Types aren't checked, you can't trust the annotations, and you give up a big optimization opportunity. But Guido's naive interpreter will still work. If you say something is a float, the compiler needs to both enforce that and use that. Then maybe the Python crowd wouldn't need to call out to C code whenever they needed to do some number crunching.
  - orf 7y ago
    Why is calling out to C code via something like numpy a bad thing? Python is never going to be as fast as C. Ever. Even if you somehow managed to decouple numeric types from PyObject in a backwards compatible way it would not be as fast. Type Hints can’t be used as an optimization in the interpreter due to the way they are resolved. And in any case, it would not be safe to do so and there are very few cases where optimizations make sense.
  - coldtea 7y ago
    >That comes close to being the worst idea in the history of programming languages. Types aren't checked, you can't trust the annotations As for types not being checked, that's not a problem. JITs do extra specialization on assumed types even without annotations all the time, and just drop them at runtime when they are not valid anymore (e.g. a variable that only pointed to integers now stores a string). Still, this runtime overhead of dropping specialized code aside, those optimizations make e.g. v8 hella fast. (And of course you could also just enable optimizations after you've statically checked the whole program types with something like mypy -- this is even easier and less dynamic than what v8 does). >and you give up a big optimization opportunity You don't give anything up, since an implementation can take advantage of this "big optimization opportunity" if it wants and has the manpower to add it. >But Guido's naive interpreter will still work. It's also less work, which unless we have a volunteer here, was and remains the intention.

ehsankia 7y ago

Yeah, I'm pretty sure Python itself explicitly uses the term "annotations" everywhere instead of "type hints". The point is that they can be used for anything, typing being one such use. As for libraries, other than MyPy, there's also Pyre by Facebook and Pytype by Google. I'm definitely excited to see more advanced static analysis being implemented beyond simple type checking. Some of these libraries are starting to explore that.

Sharlin 7y ago
Permitting arbitrary annotations is a rather nice feature that, besides static analysis, opens the door for expressive runtime pre/postcondition checking functionality like the Clojure spec library[1]. Does Python have something similar? [1] https://clojure.org/guides/spec https://clojure.org/guides/spec
- jacobush 7y ago
  https://andreacensi.github.io/contracts/tour.html#quick-tour https://andreacensi.github.io/contracts/tour.html#quick-tour looks to be a stable solution for that.
- auscompgeek 7y ago
  Unfortunately PEP 563 [1] means that any non-type-hint usage of annotations is considered deprecated. You could still do things with decorators (like mentioned in a sibling comment), although I suppose that doesn't offer such an elegant syntax. [1]: https://www.python.org/dev/peps/pep-0563/#non-typing-usage-of-annotations https://www.python.org/dev/peps/pep-0563/#non-typing-usage-o...
- DonaldPShimoda 7y ago
  Although not really what they were intended for, somebody used annotations to provide a macro system [0]. [0] https://tinkering.xyz/abusing-type-annotations/ https://tinkering.xyz/abusing-type-annotations/
  - z_mitchell 7y ago

BlackFly 7y ago

They are available in the runtime, so you can do runtime reflection on the annotations and perform dependency injection if that is your thing. In that sense they do something: they provide information to the runtime which you can use if you want.

DonaldPShimoda 7y ago
I think this is just a semantic argument at this point, but to me what you mention isn't evidence of the annotations doing anything. They make it possible for things to be done, just as I said. The distinction I'm drawing is to statically-typed languages where simply adding annotations will actually change semantics of the program (e.g., by forcing a downcast or something). But you're absolutely right that annotations can be useful for many things!

BerislavLopac 7y ago

The most underrated part of type annotations are so-called stub files [0], which allow you to keep your annotations separate from the actual code, thus reducing its size while still benefiting from the static analysis with mypy. Unfortunately the IDE support for those files (.pyi extension) is quite limited -- I have even suggested to the PyCharm team to implement an option to automatically save all annotations to stub files, while still displaying them alongside the code in the editor. [0] https://www.python.org/dev/peps/pep-0484/#stub-files https://www.python.org/dev/peps/pep-0484/#stub-files

NelsonMinar 7y ago
I'm sort of sad that Python has reinvented .h files.
- ASalazarMX 7y ago
  Same here. Fortunately those files are entirely optional and apparently seldomly used, judging by the limited IDE support after 5 years.
DonaldPShimoda 7y ago
Whoa, that's neat! I had no idea! Does PyCharm support these? Like if I write the stub files manually, will type hints and autocompletion work correctly in the main file? This could be a great way to reduce the amount of syntactic overhead, at least in function definitions and top-level code.
- BerislavLopac 7y ago
  At the moment, you can click on a (say) class definition, and, if it has the corresponding entry in the pyi file, it will open it. But I would like it to be a bit more integrated. Another advantage is that you can use pyi files for static checking, but remove them when deploying the application.
tlavoie 7y ago
Sounds sort of like the .mli files in Ocaml. As I understand it, the compiler uses (and enforces) the types found therein, though can infer a lot from the code in .ml files as well.

mrfusion 7y ago

Are there any tools that can use the annotations and watch a running script and warn you if they’re violated? Maybe during your test suite. Or linters could do some basic analysis.

uryga 7y ago
off the top of my head there's pytypes¹ and typeguard² – both look at type annotations and enforce them at runtime. ¹ https://github.com/Stewori/pytypes/ https://github.com/Stewori/pytypes/ ² https://github.com/agronholm/typeguard https://github.com/agronholm/typeguard

deleted 7y ago

[deleted]

bitcurious 7y ago

What's the best practice for type hinting a class that's not being imported? I.e. A module contains a function that accepts a pandas.DataFrame. This module doesn't import pandas.

jimbo1qaz 7y ago
from typing import TYPE_CHECKING if TYPE_CHECKING: import whatever def foo(param: 'whatever'): ...
- bitcurious 7y ago
  Thanks! :)