5 ms·

Basic as it is, it is worth reminding folks reading this and similar articles that Python, even with type annotations, does not have a compile pass and that IDE

Basic as it is, it is worth reminding folks reading this and similar articles that Python, even with type annotations, does not have a compile pass and that IDEs like PyCharm tend to fail open when they can't figure out how to type check something. I see a lot of folks coming to Python from TypeScript and assuming that there's any enforcement of type annotations a la tsc. There's not, not unless you run a tool like MyPy on your code and configure it to be strict enough to provide the type assurance you need (what level of strictness is appropriate depends on the project and the programmers). I also see folks getting lulled into a false sense of security by their IDE. PyCharm is great, and it highlights many typing errors in code as you write them. But it gives up very easily in the presence of any indirection at all (many basic decorators which don't change function signatures, for example, are enough to cause it to stop trying to type check decorated functions at all). And when it gives up, there's no indication of that; giving up looks the same as error-free code. That's not an indictment of any tools. Python is the way it is for good reason, and it's type annotation system is good and getting better all the time (e.g. Protocols as discussed in this article). PyCharm is also super helpful and given how it performs it's type checking it's borderline miraculous that it works as well as it does in the first place. Rather, this is a reminder, especially for new python programmers: don't assume that type annotations give you anything (except documentation, which it's on you to keep accurate) for free.

tyingq 5y ago
I'm not an expert in either, but the difference doesn't seem dramatic to me. You can put type annotations in JS also and just use typescript in a way that's similar to MyPy. Adding types to either JS or Python seems "not free" to me. You pull in extra tools and syntax for either.
- zbentley 5y ago
  In TypeScript, there's no* way to run your code without invoking the type checker, as it has to transpile TS to JS. While you can suppress/reduce the strictness of the checker, you can't avoid it entirely, and opting out takes work. Python is the opposite; using type annotations for any sort of verification is opt-in exclusively, and not part of the language/runtime itself. *Yes, there are some direct TS runtimes/compilers, but they aren't how the vast majority of TS code runs.
mynameisash 5y ago
I just got bit tonight by Python's duck typing. I was trying to use PIL to draw rectangles for a super simple script to visualize results from another component. I found an example and tweaked accordingly: w, h = 220, 190 shape = [(40, 40), (w - 10, h - 10)] # ... img.rectangle(shape) When my results came out totally wrong, I spent 45 minutes trying to figure out if my code that built the shapes was off or if my drawing was off. What I had assumed were rectangle dimensions in the form of `(width, height)` were actually bounding box coordinates `(x2, y2)`. Both of those are tuples of ints, but semantically, they're very different. I'm growing increasingly frustrated with Python, and in my frustration, I feel like this nonsense could be wholly avoided by stronger typing. Were I to use Rust or C#, two languages with which I have much more experience, I have to imagine the library would prevent this loss of productivity. Simply define a Point struct and it's perfectly clear what is being passed in. Edit: I shouldn't have said duck typing but rather the lack of strong/compile checks. Apologies for my sloppy thinking.
- pgorczak 5y ago
  You don’t need strong typing but good naming to avoid these issues. While a type system kind of requires everything to be named, it really doesn’t matter to the type checker whether these names are semantically correct or have any semantic meaning at all.
- framecowbird 5y ago
  There's nothing stopping you doing that in Python. If it's important to you to define a point struct, do that, add a type annotation, and add mypy to your compile. When i used to work with C++ i would spend hours debugging compile issues that were just the wrong type (templates are especially bad!) that really had no impact on the code, it was just a waste of my time. What I like about python is that the static typing is there if you want it, but you don't have to.
  - still_grokking 5y ago
    To my knowledge Python doesn't have any sound type system. Mypy doesn't provide that. So, no you can't have (currently) proper static typing in Python. It's more like in TypeScript where it's a best effort thing. (At least you get notified when the typer gives up. That's much better than nothing of course!) C++ templates aren't a good example either. Templates are a Turning complete language so they can't provide guaranties in all cases and aren't as such type safe at all. Better examples for type systems and their usefulness would be languages form the ML family (also including Haskell and similar in this case).
JackMcMack 5y ago
PyCharm has some longstanding issues with type checking aliased and union types. My fix coming in 2021.3.1 [0] [0] https://github.com/JetBrains/intellij-community/pull/1655 https://github.com/JetBrains/intellij-community/pull/1655
- nitred 5y ago
  I face these issues on the daily. Pycharm's type checking is sufficient for my use case but the TypeVar and Union type checks are a bother and I always pay close attention to it. Really appreciate the fix(es)!
ossopite 5y ago
people talk a lot about MyPy but I must recommend Pyright as a better alternative. It's much more robust. In my experience the main problem with MyPy is that when it's unsure about what's going on (especially due to imports it doesn't understand, sometimes because of misconfiguration) it just gives up on type checking with absolutely no warning. Pyright does not have this failure mode and handles Unknown types properly (with configurable strictness)
- ramraj07 5y ago
  Have you compared pytype ? Want to know if pyright is wort switching.
dragonwriter 5y ago
> Basic as it is, it is worth reminding folks reading this and similar articles that Python, even with type annotations, does not have a compile pass This is literally false, Python does have a compilation phase, and with its default settings it saves the compiled results for reuse. You probably mean a static analysis phase, and that's...up to your workflow. If you are using mypy, pyright/pylance, or a other typechecker, it does have a static type analysis phase, that's what those are. And if you are getting typw feedback in an IDE, that's probably what you are doing. > PyCharm is great, and it highlights many typing errors in code as you write them. But it gives up very easily in the presence of any indirection at all (many basic decorators which don't change function signatures, for example, are enough to cause it to stop trying to type check decorated functions at all) Most of Python's major typecheckers (not sure about PyCharm’s custom one, definitely Pyright/Pylance and mypy) have configurable levels of strictness. They can definitely be configured not to give up easily. But a decorator is a function, and if it is untyped rather than expressing how it transforms the wrapped function signature—even if it does not, in fact, transform the function signature—then it will probably have an Any type (none of typecheckers are great at inference) and either defeat or fail typechecking, depending on strictness settings. > Rather, this is a reminder, especially for new python programmers: don't assume that type annotations give you anything (except documentation, which it's on you to keep accurate) for free. Annotations alone don't give you anything (and aren't free, as you have to crrate them), but static analysis definitely gives you something.
- zbentley 5y ago
  You're right, my phrasing was inaccurate. The intent was: Python does not perform type checking based on annotations (e.g. Protocols as discussed in the article) when it starts. Unfortunately PyCharm's analyzer is pretty fundamentally oriented towards failing open, rather than closed. For decorators, PyCharm often infers whether or not a decorator changes a signature. However, it does this inconsistently (and it may be backed by a specific list of supported decorators by name/module, I'm not sure). When it can't tell what a decorator does, it doesn't try to validate calls to decorated functions, which is kind of a bummer. What I'd really like is a way to tell the IDE that unrecognized/unparseable decorators should be assumed not to emit functions with a different signature; that would be wrong sometimes, but useful in what is for me the most common case.

tyingq 5y ago

I'm not an expert in either, but the difference doesn't seem dramatic to me. You can put type annotations in JS also and just use typescript in a way that's similar to MyPy. Adding types to either JS or Python seems "not free" to me. You pull in extra tools and syntax for either.

zbentley 5y ago
In TypeScript, there's no* way to run your code without invoking the type checker, as it has to transpile TS to JS. While you can suppress/reduce the strictness of the checker, you can't avoid it entirely, and opting out takes work. Python is the opposite; using type annotations for any sort of verification is opt-in exclusively, and not part of the language/runtime itself. *Yes, there are some direct TS runtimes/compilers, but they aren't how the vast majority of TS code runs.

mynameisash 5y ago

I just got bit tonight by Python's duck typing. I was trying to use PIL to draw rectangles for a super simple script to visualize results from another component. I found an example and tweaked accordingly: w, h = 220, 190 shape = [(40, 40), (w - 10, h - 10)] # ... img.rectangle(shape) When my results came out totally wrong, I spent 45 minutes trying to figure out if my code that built the shapes was off or if my drawing was off. What I had assumed were rectangle dimensions in the form of `(width, height)` were actually bounding box coordinates `(x2, y2)`. Both of those are tuples of ints, but semantically, they're very different. I'm growing increasingly frustrated with Python, and in my frustration, I feel like this nonsense could be wholly avoided by stronger typing. Were I to use Rust or C#, two languages with which I have much more experience, I have to imagine the library would prevent this loss of productivity. Simply define a Point struct and it's perfectly clear what is being passed in. Edit: I shouldn't have said duck typing but rather the lack of strong/compile checks. Apologies for my sloppy thinking.

pgorczak 5y ago
You don’t need strong typing but good naming to avoid these issues. While a type system kind of requires everything to be named, it really doesn’t matter to the type checker whether these names are semantically correct or have any semantic meaning at all.
framecowbird 5y ago
There's nothing stopping you doing that in Python. If it's important to you to define a point struct, do that, add a type annotation, and add mypy to your compile. When i used to work with C++ i would spend hours debugging compile issues that were just the wrong type (templates are especially bad!) that really had no impact on the code, it was just a waste of my time. What I like about python is that the static typing is there if you want it, but you don't have to.
- still_grokking 5y ago
  To my knowledge Python doesn't have any sound type system. Mypy doesn't provide that. So, no you can't have (currently) proper static typing in Python. It's more like in TypeScript where it's a best effort thing. (At least you get notified when the typer gives up. That's much better than nothing of course!) C++ templates aren't a good example either. Templates are a Turning complete language so they can't provide guaranties in all cases and aren't as such type safe at all. Better examples for type systems and their usefulness would be languages form the ML family (also including Haskell and similar in this case).

JackMcMack 5y ago

PyCharm has some longstanding issues with type checking aliased and union types. My fix coming in 2021.3.1 [0] [0] https://github.com/JetBrains/intellij-community/pull/1655 https://github.com/JetBrains/intellij-community/pull/1655

nitred 5y ago
I face these issues on the daily. Pycharm's type checking is sufficient for my use case but the TypeVar and Union type checks are a bother and I always pay close attention to it. Really appreciate the fix(es)!

ossopite 5y ago

people talk a lot about MyPy but I must recommend Pyright as a better alternative. It's much more robust. In my experience the main problem with MyPy is that when it's unsure about what's going on (especially due to imports it doesn't understand, sometimes because of misconfiguration) it just gives up on type checking with absolutely no warning. Pyright does not have this failure mode and handles Unknown types properly (with configurable strictness)

ramraj07 5y ago
Have you compared pytype ? Want to know if pyright is wort switching.

dragonwriter 5y ago

> Basic as it is, it is worth reminding folks reading this and similar articles that Python, even with type annotations, does not have a compile pass This is literally false, Python does have a compilation phase, and with its default settings it saves the compiled results for reuse. You probably mean a static analysis phase, and that's...up to your workflow. If you are using mypy, pyright/pylance, or a other typechecker, it does have a static type analysis phase, that's what those are. And if you are getting typw feedback in an IDE, that's probably what you are doing. > PyCharm is great, and it highlights many typing errors in code as you write them. But it gives up very easily in the presence of any indirection at all (many basic decorators which don't change function signatures, for example, are enough to cause it to stop trying to type check decorated functions at all) Most of Python's major typecheckers (not sure about PyCharm’s custom one, definitely Pyright/Pylance and mypy) have configurable levels of strictness. They can definitely be configured not to give up easily. But a decorator is a function, and if it is untyped rather than expressing how it transforms the wrapped function signature—even if it does not, in fact, transform the function signature—then it will probably have an Any type (none of typecheckers are great at inference) and either defeat or fail typechecking, depending on strictness settings. > Rather, this is a reminder, especially for new python programmers: don't assume that type annotations give you anything (except documentation, which it's on you to keep accurate) for free. Annotations alone don't give you anything (and aren't free, as you have to crrate them), but static analysis definitely gives you something.

zbentley 5y ago
You're right, my phrasing was inaccurate. The intent was: Python does not perform type checking based on annotations (e.g. Protocols as discussed in the article) when it starts. Unfortunately PyCharm's analyzer is pretty fundamentally oriented towards failing open, rather than closed. For decorators, PyCharm often infers whether or not a decorator changes a signature. However, it does this inconsistently (and it may be backed by a specific list of supported decorators by name/module, I'm not sure). When it can't tell what a decorator does, it doesn't try to validate calls to decorated functions, which is kind of a bummer. What I'd really like is a way to tell the IDE that unrecognized/unparseable decorators should be assumed not to emit functions with a different signature; that would be wrong sometimes, but useful in what is for me the most common case.