11 ms·

This article is so hopelessly mis-informed, that it's practically hard to read without screaming "that's not how any of this works!" Most of their main argument

by kortex 4y ago

This article is so hopelessly mis-informed, that it's practically hard to read without screaming "that's not how any of this works!" Most of their main arguments - type hints can be wrong, they can be ignored, and they don't inform you in a useful way about program state because of this - all evaporate as soon as you start using the correct tooling. My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy plugins, black, isort, tabNine completion, and a bunch of custom scripts/Make to automate the common tasks. I'm not even sure I'm on the same planet as the author. I can practically spot incorrect type hints, mismatched types, weakly defined interfaces, from orbit, that's how well they are highlighted. Since every type is known, between pycharm and tabnine I can type 2-5 keys and spit out entire lines of code. And since it's all typed, automatic refactor is a breeze. I remember the dark ages, before type hints. It was rough. I lived in the REPL. Write 3 lines, run, Exception. Write 3 more lines, run, NoneType error. Ad nauseum. Now I write for hours on end and don't wince when I hit execute. And I don't have to go bug coworkers with questions like "hey what keys are in this dict when we redirect from the oauth endpoint". Sure, I have my gripes, but it's mostly with the lack of power of the type system, like lack of higher kinded types. But I'm confident it'll get there. Come to the typed side. We have less mental fatigue and screaming at neverending runtime oopsies.

hummingn3rd 4y ago
I was curious of seeing if pydantic was mentioned, thanks! We use most of the same tooling as you, I'm always interested in improving our tooling, would you be willing to share your mypy plug-ins names or maybe most useful tools for inspiration?
qayxc 4y ago
As long as you don't have to rely on 3rd party libraries, sure, type hints are fine. Many popular libraries, however, don't have type hints or use them inconsistently (looking at you numpy, pytorch, tensorflow, scipy, ...). Always great fun to see function signatures that use type hints for the parameters and return Any. So it really depends what kind of code you write - web frameworks might be fine, data science and ML is definitely not.
- Mehdi2277 4y ago
  You can write type stubs for other libraries. Over past year my team had a 100k line codebase that we’ve gotten mostly typed. But we still use a lot of other untyped libraries like tensorflow. So in past couple months we’ve started adding type stubs to the parts of the api we use. While tensorflow/numpy/etc are massive libraries most people only use a small subset of api. You can write type stubs for the subset you use. It definitely takes time but I find it very helpful to have useful type checking of tensorflow so I bit bullet and started writing stubs. I explored upstreaming them to type shed but got lazy on dealing with CI issue although I hope to revisit it later.
  - zhfliz 4y ago
    https://github.com/python/typeshed https://github.com/python/typeshed also provides community maintained stub packages for packages that are lacking upstream type hints
- wheelerof4te 4y ago
  Type hints are good if you stick to the std + your own code that mostly relies on std. Otherwise, don't bother.
  - Spivak 4y ago
    If you expect type hints to be perfect and completely disallow you to write type unsound code then this is right, if you expect type hints to be useful when developing then this is very wrong. The automatic type hints produced by pyright on completely untyped code are fantastic.
tonyg 4y ago
> My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy plugins, black, isort, tabNine completion, and a bunch of custom scripts/Make to automate the common tasks. Christ. What a mess. With a stack like that, it's a stretch to say you're "writing in Python" anymore.
- serial_dev 4y ago
  > Most of their main arguments (...) evaporate as soon as you start using the correct tooling. You silly! Just use these carefully curated list of 28 different tools (that I got to after years of trial and error) and you'll have a somewhat acceptable tooling and type support in Python.
  - danmur 4y ago
    Anything that can use an LSP server will give you most/all of those benefits. Your favourite editor + LSP, that's the tooling. It's very much worth it.
  - Spivak 4y ago
    That list is an editor, a type checker, a package to enforce type checks at runtime, 3rd party type annotations for a popular SQL ORM, some plugins for the type checker (probably the pydantic one), a code formatter, an import sorter, and (optionally) a Github co-pilot alternative.
- avianlyric 4y ago
  To be fair, I don’t think you need that entire stack. Just using VSCode + pyright (which is a one button install in VSCode) + type stubs for libraries your using without native hints, gets you 99% of the benefits. If you really want, you can even throw in a little pre-commit hit hook that runs pyright and prevents you from commuting code with type errors.
- ramraj07 4y ago
  Wow gatekeeping types out of python are we? Of course the language looks different when typing is enforced, let it evolve.
ramraj07 4y ago
Have you tried pytype? Curious if anyones compared it properly To mypy. Otherwise the exact same stack as yours, we use fast api as well. Also a big fan of using dataclasses everywhere.
da39a3ee 4y ago
You’ve never used a compiled statically typed language. I’m sorry to say it but your comment would sound naive to you if you had. The article is spot on.
igortg 4y ago
My thoughts exactly. Obviously, Python Type hints are an attempt to add type-check to a language "after the fact" and keep it optional. So it's unfair to compare that to a static typed language and conclude that it's bad. I work with large code bases (100K-1M lines) and can't look back to the time we didn't use Type Hints. Would maybe dismiss it only for scripts or very small projects.
nine_zeros 4y ago
Python types just work. Just use these 30 libraries, spice up your IDE with settings and voila /s. This is garbage and moving backwards as a programmer. I loved python when it was easy to use with any editor and with few plugins. Now I'm just going to switch to a statically typed language where I don't have to install a zillion plugins to get first class typing.
- wheelerof4te 4y ago
  This is the crust of the problem. Python type-hints bind you to a lot of other sister projects of Python, which makes you question if they are even official addition to the language. I think that's also what the OP's article tries to convey.
- Spivak 4y ago
  Python type annotations in the language are nothing more then allowing you put any Python expression in the "type slot" and have it be syntatically valid. They don't do anything. All the functionality is 3rd party packages taking advantage of their existence.
  - nine_zeros 4y ago
    My codebase is more complex than str, text, dict, int, bool. So developers are creating complex custom types with union[custom_type_1, custom_type_2] where each custom type could have more unions of other custom types. These custom types then get imported everywhere. It is utter garbage.
    - Spivak 4y ago
      I don't follow. So some 3rd party library will define a type like Routes = Union[RouteSet, List[Union[str, Route]]] and this is bad? The complexity is already there, you just don't want to see it?
      - nine_zeros 4y ago
        No, my own devs will write insane nested types at the detriment of all developers who end up looking at that code later. Routes = Union[GooglyRoutes, MetaRoutes] Where, GooglyRoutes=Union[List[CloudyRoutes], Sequence[AdseyRoutes]] Where CloudyRoutes = Union[RouteContainers] And then they'd import these types all over the python repo creating all kinds of import issues. Fml.
bredren 4y ago
I used tabnine from early on but found its capability pretty well subsumed by GitHub copilot. What has kept you on tabnine?
stackedinserter 4y ago
What opensource project is closest to "doing python the right way", in your opinion? I'm in "python is not suitable for production" camp, would be happy to change my mind. > I remember the dark ages, before type hints. It was rough. I lived in the REPL. Write 3 lines, run, Exception. Write 3 more lines, run, NoneType error. That's why I switched (back) to typed compiled languages and never went back.
twelve40 4y ago
Definitely types, but it's still weaker than in other languages. Just the other day found a maddening bug, where Django ORM query accepted `date` type comparison against a datetime field, and of course that resulted in crazy things in production. Other languages would have caught this compile-time.
ok_dad 4y ago
I use this very strict config with mypy and it's effectively a typed language. I rarely have any errors in runtime now, at least from type-related problems like assigning 'vals = 3' and later trying to call 'vals.append(4)'. I sometimes use 'Any' because it's basically required for things like loading JSON or retrieving the results from a GET request, but I have several strictness settings that ensure I never pass around wild Any values and I try to keep their use to within a single function or method and always return a well-typed value. Sure it's all in my IDE right now, but if I import 1 or 2 packages suddenly my well-typed Python code starts to act more like a typed language in runtime, too! disallow_any_unimported = true disallow_any_expr = true disallow_any_decorated = false # true if you never plan to use Any disallow_any_explicit = false # true if you never plan to use Any disallow_any_generics = false # true if you never plan to use Any disallow_subclassing_any = true disallow_untyped_calls = true disallow_untyped_defs = true disallow_incomplete_defs = true disallow_untyped_decorators = true no_implicit_optional = true warn_redundant_casts = true warn_unused_ignores = true no_warn_no_return = true warn_return_any = true warn_unreachable = true strict_equality = true strict = true

by kortex 4y ago

hummingn3rd 4y ago

I was curious of seeing if pydantic was mentioned, thanks! We use most of the same tooling as you, I'm always interested in improving our tooling, would you be willing to share your mypy plug-ins names or maybe most useful tools for inspiration?

qayxc 4y ago

As long as you don't have to rely on 3rd party libraries, sure, type hints are fine. Many popular libraries, however, don't have type hints or use them inconsistently (looking at you numpy, pytorch, tensorflow, scipy, ...). Always great fun to see function signatures that use type hints for the parameters and return Any. So it really depends what kind of code you write - web frameworks might be fine, data science and ML is definitely not.

Mehdi2277 4y ago
You can write type stubs for other libraries. Over past year my team had a 100k line codebase that we’ve gotten mostly typed. But we still use a lot of other untyped libraries like tensorflow. So in past couple months we’ve started adding type stubs to the parts of the api we use. While tensorflow/numpy/etc are massive libraries most people only use a small subset of api. You can write type stubs for the subset you use. It definitely takes time but I find it very helpful to have useful type checking of tensorflow so I bit bullet and started writing stubs. I explored upstreaming them to type shed but got lazy on dealing with CI issue although I hope to revisit it later.
- zhfliz 4y ago
  https://github.com/python/typeshed https://github.com/python/typeshed also provides community maintained stub packages for packages that are lacking upstream type hints
wheelerof4te 4y ago
Type hints are good if you stick to the std + your own code that mostly relies on std. Otherwise, don't bother.
- Spivak 4y ago
  If you expect type hints to be perfect and completely disallow you to write type unsound code then this is right, if you expect type hints to be useful when developing then this is very wrong. The automatic type hints produced by pyright on completely untyped code are fantastic.

tonyg 4y ago

> My stack is pycharm, mypy, pydantic, sqlalchemy stub, several mypy plugins, black, isort, tabNine completion, and a bunch of custom scripts/Make to automate the common tasks. Christ. What a mess. With a stack like that, it's a stretch to say you're "writing in Python" anymore.

serial_dev 4y ago
> Most of their main arguments (...) evaporate as soon as you start using the correct tooling. You silly! Just use these carefully curated list of 28 different tools (that I got to after years of trial and error) and you'll have a somewhat acceptable tooling and type support in Python.
- danmur 4y ago
  Anything that can use an LSP server will give you most/all of those benefits. Your favourite editor + LSP, that's the tooling. It's very much worth it.
- Spivak 4y ago
  That list is an editor, a type checker, a package to enforce type checks at runtime, 3rd party type annotations for a popular SQL ORM, some plugins for the type checker (probably the pydantic one), a code formatter, an import sorter, and (optionally) a Github co-pilot alternative.
avianlyric 4y ago
To be fair, I don’t think you need that entire stack. Just using VSCode + pyright (which is a one button install in VSCode) + type stubs for libraries your using without native hints, gets you 99% of the benefits. If you really want, you can even throw in a little pre-commit hit hook that runs pyright and prevents you from commuting code with type errors.
ramraj07 4y ago
Wow gatekeeping types out of python are we? Of course the language looks different when typing is enforced, let it evolve.

ramraj07 4y ago

Have you tried pytype? Curious if anyones compared it properly To mypy. Otherwise the exact same stack as yours, we use fast api as well. Also a big fan of using dataclasses everywhere.

da39a3ee 4y ago

You’ve never used a compiled statically typed language. I’m sorry to say it but your comment would sound naive to you if you had. The article is spot on.

igortg 4y ago

My thoughts exactly. Obviously, Python Type hints are an attempt to add type-check to a language "after the fact" and keep it optional. So it's unfair to compare that to a static typed language and conclude that it's bad. I work with large code bases (100K-1M lines) and can't look back to the time we didn't use Type Hints. Would maybe dismiss it only for scripts or very small projects.

nine_zeros 4y ago

Python types just work. Just use these 30 libraries, spice up your IDE with settings and voila /s. This is garbage and moving backwards as a programmer. I loved python when it was easy to use with any editor and with few plugins. Now I'm just going to switch to a statically typed language where I don't have to install a zillion plugins to get first class typing.

wheelerof4te 4y ago
This is the crust of the problem. Python type-hints bind you to a lot of other sister projects of Python, which makes you question if they are even official addition to the language. I think that's also what the OP's article tries to convey.
Spivak 4y ago
Python type annotations in the language are nothing more then allowing you put any Python expression in the "type slot" and have it be syntatically valid. They don't do anything. All the functionality is 3rd party packages taking advantage of their existence.
- nine_zeros 4y ago
  My codebase is more complex than str, text, dict, int, bool. So developers are creating complex custom types with union[custom_type_1, custom_type_2] where each custom type could have more unions of other custom types. These custom types then get imported everywhere. It is utter garbage.
  - Spivak 4y ago
    I don't follow. So some 3rd party library will define a type like Routes = Union[RouteSet, List[Union[str, Route]]] and this is bad? The complexity is already there, you just don't want to see it?
    - nine_zeros 4y ago
      No, my own devs will write insane nested types at the detriment of all developers who end up looking at that code later. Routes = Union[GooglyRoutes, MetaRoutes] Where, GooglyRoutes=Union[List[CloudyRoutes], Sequence[AdseyRoutes]] Where CloudyRoutes = Union[RouteContainers] And then they'd import these types all over the python repo creating all kinds of import issues. Fml.

bredren 4y ago

I used tabnine from early on but found its capability pretty well subsumed by GitHub copilot. What has kept you on tabnine?

stackedinserter 4y ago

What opensource project is closest to "doing python the right way", in your opinion? I'm in "python is not suitable for production" camp, would be happy to change my mind. > I remember the dark ages, before type hints. It was rough. I lived in the REPL. Write 3 lines, run, Exception. Write 3 more lines, run, NoneType error. That's why I switched (back) to typed compiled languages and never went back.

twelve40 4y ago

Definitely types, but it's still weaker than in other languages. Just the other day found a maddening bug, where Django ORM query accepted `date` type comparison against a datetime field, and of course that resulted in crazy things in production. Other languages would have caught this compile-time.

ok_dad 4y ago

I use this very strict config with mypy and it's effectively a typed language. I rarely have any errors in runtime now, at least from type-related problems like assigning 'vals = 3' and later trying to call 'vals.append(4)'. I sometimes use 'Any' because it's basically required for things like loading JSON or retrieving the results from a GET request, but I have several strictness settings that ensure I never pass around wild Any values and I try to keep their use to within a single function or method and always return a well-typed value. Sure it's all in my IDE right now, but if I import 1 or 2 packages suddenly my well-typed Python code starts to act more like a typed language in runtime, too! disallow_any_unimported = true disallow_any_expr = true disallow_any_decorated = false # true if you never plan to use Any disallow_any_explicit = false # true if you never plan to use Any disallow_any_generics = false # true if you never plan to use Any disallow_subclassing_any = true disallow_untyped_calls = true disallow_untyped_defs = true disallow_incomplete_defs = true disallow_untyped_decorators = true no_implicit_optional = true warn_redundant_casts = true warn_unused_ignores = true no_warn_no_return = true warn_return_any = true warn_unreachable = true strict_equality = true strict = true