4 ms·

Python's type system is a disappointment, but I don't think this article does a great job of explaining why. My biggest gripes: 1. There is no way to get typin

Python's type system is a disappointment, but I don't think this article does a great job of explaining why. My biggest gripes: 1. There is no way to get typing like dataclasses without writing your own mypy plugin. The syntax is simply not expressive enough to do it. This means that if you want to be productive with a library like Pydantic, you also have to add the mypy plugin to your dependencies and add it to your mypy config. Otherwise, no typing. 2. You cannot compose types. There is no way to say "hey this type is the same as this dictionary here, except the keys are all optional." If you're trying to type a RESTful API, your only option is to repeat yourself and carefully keep all your types in sync. 3. To this day, there is still no way to express optional keys. Not "Optional" keys, but keys that can be left out of your dictionary. The closest you can get is this weird hack where you can set `total=False` in your TypedDict, which makes all keys optional. I really wish Python learned from Typescript, but it's too late at this point. Too many half-baked measures have already made it into the spec.

kirbyfan64sos 5y ago
> There is no way to get typing like dataclasses without writing your own mypy plugin. The syntax is simply not expressive enough to do it. This means that if you want to be productive with a library like Pydantic, you also have to add the mypy plugin to your dependencies and add it to your mypy config. Otherwise, no typing. This would be solved by PEP 681, which originates from (and is already used in) pyright: https://peps.python.org/pep-0681/ https://peps.python.org/pep-0681/ > 3. To this day, there is still no way to express optional keys. Not "Optional" keys, but keys that can be left out of your dictionary. The closest you can get is this weird hack where you can set `total=False` in your TypedDict, which makes all keys optional. This is solved by PEP 655, which introduces NotRequired: https://peps.python.org/pep-0655/ https://peps.python.org/pep-0655/ It's implemented in mypy 0.930: https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.ht... > I really wish Python learned from Typescript, but it's too late at this point Personally, I feel like it's partly just entirely different styles: Python relies on a mix of nominal and duck types, whereas JavaScript is much more focused on structural typing. As a result, there are a lot of things like TypedDict that are absolutely essential for JS to even function but a decent bit less so in idiomatic Python code (keeping a significant amount of dicts around uses much more memory than objects, I recall a PyCon talk explicitly mentioning this too).
- meatmanek 5y ago
  Even without NotRequired, you've been able to have some-keys-optional-and-some-keys-required for a long time by putting the required keys in one TypedDict, then adding the optional keys in a subclass with total=False: https://mypy.readthedocs.io/en/stable/more_types.html#mixing-required-and-non-required-items https://mypy.readthedocs.io/en/stable/more_types.html#mixing... It's not exactly elegant (hence PEP655), but it works just fine.
RhysU 5y ago
If you have a dictionary with heterogenous, optional keys then I believe you have a struct-like class. If you have any constraints on the keys beyond a homogeneous domain and a homogeneous range then you have a struct-like class.
- phailhaus 5y ago
  Avoiding classes has a couple of benefits: 1. It reduces the overhead to use your interface. Your users don't have to import your struct classes, and can just pass in bare dictionaries. If Python's typing was more effective, these dictionaries would also be type checked to make sure you're calling the library's APIs correctly. 2. It deliberately closes off the option of adding state or initialization logic. Classes let you do way more than just encode a struct, and as a codebase grows they have a tendency to become more complex just because they can (like a gas filling its container). This makes APIs harder to maintain and work with. Sometimes, all you need is "raw structs".
  - RhysU 5y ago
    You want keyword arguments given these constraints. The type checker will confirm anything you like about those.
goodoldneon 5y ago
Agreed. Also, Mypy's dict inference is too broad, making TypedDict a little less useful. For example, the following code raises a Mypy error even though `foo` conforms to `Foo`: import typing class Foo(typing.TypedDict): a: int def do_thing(value: Foo) -> None: return foo = {"a": 1} # error: Argument 1 to "do_thing" has incompatible type "Dict[str, int]"; expected "Foo" do_thing(foo) But TypeScript can figure it out: interface Foo { a: number } function doThing(value: Foo): void { return } const foo = { a: 1 } doThing(foo)
- bayesian_horse 5y ago
  This is a misunderstanding of the Python type system. In the Typescript example you are depending on "foo" being structurally compatible to the "Foo" interface. In the Python case, "foo" and "Foo" might look structurally compatible, but they aren't. "Foo" isn't just a type, it's an object of type "class". You can for example print(Foo), you can't console.log(Foo). "foo" is not an instance of the Foo class. What you are looking for is "dict[str, object]".
  - goodoldneon 5y ago
    TypedDicts fall under structural typing, not nominal typing. So Mypy is cool with this: import typing class Foo(typing.TypedDict): a: int def do_thing(value: Foo) -> None: return foo: Foo = {"a": 1} do_thing(foo) My complaint here is that Mypy doesn't go far enough with structural inference. I assume this is because it doesn't support anonymous TypedDicts, whereas TypeScript supports anonymous interfaces/types
ledauphin 5y ago
I'm a big fan of mypy and static typing as a concrete improvement to Python, but this criticism is so superior to the original article that all I can say is bravo. Typescript's type system, of course, had and has the advantage of being a clean sheet design with an intermediate compilation step, so the comparison is hardly apples to apples. But there's no denying that I wish frequently that Python could have opted for a more powerful static type system.

kirbyfan64sos 5y ago

> There is no way to get typing like dataclasses without writing your own mypy plugin. The syntax is simply not expressive enough to do it. This means that if you want to be productive with a library like Pydantic, you also have to add the mypy plugin to your dependencies and add it to your mypy config. Otherwise, no typing. This would be solved by PEP 681, which originates from (and is already used in) pyright: https://peps.python.org/pep-0681/ https://peps.python.org/pep-0681/ > 3. To this day, there is still no way to express optional keys. Not "Optional" keys, but keys that can be left out of your dictionary. The closest you can get is this weird hack where you can set `total=False` in your TypedDict, which makes all keys optional. This is solved by PEP 655, which introduces NotRequired: https://peps.python.org/pep-0655/ https://peps.python.org/pep-0655/ It's implemented in mypy 0.930: https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.ht... > I really wish Python learned from Typescript, but it's too late at this point Personally, I feel like it's partly just entirely different styles: Python relies on a mix of nominal and duck types, whereas JavaScript is much more focused on structural typing. As a result, there are a lot of things like TypedDict that are absolutely essential for JS to even function but a decent bit less so in idiomatic Python code (keeping a significant amount of dicts around uses much more memory than objects, I recall a PyCon talk explicitly mentioning this too).

meatmanek 5y ago
Even without NotRequired, you've been able to have some-keys-optional-and-some-keys-required for a long time by putting the required keys in one TypedDict, then adding the optional keys in a subclass with total=False: https://mypy.readthedocs.io/en/stable/more_types.html#mixing-required-and-non-required-items https://mypy.readthedocs.io/en/stable/more_types.html#mixing... It's not exactly elegant (hence PEP655), but it works just fine.

RhysU 5y ago

If you have a dictionary with heterogenous, optional keys then I believe you have a struct-like class. If you have any constraints on the keys beyond a homogeneous domain and a homogeneous range then you have a struct-like class.

phailhaus 5y ago
Avoiding classes has a couple of benefits: 1. It reduces the overhead to use your interface. Your users don't have to import your struct classes, and can just pass in bare dictionaries. If Python's typing was more effective, these dictionaries would also be type checked to make sure you're calling the library's APIs correctly. 2. It deliberately closes off the option of adding state or initialization logic. Classes let you do way more than just encode a struct, and as a codebase grows they have a tendency to become more complex just because they can (like a gas filling its container). This makes APIs harder to maintain and work with. Sometimes, all you need is "raw structs".
- RhysU 5y ago
  You want keyword arguments given these constraints. The type checker will confirm anything you like about those.

goodoldneon 5y ago

Agreed. Also, Mypy's dict inference is too broad, making TypedDict a little less useful. For example, the following code raises a Mypy error even though `foo` conforms to `Foo`: import typing class Foo(typing.TypedDict): a: int def do_thing(value: Foo) -> None: return foo = {"a": 1} # error: Argument 1 to "do_thing" has incompatible type "Dict[str, int]"; expected "Foo" do_thing(foo) But TypeScript can figure it out: interface Foo { a: number } function doThing(value: Foo): void { return } const foo = { a: 1 } doThing(foo)

bayesian_horse 5y ago
This is a misunderstanding of the Python type system. In the Typescript example you are depending on "foo" being structurally compatible to the "Foo" interface. In the Python case, "foo" and "Foo" might look structurally compatible, but they aren't. "Foo" isn't just a type, it's an object of type "class". You can for example print(Foo), you can't console.log(Foo). "foo" is not an instance of the Foo class. What you are looking for is "dict[str, object]".
- goodoldneon 5y ago
  TypedDicts fall under structural typing, not nominal typing. So Mypy is cool with this: import typing class Foo(typing.TypedDict): a: int def do_thing(value: Foo) -> None: return foo: Foo = {"a": 1} do_thing(foo) My complaint here is that Mypy doesn't go far enough with structural inference. I assume this is because it doesn't support anonymous TypedDicts, whereas TypeScript supports anonymous interfaces/types

ledauphin 5y ago

I'm a big fan of mypy and static typing as a concrete improvement to Python, but this criticism is so superior to the original article that all I can say is bravo. Typescript's type system, of course, had and has the advantage of being a clean sheet design with an intermediate compilation step, so the comparison is hardly apples to apples. But there's no denying that I wish frequently that Python could have opted for a more powerful static type system.