10 ms·

Python 3 Types in the Wild

68 points by zdw 5y ago

ghostwriter 5y ago
> which allow developers to say that a function returns Dict[List[Set[FrozenSet[int]]], str] this type is impossible in Python
- mivade 5y ago
  Correct. Keys must be immutable and hashable.
  - pdonis 5y ago
    Technically keys only have to be hashable, but an object that is hashable but not immutable is not a good idea and in practice the two will go together.
- gnulinux 5y ago
  Not only impossible in python, but also seems conceptually useless (why would you have a set of frozen sets?).
  - yifanl 5y ago
    Ensure uniqueness, i guess? Though you'd lose that when you make the list of sets. I could maybe see something like this as an intermediary result of parsing some horrible, nested obscure message format (except the dict, that baffles me unless the key-value types are reversed?)
  - raymondh 5y ago
    Almost the entire purpose of frozensets was so that you could have sets of frozensets. It's the only way to have nested sets. Frozensets also work as dictionary keys.
  - deltaonefour 5y ago
    This is not useless. Why would you have a list of lists? A list is just an iterable container class, which is what a set is. If the need exists for a list of lists then it exists for a set of sets. An example of this is finding all combinations possible non ordered groups of letters given a collection of letters: abadfjiik. A typical algorithm will produce dupes. But if your accumulator is a set of frozen sets you can produce a result with no dupes.
  - nisegami 5y ago
    I guarantee something like that is running in production somewhere in the world (maybe without the list part).
- lolive 5y ago
  With this type system is it possible to subclass string and int, to enforce types for, let’s say, firstName, postalCode, etc.
  - paisawalla 5y ago
    Yes, simply FirstNameType = str (But don't use ints for postal codes, even if the postal codes are numeric)
    - lolive 5y ago
      Huuum. If I sign a function so it only accepts a FirstNameType, and I use the statement you propose, won’t any string be acceptable as an input parameter, from the type system point of view?
      - jamincan 5y ago
        Yeah, that is the syntax for a type alias. You can use NewType to create an actual subclass of the type instead. https://docs.python.org/3/library/typing.html#newtype https://docs.python.org/3/library/typing.html#newtype
      - paisawalla 5y ago
        In that case, you need a subclass of string that does nothing but has its type as FirstNameType. Wherever you create or return strings that are first names, you'll need to instantiate FirstNameType instead.
    - colejohnson66 5y ago
      > ... even if the postal codes are numeric Maybe in the US. In Canada, they are 6 characters of letter-digit-letter-digit-letter-digit (in regex: /((?:[A-Z][0-9]){3})/)
      - kennu 5y ago
        And in the future, they could be anything. Finland expanded the allowed characters in person identification numbers to support people born after 2000. Usually bad idea to be too strict in checking fields where people manually enter text.
- lolive 5y ago
  Does IntelliJ use any kind of static type analysis to help the user autocomplete properly? [== do we have something like Java???]
  - deltaonefour 5y ago
    Yes. Try pycharm.
    - aenis 5y ago
      Within reason. I use type annotations solely as means of documenting code, with the added benefit of having autosuggest work all of the time. I am one of those guys who put a lot of comments in their code.
      - deltaonefour 5y ago
        I strictly use type annotations in python as if python was a typed language. This is within reason as typed languages exist.
- nickysielicki 5y ago
  This bugged me, too. You can have a dictionary which maps to strings, you can have a list containing sets of frozensets of ints, but you can’t have a dictionary of unhashable types.
  - mturmon 5y ago
    Yep, here's how it plays out: Python 3.8.5 (default, Sep 4 2020, 02:22:02) [Clang 10.0.0 ] :: Anaconda, Inc. on darwin >>> d=dict() >>> l=[1,2,3] >>> d[l]=42 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'list'
    - setr 5y ago
      https://pypi.org/project/forbiddenfruit/ https://pypi.org/project/forbiddenfruit/ Apparently there are ways to monkey patch core types, so you could probably add a __hash__ method to list…
- bobbylarrybobby 5y ago
  While it's impossible for this type to exist in python, will a type checker complain if you write that type out? Does it have the notion that the first type of `Dict` (or just `dict` in Python 3.9+, iirc) must be `Hashable`
kodablah 5y ago
> stripped annotations out of files and askedPyType to infer them, it failed to do so in 77% of cases [...] MyPy found that only 15% of the 2,678 repositories examined were type-correct; this may be a result of MyPy being very conservative and producing false positives The paper referenced is a year and a half old and type checkers have fixed/added a lot since then. Also, I'd guess that more projects use Mypy than Pytype and fix errors reported, so that may be another reason more pass Mypy even without annotations (though conservatism does play).
- aitchnyu 5y ago
  Anybody swears by Pytype? I'm puzzled by it existing between MyPy(static) and unit tests (dynamic).
  - whimsicalism 5y ago
    Wdym "between"? Pytype is another static analyzer.
deleted 5y ago
[deleted]
assbuttbuttass 5y ago
Python type annotations are more useful for a human as a form of documentation, than they ever will be to a machine. I think Python completely ignores them at runtime, and tools like pytype or mypy will never capture all the complicated hacks possible in the Python type system (as evidenced by only 15% of repos passing mypy)
- IshKebab 5y ago
  This is only because the Python type hinting ecosystem is so bad. Tons of projects don't have type hints, there are multiple conflicting interpretations of the types, MyPy is very buggy (Pyright is much much better but few people use it). Compare it to Typescript which is a similar sort of tacked on type hinting system - Typescript is basically always machine checked.
- goodoldneon 5y ago
  Python does ignore them at runtime, but as long as you're validating your inputs (e.g. using Pydantic for HTTP request validation) then you can trust that the static types are truthful
  - musingsole 5y ago
    > trust that the static types are truthful That's often a mistake even in typed languages like typescript. Satisfying a static type analyzer isn't enough. Real data is whatever it is going to be. If your types are meaningless at runtime, you can easily encounter type mismatches. Static typing as implemented in typescript and python type hints is a tool for engineers instead of a tool for systems.
    - roywiggins 5y ago
      The types are usually ignored, but not entirely erased- Pydantic uses type hints to do validation at runtime, so if you construct a Pydantic object with invalid data it will throw an error, so eg foo=PydanticFoo(**request.json()) Will try and validate a json request against the implied schema provided by the type hints on PydanticFoo while constructing it and will throw if it fails.
ohmanjjj 5y ago
After many years of writing Python code, I just can’t stand the distraction and messiness that Types results in and the appearance of clutter and unnecessary characters and symbols to otherwise neat code.
- lolive 5y ago
  Funnily, I am about to accept a job to semantize some python code where nobody knows exactly what all those float, int, string and arrays in all those functions really correspond to. [note: i am a java developper :)]
  - IshKebab 5y ago
    Good luck! Adding type hints to Python code that was written without them is extremely difficult.
  - deltaonefour 5y ago
    I'm a python developer and within the last couple years I've encountered some interviews where they explicitly told me not to code in python. Its because it's the language of choice for bad programmers who dont learn languages with any amount of depth. It's getting associated with mediocrity because it's so easy to pick up and low quality coders turn to it. Your job solidifies this point. Literally, they need to hire someone to type annotate their code base because they don't know how? How low can you go?
    - coward123 5y ago
      > the language of choice for bad programmers Sure, and Java went through this phase too, and I'm sure so have plenty of other languages. It's the stage at which a language has reached mass adoption and it is inevitable, but it doesn't mean there's anything wrong with the language.
    - lolive 5y ago
      The problem they face is that they need to chain function calls, and spend ages discovering that a given function returns speed in m/s, whereas the next one expects miles/hour as an input. That is just a small example of the issue they face (that could solved by other means than statically typing the Python code), they really want to make things more robust.
carapace 5y ago
The first time I tried typing some Python code was a little experimental HTML generator, pure functional style, that used the keyword args dict as a kind of miscellaneous grab-bag to pass functions and data to sub-functions. You set up a dict and called a top-level function with it and then it worked like a kind of template system, but again all through functions & applications of functions rather than "mail merge" of templates, etc. It was simple, elegant, perfectly cromulent Python code. However, there was no way to specify the type of the dicts because they were so heterogeneous. - - - - I have to say, I was a Python fan for ages, used it professionally for over a decade, so this isn't an outsider's opinion: Python is kind of over. It won't die any time soon, but it has been "improved" out of its sweet spot of applicability, and no longer has a compelling story for adoption going forward. Every niche Python is known for serving well now has even better options: - Rapid Application Development: Nim, Red - Readability: Go, Nim, D, Zig - Science: Julia - Large multi-dev projects: Java, Go - Systems programming (python glue): D, Rust, Zig - Web app backends: Erlang/Elixir And so on... Really, as much as I still love Python (and I do) the only things I feel it's appropriate for these days are small scripts that need to remain flexible (in other words, that are likely to be modified often), things that are too complex to be a CLI command, but less complex or more dynamic than, say, an email client or RSS reader. If the type domain of the program doesn't change rapidly or unpredictably then you don't need Python's dynamic typing. If you need static typing I think you should use a language that supports that out-of-the-box, not tacked on later.
- oblvious-earth 5y ago
  I think what is missing from all of those languages you suggest is them providing an intersection of those niches. Yes I am quite sure for any specific niche there is a language that is better suited than Python (though I would argue with a lot of your choices there). But let's say I want a readable language that is good at data science, web backend, and provides a system glue. Python handles that much better than all the others you mentioned. Also the one choice I am going to explicitly call out is Rust as a good systems glue language. In fact a very common practise I see now is all the good Rust libraries are getting Python bindings and made available on PyPI so all that power of Rust for systems programming can be used in Python as the glue language.
plainnoodles 5y ago
Python type hints are such a huge boon to the language. I worked in what I would imagine was a pretty typical mature medium-size python codebase. Typically, it was never too hard to figure out what a type was. Sometimes types were passed through several layers of functions and had the same short variable name through all of them - quite a few times I had to click around quite a bit to figure out what "f" was. Especially when I could infer it was some kind of file-ish thing but whether it was a file object, a string, or some composite type that held one of the two as a member, or what?! was sometimes difficult to determine, especially if the actual use of the parameter wasn't immediately obvious either. When we updated to python 3, I started leaving type hints both in old code as I answered these questions for myself and new code as I created what otherwise would've been new questions for my future self. And I noticed two things happened: * annoyance went down as I fixed the most well-trod of these problems * I became less averse to using slightly more complicated types I think python pushes you to keep things really simple and not use custom types unless really necessary. This is, overall, good? I do think Java developers, for instance (of which I consider myself one), generally reach to create datatypes that are basically just a simpler wrapper or a 2-tuple of collections or other primitives, and it can make the code a bit annoying to grok. But the trouble is, in un-type-hinted python, I already start getting nervous about things like: [('2022-03-18', "something"), ('2022-03-19', "something else")]. And if your data content doesn't make it obvious (or at least somewhat guessable) what it is, it can make it hard to grok in a slightly worse way than the Java code would be. In python 2 I'd usually make a namedtuple in these situations, but oftentimes I felt a bit weird there because I'd usually reach for it in lightweight situations when I feared they were becoming more complex. But finally, in python 3, I feel like I'm generally happy with, in this order: 1. just use plain primitive types. no type hints. we all know what's in dates = ['2022-03-18']. 2. just use a type hint. I feel better about a Tuple[str, Dict[int, int]] if it's type hinted than not. 3. Use a namedtuple. This puts names onto the fields. so maybe my Tuple[str, Dict[int, int]] becomes a MyEntry(token: str, settings: Dict[int, int]) or something. 4. use a fully-fledged custom data type class.
- packetlost 5y ago
  I would recommend using `dataclasses.dataclass` over namedtuple. Namedtuples are factories that generate objects with different identities every time you create one and will not behave like you expect when comparing them. With dataclasses you can type-annotate fields as well as generate various special functions just as args to `@dataclass` so you get better safety, immutable objects (when using `frozen=True`), `__eq__` generation, `asdict` to serialize complex objects (recursively!), and a bunch of other great stuff. I come from a massive hard-realtime system codebase that's mostly in Python and there's a lot of moving parts and complex data types. Type hints, `typing.Protocol`, and `dataclass` are all godsends for having any sort of sane, human parseable structure to the code. Being able to navigate to type definitions with `gd`/ctrl + click is massively helpful.
progval 5y ago
> On the other hand, MyPy found that only 15% of the 2,678 repositories examined were type-correct Not surprising. In my experience, upgrading or downgrading Mypy always breaks type-checking on real code bases: older versions because they fail to infer types or have bugs, and newer versions because they are stricter or error on '# type: ignore' that are now useless (but were needed to work around bugs that are now fixed). And there are 10 Mypy version released over the last 12 months, so projects probably used different versions than the one used by the authors of the paper. While the article does not mention when they downloaded the repositories, I am guessing they started in August 2019, given section 3.1 (with no specified end date, besides the paper's publication in November 2020). And the single Mypy version they tried (0.770) was released in March 2020.
Bostonian 5y ago
I learned Python before it had type hints. What books cover this topic well?
- clawlor 5y ago
  I don't know of any books, but I've found this Pycon talk to be useful: https://www.youtube.com/watch?v=pMgmKJyWKn8 https://www.youtube.com/watch?v=pMgmKJyWKn8
- sigzero 5y ago
  Not a book but: https://realpython.com/python-data-classes/ https://realpython.com/python-data-classes/
- digisign 5y ago
  It may be time for a book, but it has been changing so rapidly I'd not want a hardcopy yet.
qalmakka 5y ago
Python 3 types are very useful, especially when paired with something like PyLance. I started putting them everywhere and that's been a big boon to my productivity when writing Python. Python is not Perl, it's very strongly typed, has operator overloading and close to no "special syntax" (i.e. Perl's sigils, etc), so it's kinda hard to follow code sometimes. `typing` helps a lot.

68 points by zdw 5y ago

> which allow developers to say that a function returns Dict[List[Set[FrozenSet[int]]], str] this type is impossible in Python

mivade 5y ago
Correct. Keys must be immutable and hashable.
- pdonis 5y ago
  Technically keys only have to be hashable, but an object that is hashable but not immutable is not a good idea and in practice the two will go together.
gnulinux 5y ago
Not only impossible in python, but also seems conceptually useless (why would you have a set of frozen sets?).
- yifanl 5y ago
  Ensure uniqueness, i guess? Though you'd lose that when you make the list of sets. I could maybe see something like this as an intermediary result of parsing some horrible, nested obscure message format (except the dict, that baffles me unless the key-value types are reversed?)
- raymondh 5y ago
  Almost the entire purpose of frozensets was so that you could have sets of frozensets. It's the only way to have nested sets. Frozensets also work as dictionary keys.
- deltaonefour 5y ago
  This is not useless. Why would you have a list of lists? A list is just an iterable container class, which is what a set is. If the need exists for a list of lists then it exists for a set of sets. An example of this is finding all combinations possible non ordered groups of letters given a collection of letters: abadfjiik. A typical algorithm will produce dupes. But if your accumulator is a set of frozen sets you can produce a result with no dupes.
- nisegami 5y ago
  I guarantee something like that is running in production somewhere in the world (maybe without the list part).
lolive 5y ago
With this type system is it possible to subclass string and int, to enforce types for, let’s say, firstName, postalCode, etc.
- paisawalla 5y ago
  Yes, simply FirstNameType = str (But don't use ints for postal codes, even if the postal codes are numeric)
  - lolive 5y ago
    Huuum. If I sign a function so it only accepts a FirstNameType, and I use the statement you propose, won’t any string be acceptable as an input parameter, from the type system point of view?
    - jamincan 5y ago
      Yeah, that is the syntax for a type alias. You can use NewType to create an actual subclass of the type instead. https://docs.python.org/3/library/typing.html#newtype https://docs.python.org/3/library/typing.html#newtype
    - paisawalla 5y ago
      In that case, you need a subclass of string that does nothing but has its type as FirstNameType. Wherever you create or return strings that are first names, you'll need to instantiate FirstNameType instead.
  - colejohnson66 5y ago
    > ... even if the postal codes are numeric Maybe in the US. In Canada, they are 6 characters of letter-digit-letter-digit-letter-digit (in regex: /((?:[A-Z][0-9]){3})/)
    - kennu 5y ago
      And in the future, they could be anything. Finland expanded the allowed characters in person identification numbers to support people born after 2000. Usually bad idea to be too strict in checking fields where people manually enter text.
lolive 5y ago
Does IntelliJ use any kind of static type analysis to help the user autocomplete properly? [== do we have something like Java???]
- deltaonefour 5y ago
  Yes. Try pycharm.
  - aenis 5y ago
    Within reason. I use type annotations solely as means of documenting code, with the added benefit of having autosuggest work all of the time. I am one of those guys who put a lot of comments in their code.
    - deltaonefour 5y ago
      I strictly use type annotations in python as if python was a typed language. This is within reason as typed languages exist.
nickysielicki 5y ago
This bugged me, too. You can have a dictionary which maps to strings, you can have a list containing sets of frozensets of ints, but you can’t have a dictionary of unhashable types.
- mturmon 5y ago
  Yep, here's how it plays out: Python 3.8.5 (default, Sep 4 2020, 02:22:02) [Clang 10.0.0 ] :: Anaconda, Inc. on darwin >>> d=dict() >>> l=[1,2,3] >>> d[l]=42 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unhashable type: 'list'
  - setr 5y ago
    https://pypi.org/project/forbiddenfruit/ https://pypi.org/project/forbiddenfruit/ Apparently there are ways to monkey patch core types, so you could probably add a __hash__ method to list…
bobbylarrybobby 5y ago
While it's impossible for this type to exist in python, will a type checker complain if you write that type out? Does it have the notion that the first type of `Dict` (or just `dict` in Python 3.9+, iirc) must be `Hashable`

kodablah 5y ago

> stripped annotations out of files and askedPyType to infer them, it failed to do so in 77% of cases [...] MyPy found that only 15% of the 2,678 repositories examined were type-correct; this may be a result of MyPy being very conservative and producing false positives The paper referenced is a year and a half old and type checkers have fixed/added a lot since then. Also, I'd guess that more projects use Mypy than Pytype and fix errors reported, so that may be another reason more pass Mypy even without annotations (though conservatism does play).

aitchnyu 5y ago
Anybody swears by Pytype? I'm puzzled by it existing between MyPy(static) and unit tests (dynamic).
- whimsicalism 5y ago
  Wdym "between"? Pytype is another static analyzer.

deleted 5y ago

[deleted]

assbuttbuttass 5y ago

Python type annotations are more useful for a human as a form of documentation, than they ever will be to a machine. I think Python completely ignores them at runtime, and tools like pytype or mypy will never capture all the complicated hacks possible in the Python type system (as evidenced by only 15% of repos passing mypy)

IshKebab 5y ago
This is only because the Python type hinting ecosystem is so bad. Tons of projects don't have type hints, there are multiple conflicting interpretations of the types, MyPy is very buggy (Pyright is much much better but few people use it). Compare it to Typescript which is a similar sort of tacked on type hinting system - Typescript is basically always machine checked.
goodoldneon 5y ago
Python does ignore them at runtime, but as long as you're validating your inputs (e.g. using Pydantic for HTTP request validation) then you can trust that the static types are truthful
- musingsole 5y ago
  > trust that the static types are truthful That's often a mistake even in typed languages like typescript. Satisfying a static type analyzer isn't enough. Real data is whatever it is going to be. If your types are meaningless at runtime, you can easily encounter type mismatches. Static typing as implemented in typescript and python type hints is a tool for engineers instead of a tool for systems.
  - roywiggins 5y ago
    The types are usually ignored, but not entirely erased- Pydantic uses type hints to do validation at runtime, so if you construct a Pydantic object with invalid data it will throw an error, so eg foo=PydanticFoo(**request.json()) Will try and validate a json request against the implied schema provided by the type hints on PydanticFoo while constructing it and will throw if it fails.

ohmanjjj 5y ago

After many years of writing Python code, I just can’t stand the distraction and messiness that Types results in and the appearance of clutter and unnecessary characters and symbols to otherwise neat code.

lolive 5y ago
Funnily, I am about to accept a job to semantize some python code where nobody knows exactly what all those float, int, string and arrays in all those functions really correspond to. [note: i am a java developper :)]
- IshKebab 5y ago
  Good luck! Adding type hints to Python code that was written without them is extremely difficult.
- deltaonefour 5y ago
  I'm a python developer and within the last couple years I've encountered some interviews where they explicitly told me not to code in python. Its because it's the language of choice for bad programmers who dont learn languages with any amount of depth. It's getting associated with mediocrity because it's so easy to pick up and low quality coders turn to it. Your job solidifies this point. Literally, they need to hire someone to type annotate their code base because they don't know how? How low can you go?
  - coward123 5y ago
    > the language of choice for bad programmers Sure, and Java went through this phase too, and I'm sure so have plenty of other languages. It's the stage at which a language has reached mass adoption and it is inevitable, but it doesn't mean there's anything wrong with the language.
  - lolive 5y ago
    The problem they face is that they need to chain function calls, and spend ages discovering that a given function returns speed in m/s, whereas the next one expects miles/hour as an input. That is just a small example of the issue they face (that could solved by other means than statically typing the Python code), they really want to make things more robust.

carapace 5y ago

The first time I tried typing some Python code was a little experimental HTML generator, pure functional style, that used the keyword args dict as a kind of miscellaneous grab-bag to pass functions and data to sub-functions. You set up a dict and called a top-level function with it and then it worked like a kind of template system, but again all through functions & applications of functions rather than "mail merge" of templates, etc. It was simple, elegant, perfectly cromulent Python code. However, there was no way to specify the type of the dicts because they were so heterogeneous. - - - - I have to say, I was a Python fan for ages, used it professionally for over a decade, so this isn't an outsider's opinion: Python is kind of over. It won't die any time soon, but it has been "improved" out of its sweet spot of applicability, and no longer has a compelling story for adoption going forward. Every niche Python is known for serving well now has even better options: - Rapid Application Development: Nim, Red - Readability: Go, Nim, D, Zig - Science: Julia - Large multi-dev projects: Java, Go - Systems programming (python glue): D, Rust, Zig - Web app backends: Erlang/Elixir And so on... Really, as much as I still love Python (and I do) the only things I feel it's appropriate for these days are small scripts that need to remain flexible (in other words, that are likely to be modified often), things that are too complex to be a CLI command, but less complex or more dynamic than, say, an email client or RSS reader. If the type domain of the program doesn't change rapidly or unpredictably then you don't need Python's dynamic typing. If you need static typing I think you should use a language that supports that out-of-the-box, not tacked on later.

oblvious-earth 5y ago
I think what is missing from all of those languages you suggest is them providing an intersection of those niches. Yes I am quite sure for any specific niche there is a language that is better suited than Python (though I would argue with a lot of your choices there). But let's say I want a readable language that is good at data science, web backend, and provides a system glue. Python handles that much better than all the others you mentioned. Also the one choice I am going to explicitly call out is Rust as a good systems glue language. In fact a very common practise I see now is all the good Rust libraries are getting Python bindings and made available on PyPI so all that power of Rust for systems programming can be used in Python as the glue language.

plainnoodles 5y ago

Python type hints are such a huge boon to the language. I worked in what I would imagine was a pretty typical mature medium-size python codebase. Typically, it was never too hard to figure out what a type was. Sometimes types were passed through several layers of functions and had the same short variable name through all of them - quite a few times I had to click around quite a bit to figure out what "f" was. Especially when I could infer it was some kind of file-ish thing but whether it was a file object, a string, or some composite type that held one of the two as a member, or what?! was sometimes difficult to determine, especially if the actual use of the parameter wasn't immediately obvious either. When we updated to python 3, I started leaving type hints both in old code as I answered these questions for myself and new code as I created what otherwise would've been new questions for my future self. And I noticed two things happened: * annoyance went down as I fixed the most well-trod of these problems * I became less averse to using slightly more complicated types I think python pushes you to keep things really simple and not use custom types unless really necessary. This is, overall, good? I do think Java developers, for instance (of which I consider myself one), generally reach to create datatypes that are basically just a simpler wrapper or a 2-tuple of collections or other primitives, and it can make the code a bit annoying to grok. But the trouble is, in un-type-hinted python, I already start getting nervous about things like: [('2022-03-18', "something"), ('2022-03-19', "something else")]. And if your data content doesn't make it obvious (or at least somewhat guessable) what it is, it can make it hard to grok in a slightly worse way than the Java code would be. In python 2 I'd usually make a namedtuple in these situations, but oftentimes I felt a bit weird there because I'd usually reach for it in lightweight situations when I feared they were becoming more complex. But finally, in python 3, I feel like I'm generally happy with, in this order: 1. just use plain primitive types. no type hints. we all know what's in dates = ['2022-03-18']. 2. just use a type hint. I feel better about a Tuple[str, Dict[int, int]] if it's type hinted than not. 3. Use a namedtuple. This puts names onto the fields. so maybe my Tuple[str, Dict[int, int]] becomes a MyEntry(token: str, settings: Dict[int, int]) or something. 4. use a fully-fledged custom data type class.

packetlost 5y ago
I would recommend using `dataclasses.dataclass` over namedtuple. Namedtuples are factories that generate objects with different identities every time you create one and will not behave like you expect when comparing them. With dataclasses you can type-annotate fields as well as generate various special functions just as args to `@dataclass` so you get better safety, immutable objects (when using `frozen=True`), `__eq__` generation, `asdict` to serialize complex objects (recursively!), and a bunch of other great stuff. I come from a massive hard-realtime system codebase that's mostly in Python and there's a lot of moving parts and complex data types. Type hints, `typing.Protocol`, and `dataclass` are all godsends for having any sort of sane, human parseable structure to the code. Being able to navigate to type definitions with `gd`/ctrl + click is massively helpful.

progval 5y ago

> On the other hand, MyPy found that only 15% of the 2,678 repositories examined were type-correct Not surprising. In my experience, upgrading or downgrading Mypy always breaks type-checking on real code bases: older versions because they fail to infer types or have bugs, and newer versions because they are stricter or error on '# type: ignore' that are now useless (but were needed to work around bugs that are now fixed). And there are 10 Mypy version released over the last 12 months, so projects probably used different versions than the one used by the authors of the paper. While the article does not mention when they downloaded the repositories, I am guessing they started in August 2019, given section 3.1 (with no specified end date, besides the paper's publication in November 2020). And the single Mypy version they tried (0.770) was released in March 2020.

Bostonian 5y ago

I learned Python before it had type hints. What books cover this topic well?

clawlor 5y ago
I don't know of any books, but I've found this Pycon talk to be useful: https://www.youtube.com/watch?v=pMgmKJyWKn8 https://www.youtube.com/watch?v=pMgmKJyWKn8
sigzero 5y ago
Not a book but: https://realpython.com/python-data-classes/ https://realpython.com/python-data-classes/
digisign 5y ago
It may be time for a book, but it has been changing so rapidly I'd not want a hardcopy yet.

qalmakka 5y ago

Python 3 types are very useful, especially when paired with something like PyLance. I started putting them everywhere and that's been a big boon to my productivity when writing Python. Python is not Perl, it's very strongly typed, has operator overloading and close to no "special syntax" (i.e. Perl's sigils, etc), so it's kinda hard to follow code sometimes. `typing` helps a lot.