3 ms·

> Hitting a type annotation when reading Python forces my brain into little backtracking loops which hugely diminishes my ability to form a mental model of the

> Hitting a type annotation when reading Python forces my brain into little backtracking loops which hugely diminishes my ability to form a mental model of the code from a quick read. This makes me think you've never dealt with untyped Python code in a large production repo. The frustration seeps in the hundredth time you encounter an untyped `get_user_ids(...)` written by a co-worker (or yourself in the past), and wonder "Hmm, is this going to return a list of ints? Strings? UUIDs? A generator of those things?" and then you have to dig into the function to find out what it's actually going to give you, and then each function inside of that will have the same issues, and pretty soon you're building a mental model of a call stack to discern types anyway, you'll be happy to embrace the type system of python. "Readability" doesn't matter if you're unable to simultaneously quickly build a mental model of what is actually happening in the code. Sure, `def get_user_ids(...)` is super readable in that I quickly understand "this is going to give me some user ids", but that's useless when you consider the immediate next step of "what format are they in, so I know what I can do with them afterwards". `def get_user_ids(...) -> Optional[Iterable[int]]` is infinitely more useful, because I know that I can't use the function unchecked inside a list comprehension, because it might return `None`. And if it doesn't return `None`, I know I have, for example, integer user ids which I can compare with `==`, vs a custom type which might not implement `__eq__`.

j4mie 4y ago
> This makes me think you've never dealt with untyped Python code in a large production repo. I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). I do understand the arguments and examples in the rest of your post, and intuitively they make sense, but practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. > you have to dig into the function to find out what it's actually going to give you This is exactly what I do, and it's just fine. I find it more difficult to build a mental model of what's happening in type-annotated code. Like I said, it's subjective. That's why I find "types improve readability" arguments in posts such as this to be problematic. They are stated as basic axioms with no supporting data. It's easier to read for you.
- PurpleRamen 4y ago
  > I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). 100k is not small, but also not particular large; even for one single app with highly interweaved code. My companies main codebase is around 800k lines and I consider it as middle sized. But lines of code are not the problem. The length of your code paths is it. Understanding the flow of one or two functions, ok, not that hard of a problem. Having your data flowing through some dozen functions, big problem. Complexity kills any understanding at some point. Or you need to invest unnecessary much time for it.
- a4a4a4a4 4y ago
  > It's easier to read for you. It would be easier for me to read if everyone wrote their code in French. Is that a good enough reason to make the whole company switch? There's subjectivity, and then there is "I'm not frustrated by having to dig for the return type of every function I call, so no one should using typing in their function signatures because I don't like to look at it". > I run an agency ... up to ~100k lines of code each ... practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. You're either a genius who is able to maintain a huge graph of functions calls and typing in your mind, across multiple repos and many years, or your codebase is an absolute nightmare to work on (it could also be both, and you're just able to deal with it). If I interviewed at a company in 2022 and they told me they have multiple 100k+ line repos of untyped Python, I would run away as fast as possible, as would the coworkers that I value the most. The ones that wouldn't run away are the ones that think types are inconvenient and unit tests are annoying to write, which really aren't the people I enjoy working with. Edit: if you want evidence, I pulled some examples from your profile. I've never worked with Django, and let's pretend I'm onboarding at your company and need to get into your `django_dbq` repo. You have a `Job.get_queue_depths()` method here: https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66e9a46ca97e5fdd16fcdc67065e57/django_dbq/models.py#L143 https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66.... What does this return? Okay... it builds this `annotation_dicts` object from the objects in a `Job`, then filters them, gets the queue names, sorts them, then annotates them (?). `annotation_dicts` also seems to start with a `JobManager`, and I can't tell how/if it ends up actually being a `dict` of something, so I'm not sure if the name is right. Then we return a dict of name:queue_depth, okay good. I guess the keys to the dict are strings?... Wrong, they're `models.CharField`, and I only can find that because it happens to be in the same file/class. If the function was typed, and mypy enforced, I would have `def get_queue_depths() -> Dict[models.Charfield, int]:` (assuming the value is an `int`, I also can't verify the `.annotate(Count(...))` returns an `int`...). This would have saved me all of that above, which is basically just doing static analysis in my head, and not adding any value to the business.
Thorentis 4y ago
This is what comment blocks are for. Putting the return type in the funcfions docstring should be a requirement during linting, and ensuring the correct type is used should be part of PR review. Easy to find, easy to read, no static type checker required.
- manfre 4y ago
  Ensuring consistency and long term accuracy of the docstring is more effort and more verbose than type annotations.
- a4a4a4a4 4y ago
  > Putting the return type in the funcfions docstring should be a requirement during linting, and ensuring the correct type is used should be part of PR review. You're suggesting pushing the types from the function signature + docstring into only the docstring, so that we can offload the static type-checking from mypy to the reviewers of the PR? Why? This is at best exactly as good as mypy (100% accurate and repeatable) and at worst (and most likely) will lead to human reviewers making mistakes/not catching edge cases/forgetting to update docstrings.
- noitpmeder 4y ago
  If I interviewed a python developer and he said anything close to what your post contains I would veto them on the spot.

j4mie 4y ago

> This makes me think you've never dealt with untyped Python code in a large production repo. I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). I do understand the arguments and examples in the rest of your post, and intuitively they make sense, but practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. > you have to dig into the function to find out what it's actually going to give you This is exactly what I do, and it's just fine. I find it more difficult to build a mental model of what's happening in type-annotated code. Like I said, it's subjective. That's why I find "types improve readability" arguments in posts such as this to be problematic. They are stated as basic axioms with no supporting data. It's easier to read for you.

PurpleRamen 4y ago
> I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). 100k is not small, but also not particular large; even for one single app with highly interweaved code. My companies main codebase is around 800k lines and I consider it as middle sized. But lines of code are not the problem. The length of your code paths is it. Understanding the flow of one or two functions, ok, not that hard of a problem. Having your data flowing through some dozen functions, big problem. Complexity kills any understanding at some point. Or you need to invest unnecessary much time for it.
a4a4a4a4 4y ago
> It's easier to read for you. It would be easier for me to read if everyone wrote their code in French. Is that a good enough reason to make the whole company switch? There's subjectivity, and then there is "I'm not frustrated by having to dig for the return type of every function I call, so no one should using typing in their function signatures because I don't like to look at it". > I run an agency ... up to ~100k lines of code each ... practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. You're either a genius who is able to maintain a huge graph of functions calls and typing in your mind, across multiple repos and many years, or your codebase is an absolute nightmare to work on (it could also be both, and you're just able to deal with it). If I interviewed at a company in 2022 and they told me they have multiple 100k+ line repos of untyped Python, I would run away as fast as possible, as would the coworkers that I value the most. The ones that wouldn't run away are the ones that think types are inconvenient and unit tests are annoying to write, which really aren't the people I enjoy working with. Edit: if you want evidence, I pulled some examples from your profile. I've never worked with Django, and let's pretend I'm onboarding at your company and need to get into your `django_dbq` repo. You have a `Job.get_queue_depths()` method here: https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66e9a46ca97e5fdd16fcdc67065e57/django_dbq/models.py#L143 https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66.... What does this return? Okay... it builds this `annotation_dicts` object from the objects in a `Job`, then filters them, gets the queue names, sorts them, then annotates them (?). `annotation_dicts` also seems to start with a `JobManager`, and I can't tell how/if it ends up actually being a `dict` of something, so I'm not sure if the name is right. Then we return a dict of name:queue_depth, okay good. I guess the keys to the dict are strings?... Wrong, they're `models.CharField`, and I only can find that because it happens to be in the same file/class. If the function was typed, and mypy enforced, I would have `def get_queue_depths() -> Dict[models.Charfield, int]:` (assuming the value is an `int`, I also can't verify the `.annotate(Count(...))` returns an `int`...). This would have saved me all of that above, which is basically just doing static analysis in my head, and not adding any value to the business.

Thorentis 4y ago

This is what comment blocks are for. Putting the return type in the funcfions docstring should be a requirement during linting, and ensuring the correct type is used should be part of PR review. Easy to find, easy to read, no static type checker required.

manfre 4y ago
Ensuring consistency and long term accuracy of the docstring is more effort and more verbose than type annotations.
a4a4a4a4 4y ago
> Putting the return type in the funcfions docstring should be a requirement during linting, and ensuring the correct type is used should be part of PR review. You're suggesting pushing the types from the function signature + docstring into only the docstring, so that we can offload the static type-checking from mypy to the reviewers of the PR? Why? This is at best exactly as good as mypy (100% accurate and repeatable) and at worst (and most likely) will lead to human reviewers making mistakes/not catching edge cases/forgetting to update docstrings.
noitpmeder 4y ago
If I interviewed a python developer and he said anything close to what your post contains I would veto them on the spot.