4 ms·

The more I think about it, the more I think that the controversy around static type annotations in Python boils down to this: Improved readability This is v

by j4mie 4y ago

The more I think about it, the more I think that the controversy around static type annotations in Python boils down to this: Improved readability This is very subjective, and is particularly sensitive in a language like Python which (rightly) has such a strong historical emphasis on readability above almost anything else. My personal opinion is that static type annotations are extremely detrimental to readability. They add jarring line noise that makes reading Python much less like reading English. Hitting a type annotation when reading Python forces my brain into little backtracking loops which hugely diminishes my ability to form a mental model of the code from a quick read. I wonder if people who come to Python from other languages (that are already statically typed) are accustomed to the poor comprehension introduced by types, and so don't experience this drawback.

tajd 4y ago
I came from Python to Scala and I'm not sure I'd move back in a hurry. I like types annotations as it helps me understand what exactly is being passed between functions - sometimes that's no always clear from the code. I think "readability" is subjective so I just thought I'd throw my own opinion in there.
allendoerfer 4y ago
I want to develop Python at 80 characters per line, inside vim. I enjoy the elegance and simplicity and frankly the whole culture around it. Libraries, ecosystems and actual business reasons that matter aside, if I cannot have that, I would rather use something like Kotlin instead. PHP does it better. Types are an add-on, but integrated into the interpreter. Also it has never been pretty in the first place, so nothing was lost.
a4a4a4a4 4y ago
> Hitting a type annotation when reading Python forces my brain into little backtracking loops which hugely diminishes my ability to form a mental model of the code from a quick read. This makes me think you've never dealt with untyped Python code in a large production repo. The frustration seeps in the hundredth time you encounter an untyped `get_user_ids(...)` written by a co-worker (or yourself in the past), and wonder "Hmm, is this going to return a list of ints? Strings? UUIDs? A generator of those things?" and then you have to dig into the function to find out what it's actually going to give you, and then each function inside of that will have the same issues, and pretty soon you're building a mental model of a call stack to discern types anyway, you'll be happy to embrace the type system of python. "Readability" doesn't matter if you're unable to simultaneously quickly build a mental model of what is actually happening in the code. Sure, `def get_user_ids(...)` is super readable in that I quickly understand "this is going to give me some user ids", but that's useless when you consider the immediate next step of "what format are they in, so I know what I can do with them afterwards". `def get_user_ids(...) -> Optional[Iterable[int]]` is infinitely more useful, because I know that I can't use the function unchecked inside a list comprehension, because it might return `None`. And if it doesn't return `None`, I know I have, for example, integer user ids which I can compare with `==`, vs a custom type which might not implement `__eq__`.
- j4mie 4y ago
  > This makes me think you've never dealt with untyped Python code in a large production repo. I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). I do understand the arguments and examples in the rest of your post, and intuitively they make sense, but practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. > you have to dig into the function to find out what it's actually going to give you This is exactly what I do, and it's just fine. I find it more difficult to build a mental model of what's happening in type-annotated code. Like I said, it's subjective. That's why I find "types improve readability" arguments in posts such as this to be problematic. They are stated as basic axioms with no supporting data. It's easier to read for you.
  - PurpleRamen 4y ago
    > I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). 100k is not small, but also not particular large; even for one single app with highly interweaved code. My companies main codebase is around 800k lines and I consider it as middle sized. But lines of code are not the problem. The length of your code paths is it. Understanding the flow of one or two functions, ok, not that hard of a problem. Having your data flowing through some dozen functions, big problem. Complexity kills any understanding at some point. Or you need to invest unnecessary much time for it.
  - a4a4a4a4 4y ago
    > It's easier to read for you. It would be easier for me to read if everyone wrote their code in French. Is that a good enough reason to make the whole company switch? There's subjectivity, and then there is "I'm not frustrated by having to dig for the return type of every function I call, so no one should using typing in their function signatures because I don't like to look at it". > I run an agency ... up to ~100k lines of code each ... practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. You're either a genius who is able to maintain a huge graph of functions calls and typing in your mind, across multiple repos and many years, or your codebase is an absolute nightmare to work on (it could also be both, and you're just able to deal with it). If I interviewed at a company in 2022 and they told me they have multiple 100k+ line repos of untyped Python, I would run away as fast as possible, as would the coworkers that I value the most. The ones that wouldn't run away are the ones that think types are inconvenient and unit tests are annoying to write, which really aren't the people I enjoy working with. Edit: if you want evidence, I pulled some examples from your profile. I've never worked with Django, and let's pretend I'm onboarding at your company and need to get into your `django_dbq` repo. You have a `Job.get_queue_depths()` method here: https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66e9a46ca97e5fdd16fcdc67065e57/django_dbq/models.py#L143 https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66.... What does this return? Okay... it builds this `annotation_dicts` object from the objects in a `Job`, then filters them, gets the queue names, sorts them, then annotates them (?). `annotation_dicts` also seems to start with a `JobManager`, and I can't tell how/if it ends up actually being a `dict` of something, so I'm not sure if the name is right. Then we return a dict of name:queue_depth, okay good. I guess the keys to the dict are strings?... Wrong, they're `models.CharField`, and I only can find that because it happens to be in the same file/class. If the function was typed, and mypy enforced, I would have `def get_queue_depths() -> Dict[models.Charfield, int]:` (assuming the value is an `int`, I also can't verify the `.annotate(Count(...))` returns an `int`...). This would have saved me all of that above, which is basically just doing static analysis in my head, and not adding any value to the business.
rcthompson 4y ago
Of course it takes longer to read and comprehend the method signature when there are types in it, since it makes the method signature longer. But by the time I make it to the end of the signature, before I've even taken a look at the function body, the types have already given me important information (assuming this is a codebase where MyPy types are enforced) that will make reading the code much easier, because I don't have the mental overhead of needing to infer the types of half a dozen arguments and the return type while reading the code. And often, the type signature is the information I was looking for anyway, and as long as I trust the function to be implemented correctly, I can skip reading the body entirely. For instance, if I see a function signature like "def parse_int(x: str) -> int", I can probably guess what the function will do. It might be parsing the entire string as an integer, or it might be parsing the first integer it can find within that string, but that type signature by itself eliminates a lot of weird things that could plausibly fall within the purview of a function named "parse_int". It can't return an iterable over all integers parsed from the string; it can't conditionally return an int if there's a single integer and a list of ints if there's more than one; it only accepts a decoded string (i.e. not bytes or any other string representation); it can't take a list of strings and return the first integer it finds; it can't return a string representation of the parsed integer; it can't return None if parsing fails (meaning it will likely throw an exception instead); and so on. Thus, even if I need to read the function body to see exactly what it's doing, I go into it thinking "in what specific way is this function parsing a single integer from a single string?" rather than "I wonder what this function does".
LtWorf 4y ago
I've seen libraries whose functions only accept `*kwargs` and you need to open the documentation (or read the body of the function) to be able to use that function. If you know how many parameters it's easier. If parameters have names that you can understand it's even easier. If parameters even let you know what kind of value they want it's also easier. When I read python I'm not reading a novel. I want to know how to use a function, and possibly I want to know that without reading the function. So it might make 1 line harder, but makes me skip reading 50 lines.

by j4mie 4y ago

tajd 4y ago

I came from Python to Scala and I'm not sure I'd move back in a hurry. I like types annotations as it helps me understand what exactly is being passed between functions - sometimes that's no always clear from the code. I think "readability" is subjective so I just thought I'd throw my own opinion in there.

allendoerfer 4y ago

I want to develop Python at 80 characters per line, inside vim. I enjoy the elegance and simplicity and frankly the whole culture around it. Libraries, ecosystems and actual business reasons that matter aside, if I cannot have that, I would rather use something like Kotlin instead. PHP does it better. Types are an add-on, but integrated into the interpreter. Also it has never been pretty in the first place, so nothing was lost.

a4a4a4a4 4y ago

> Hitting a type annotation when reading Python forces my brain into little backtracking loops which hugely diminishes my ability to form a mental model of the code from a quick read. This makes me think you've never dealt with untyped Python code in a large production repo. The frustration seeps in the hundredth time you encounter an untyped `get_user_ids(...)` written by a co-worker (or yourself in the past), and wonder "Hmm, is this going to return a list of ints? Strings? UUIDs? A generator of those things?" and then you have to dig into the function to find out what it's actually going to give you, and then each function inside of that will have the same issues, and pretty soon you're building a mental model of a call stack to discern types anyway, you'll be happy to embrace the type system of python. "Readability" doesn't matter if you're unable to simultaneously quickly build a mental model of what is actually happening in the code. Sure, `def get_user_ids(...)` is super readable in that I quickly understand "this is going to give me some user ids", but that's useless when you consider the immediate next step of "what format are they in, so I know what I can do with them afterwards". `def get_user_ids(...) -> Optional[Iterable[int]]` is infinitely more useful, because I know that I can't use the function unchecked inside a list comprehension, because it might return `None`. And if it doesn't return `None`, I know I have, for example, integer user ids which I can compare with `==`, vs a custom type which might not implement `__eq__`.

j4mie 4y ago
> This makes me think you've never dealt with untyped Python code in a large production repo. I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). I do understand the arguments and examples in the rest of your post, and intuitively they make sense, but practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. > you have to dig into the function to find out what it's actually going to give you This is exactly what I do, and it's just fine. I find it more difficult to build a mental model of what's happening in type-annotated code. Like I said, it's subjective. That's why I find "types improve readability" arguments in posts such as this to be problematic. They are stated as basic axioms with no supporting data. It's easier to read for you.
- PurpleRamen 4y ago
  > I run an agency and we maintain dozens of large, mature production projects (up to ~100k lines of code each). 100k is not small, but also not particular large; even for one single app with highly interweaved code. My companies main codebase is around 800k lines and I consider it as middle sized. But lines of code are not the problem. The length of your code paths is it. Understanding the flow of one or two functions, ok, not that hard of a problem. Having your data flowing through some dozen functions, big problem. Complexity kills any understanding at some point. Or you need to invest unnecessary much time for it.
- a4a4a4a4 4y ago
  > It's easier to read for you. It would be easier for me to read if everyone wrote their code in French. Is that a good enough reason to make the whole company switch? There's subjectivity, and then there is "I'm not frustrated by having to dig for the return type of every function I call, so no one should using typing in their function signatures because I don't like to look at it". > I run an agency ... up to ~100k lines of code each ... practically I have almost never had an experience like the one you describe in 20+ years of working on Python codebases. You're either a genius who is able to maintain a huge graph of functions calls and typing in your mind, across multiple repos and many years, or your codebase is an absolute nightmare to work on (it could also be both, and you're just able to deal with it). If I interviewed at a company in 2022 and they told me they have multiple 100k+ line repos of untyped Python, I would run away as fast as possible, as would the coworkers that I value the most. The ones that wouldn't run away are the ones that think types are inconvenient and unit tests are annoying to write, which really aren't the people I enjoy working with. Edit: if you want evidence, I pulled some examples from your profile. I've never worked with Django, and let's pretend I'm onboarding at your company and need to get into your `django_dbq` repo. You have a `Job.get_queue_depths()` method here: https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66e9a46ca97e5fdd16fcdc67065e57/django_dbq/models.py#L143 https://github.com/dabapps/django-db-queue/blob/23f8ebe80b66.... What does this return? Okay... it builds this `annotation_dicts` object from the objects in a `Job`, then filters them, gets the queue names, sorts them, then annotates them (?). `annotation_dicts` also seems to start with a `JobManager`, and I can't tell how/if it ends up actually being a `dict` of something, so I'm not sure if the name is right. Then we return a dict of name:queue_depth, okay good. I guess the keys to the dict are strings?... Wrong, they're `models.CharField`, and I only can find that because it happens to be in the same file/class. If the function was typed, and mypy enforced, I would have `def get_queue_depths() -> Dict[models.Charfield, int]:` (assuming the value is an `int`, I also can't verify the `.annotate(Count(...))` returns an `int`...). This would have saved me all of that above, which is basically just doing static analysis in my head, and not adding any value to the business.

rcthompson 4y ago

Of course it takes longer to read and comprehend the method signature when there are types in it, since it makes the method signature longer. But by the time I make it to the end of the signature, before I've even taken a look at the function body, the types have already given me important information (assuming this is a codebase where MyPy types are enforced) that will make reading the code much easier, because I don't have the mental overhead of needing to infer the types of half a dozen arguments and the return type while reading the code. And often, the type signature is the information I was looking for anyway, and as long as I trust the function to be implemented correctly, I can skip reading the body entirely. For instance, if I see a function signature like "def parse_int(x: str) -> int", I can probably guess what the function will do. It might be parsing the entire string as an integer, or it might be parsing the first integer it can find within that string, but that type signature by itself eliminates a lot of weird things that could plausibly fall within the purview of a function named "parse_int". It can't return an iterable over all integers parsed from the string; it can't conditionally return an int if there's a single integer and a list of ints if there's more than one; it only accepts a decoded string (i.e. not bytes or any other string representation); it can't take a list of strings and return the first integer it finds; it can't return a string representation of the parsed integer; it can't return None if parsing fails (meaning it will likely throw an exception instead); and so on. Thus, even if I need to read the function body to see exactly what it's doing, I go into it thinking "in what specific way is this function parsing a single integer from a single string?" rather than "I wonder what this function does".

LtWorf 4y ago

I've seen libraries whose functions only accept `*kwargs` and you need to open the documentation (or read the body of the function) to be able to use that function. If you know how many parameters it's easier. If parameters have names that you can understand it's even easier. If parameters even let you know what kind of value they want it's also easier. When I read python I'm not reading a novel. I want to know how to use a function, and possibly I want to know that without reading the function. So it might make 1 line harder, but makes me skip reading 50 lines.