45 ms·

Tests aren’t enough: Case study after adding type hints to urllib3

205 points by quentinp 5y ago

gundamdoubleO 5y ago
I've seen a lot of push back on adding type checking to Python but we had a similar case at my company where we tried it out on a new project and the clarity and readability of the code was immediately beneficial to the entire team. Perhaps it's something well suited to larger codebases.
- hobs 5y ago
  I think it's well suited to anything really - the amount of casual problem solving and inference you can make from some simple types is pretty big in my experience, and Python's approach to allow you optionally buy into it is really nice.
- tester756 5y ago
  It's $current_year and there's still debate whether checking stuff at compilation time is better than at runtime?
  - novok 5y ago
    People who don't think types are a good thing need to work in a statically typed language for a year or two and then see what a difference it makes in reality. Unproductive Java bureaucracy != static typing. I think the people debating it never tried it seriously.
    - fiddlerwoaroof 5y ago
      I’ve done everything from Haskell to Java and I still strongly prefer Clojure and Common Lisp-style dynamic types.
      - lemper 5y ago
        i have a small side project in clojure [1] and i always miss type checking when working on it. not by much because it's a small project but i am tired of iseq is not a function error. [1] https://dactyl.siskam.link https://dactyl.siskam.link
        fiddlerwoaroof 5y ago
        I sort of think there are two mindsets behind this debate: people that miss the guard rails of a static type system and people that enjoy the experience of iterating quickly in a dynamically typed language. I don't really want to say everyone should pick one side or the other, just that my experience doesn't bear out the claim that "statically typed languages produce more maintainable code". And, the little bit of empirical evidence for this proposition is largely inconclusive: https://danluu.com/empirical-pl/ https://danluu.com/empirical-pl/
        simion314 5y ago
        >and people that enjoy the experience of iterating quickly in a dynamically typed language Programmers spend more time reading code then writing it. So I personally prefer the devs in the team will spend more time typing the code or use a bit more brain energy to think about types so later we can all read the code and understand it and edit faster. Dynamic works great for write-only scripts.
        fiddlerwoaroof 5y ago
        People always say this about reading code and it’s just never matched my experience working in either sort of codebase: one difference (comparing lisps and, say, Typescript or Java) is that lisps just have fewer lines to read. So, any assistance you get from the types is counteracted by having to read more code. But, additionally, I just don’t find it true to my experience that it’s easier to read and understand a dynamically typed codebase vs. a statically typed one. Especially when you have a lisp-like environment that makes accurate jump-to-definition possible. EDIT: I think I just tend to think about codebases in terms of operations rather than types. And, consequently, when I build a codebase around compositions of functions, the way I think about it isn’t very different in either paradigm.
    - jonny_eh 5y ago
      After using TypeScript for even a little bit I find it painful to go back to JavaScript for anything more complex than white-boarding.
    - wvenable 5y ago
      A lot of people grew up with Java -- especially early Java -- as their primary language. It was taught heavily in schools. I think it ruined a lot of people to static typing and exceptions because Java is/was terrible for both of those things.
  - kzrdude 5y ago
    That's not really the debate in Python :) Almost every Python user now has to "deal" with type annotations. It's tempting to gradually add type annotations, it's nice documentation. But it also rubs me the wrong way to have annotations that are never checked(!). In many codebases, you might just have "casual" style type annotations in Python, and nothing ever asserts that they hold. That's nagging on me, a bit.
    - inertiatic 5y ago
      Never checked? They're statically checked. Also, tooling like https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/ can do runtime checking for important parts of your app or you can use this https://github.com/agronholm/typeguard https://github.com/agronholm/typeguard to enforce all types at runtime (although I haven't measured the performance impact, probably something to do in a separate environment than production?).
      - kzrdude 5y ago
        They are statically checked, if you run a type checker. Which many don't.
    - tehnub 5y ago
      That's a good point. If they're never checked, then they're just like incorrect/outdated comments. They sort of get at this idea in the article, and sort of describe a compromise for it. They have a list of files that they've completely annotated, and only those files are checked by mypy. So in their case, they know which annotations to ignore, and which they can rely on.
- deleted 5y ago
  [deleted]
- handrous 5y ago
  I want type checking on pretty much anything that will ever exceed about two screenfuls of code. If I can't keep the whole thing in my head at once, I want the computer to do it for me. That's the point, right? Making computers do stuff for us so we don't have to?
  - david422 5y ago
    I kindof think of them as a giant set of unit tests. The compiler/linter etc. can check every variable and every function call to check to make sure you didn't mix up your types, which _will_ blow up at runtime if you got them wrong. So rather than write them all by hand, just get your tools to do it.
    - handrous 5y ago
      I see them as documentation of what I think something means or is, which the computer can check for accuracy (more or less), both as I write and as the codebase changes. That legibility to the computer is what makes them much better than documenting the same thing some other way. Are they out of date? Were they wrong to begin with? The computer will tell me, no action needed on my part. I need to look up something in the context of what I'm reading right now—oh, look, the computer just told me exactly what I needed.
- matsemann 5y ago
  Just wish Python's typing was better. But it's impossible to type hint the crazy "pythonic" code out there. Like the kwargs used to do a Django query.
dado3212 5y ago
Honestly will never go back to languages without type checking, it prevents so many bugs and is a huge help in understanding code you haven’t worked with previously.
- SketchySeaBeast 5y ago
  > is a huge help in understanding code you haven’t worked with previously This is huge for me. As someone who takes on already completed projects, it's a huge help with debugging and understand what's going on without requiring you to know the whole system forward and backwards. Sure, you still need to build a mental map of the general code flow, but you can look at a single function and clearly see the obvious inputs and outputs. Combine that with a a stack trace and you can debug that method as a single unit and then start to look at where it's called and what its downstream effects are. You don't need to start from the very beginning of the call and then follow it through, keeping mental track of what is available and in what form when and where.
- seiferteric 5y ago
  It is kind of ridiculous not to have types. I think in the old days handling types felt to heavy for scripting languages, but now with type inference and stuff I don't think it is any longer.
- handrous 5y ago
  > Honestly will never go back to languages without type checking, it prevents so many bugs and is a huge help in understanding code you haven’t worked with previously. I see static types as one of the most powerful communication tools around, as far as code goes. I can't relate at all to people complaining that they waste time. They must work very differently from how I do, is all I can figure. It's that, or they don't realize how much time they're losing to communication-related tasks, or refactoring, or writing (and maintaining!) extra or more verbose tests, or having even one more bug per year make it to production, or whatever, that'd be saved by static types, so aren't correctly accounting for the time savings. One of the two.
  - adrianmonk 5y ago
m12k 5y ago
I've only been in the industry for ~15 years, but it still feels like every year, some ecosystem discovers the value of something that another ecosystem has taken for granted for decades - type-checking, immutability, unidirectional data-flow, AOT-compilation, closures, pure functions, you name it. I'm glad we seem to be converging on a set of best practices as an industry, but sometimes I wish we were spending less time rediscovering the wheel and more time building on top of and adding to the actual state of the art.
- axiosgunnar 5y ago
  I think the problem is to figure out what the best practices actually are. What we are observing here is „the market fixing it“. The process is messy and redundant, but effective.
  - deleted 5y ago
    [deleted]
- taeric 5y ago
  I've been alive long enough to see that most things are useful, and all things are oversold. More, the nice easy things to build with major restrictions pretty much gets thrown out the window for complicated things that have constraints that most efforts don't have. This isn't just a software thing. Building a little shed outside? Would be silly to use the same rigor that goes into a high rise. Which would be crazy to use the same materials engineering that goes into a little shed.
  - nllsh 5y ago
    Not that I disagree, but I feel like you are overselling the simplicity of [building a shed](https://en.wiktionary.org/wiki/bikeshedding https://en.wiktionary.org/wiki/bikeshedding).
    - taeric 5y ago
      My point is that all methods and techniques probably have worked for someone doing something. And I should have leaned in on how much is still left to implementation in terms of "shed." From weather, to what is being stored. It isn't like there is a universal shed design that will make everyone happy. Nor is this saying that some things aren't truly valuable. Just recognize that some places they don't help as much as you would like. This isn't saying they are bad or worthless. Just acknowledging that they are oversold.
mdoms 5y ago
It is extremely funny to me watching Silicon Valley types slowly (very slowly) re-invent everything we knew about programming languages decades ago.
- pjmlp 5y ago
  Well for the hipster culture what we were doing wasn't cool.
david422 5y ago
I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a string - which makes all sorts of convoluted code paths when someone else was looking at "person" and only saw one type so now their function doesn't work on both types etc. I hate having to waste time figuring out the type of every variable and hold it in my head every single time I read a piece of code.
- sseagull 5y ago
  This is absolutely how I feel. I've mentioned previously taking over a project, and just not knowing the type of anything took me months to overcome. Also, type hints really help your IDE, even catching errors before you even run tests. There's also a visual cue that you are doing something wrong: If a function returns 4 levels of Union[Tuple[List[int]], Optional[str]........ Then you are doing something too complex and the function should be broken up.
- jfabre 5y ago
  I never understood this argument. In what kind of shop are you working that passing a string named person to a method expecting an object is tolerated. Or even passing different types that don't share a common interface. This would never fly in a code review in any of the companies I've worked for.
  - kennywinker 5y ago
    I've seen essentially this code in so many organically grown codebases (when they grew up without types). It's usually close the the UI, because someone had to quickly add an alternate path to support some new user interaction function find_user(person) { if user is string { query_by_name(person) } else { query_by_name(person.name) } } and yeah, we all know it's kinda messy, but also that logic has to live somewhere and we need this feature asap so it passes code review. I wrote a test for it, ship it.
spicyramen 5y ago
I wrote Python code for 4 years, then moved to GoLang I really appreciate the typing languages as prevent so many bugs I just was used to handle
exdsq 5y ago
I can’t understand programming without types - it’s just so weird…
- kzrdude 5y ago
  Python is not without types, there are dynamic types.
  - exdsq 5y ago
    Sorry to be specific I find it so weird to program without defining what I want in and out of a function
chromatin 5y ago
Even though I had previously learned some rudimentary C, C++, and Java, I really came-of-age with Python. Now, having written (and maintained) nontrivial code bases in statically typed languages including D and Rust (and dabbling in others with contributions in C, OCaml, etc.), I am never going back — except perhaps in a few cases when a library like PyTorch or Pandas has no good substitute. (edit: corrected "Linda's" to "Pandas" heh, mobile kbd)
papito 5y ago
Lack of type checking was a hot thing for a while. It made you "move faster". It was actually sold as an advantage. Until we realized that after moving faster you grind to a halt because now you have a massive codebase, with hundreds or thousands of files, and everything takes forever, and every change requires multiple rounds of testing. I believe it really has to do with the size and complexity of modern projects. With a half-decent IDE you could sort of used non-type-checked Python in 2012, but times have changed, and now we are talking about statically checking Python and Ruby. And Javascript, of course, now has it in form of TypeScript.
tabbott 5y ago
I agree with all the benefits of mypy cited in this article. For me, most important thing for the long-term health of a codebase is its readability/maintainability, and mypy static typing makes such a huge difference for that in large Python codebases. I'm really excited to see large libraries doing this migration. I'll add for folks thinking about this transition that we took a pretty different strategy for converting Zulip to be type-checked: https://blog.zulip.com/2016/10/13/static-types-in-python-oh-mypy/ https://blog.zulip.com/2016/10/13/static-types-in-python-oh-... The post is from 2016 and thus a bit stale in terms of the names of mypy options and the like, but the incremental approach we took involved only using mypy's native exclude tooling, and might be useful for some projects thinking about doing this transition. One particular convention that I think many other projects may find useful is how we do `type: ignore` in comments in the Zulip codebase, which is to have a second comment on the line explaining why we needed a `type: ignore`, like so: * # type: ignore[type-var] # https://github.com/python/typeshed/issues/4234 https://github.com/python/typeshed/issues/4234 * # type: ignore[attr-defined] # private member missing from stubs * # type: ignore[assignment] # Apparent mypy bug with Optional[int] setter. * # type: ignore[misc] # This is an undocumented internal API We've find this to be a lot more readable than using the commit message to record why we needed a `type: ignore`, and in particular it makes the work of removing these with time feel a lot more manageable to have the information organized this way. (And we can have a linter enforce that `type: ignore` always comes with such a comment).
- lrobinovitch 5y ago
  I really like this documented type ignore strategy and will start incorporating it in our codebase. Thanks for sharing.
andybak 5y ago
Having spent a decade with Python and more recently a few years with C# I still can't quite put my feelings into words but here's an attempt: "The benefits of explicit typing are obvious and clear but they downsides are subtle and hard to communicate" I still think typing in general is a net win but I'm not sure whether static typing is. You find yourself writing code that just wouldn't be neccesary in a dynamic language - and I don't just mean the direct code you write to declare and cast types. There are more subtle costs. I need to spend time with a good type inference in a language with modern typing and dynamic features to sort out how I feel about this.
- layer8 5y ago
  Types are effectively assertions about the values they represent, and statically-typed code constitutes proofs that the assertions actually hold at runtime. The static typing forces you to be sufficiently rigorous in those proofs, which may require additional code as you mention. Without static typing, one has to rely on the "proofs" in one’s head to be correct (which humans aren’t really good at), instead of having the compiler double-check one’s reasoning.
  - andybak 5y ago
    I think this falls into the category of "The benefits of explicit typing are obvious and clear". It's the other side of the equation that I'm intrigued by and struggling most to formulate.
LadyCailin 5y ago
Oh look, they're finally discovering that strong typing is actually a benefit, and using a language without it is a huge step in the wrong direction.
KingMachiavelli 5y ago
I think mandatory type hints in method signatures and optional type hints at assignment are a good compromise. But if I had to pick either a language without any type hint/inference or a verbosely strictly typed language - I would must rather use the strictly typed language.
richard_todd 5y ago
I think it's interesting that PEP 484 says ([1]): "the authors have no desire to ever make type hints mandatory, even by convention," while the opening of this article says "type hints have grown from a nice-to-have to an expectation for popular packages." Things don't always work out the way the PEP authors expect. [1]: https://www.python.org/dev/peps/pep-0484/ https://www.python.org/dev/peps/pep-0484/
- kkirsche 5y ago
  It’s the most demoralizing aspect as even just typing the standard library online documentation and examples (such as Emil autoname example) would be extremely valuable.
kraf 5y ago
It would be really interesting to see some examples of the logic errors that were found that couldn't be found by tests. This seems to have been a very robust library. What kind of problems did you find? From what is mentioned in the article it really doesn't sound like the investment of hundreds of hours from multiple people has been actually worth it.
aitchnyu 5y ago
Tangential: did anybody find success with typed Model.objects methods with Django?

205 points by quentinp 5y ago

I've seen a lot of push back on adding type checking to Python but we had a similar case at my company where we tried it out on a new project and the clarity and readability of the code was immediately beneficial to the entire team. Perhaps it's something well suited to larger codebases.

hobs 5y ago
I think it's well suited to anything really - the amount of casual problem solving and inference you can make from some simple types is pretty big in my experience, and Python's approach to allow you optionally buy into it is really nice.
tester756 5y ago
It's $current_year and there's still debate whether checking stuff at compilation time is better than at runtime?
- novok 5y ago
  People who don't think types are a good thing need to work in a statically typed language for a year or two and then see what a difference it makes in reality. Unproductive Java bureaucracy != static typing. I think the people debating it never tried it seriously.
  - fiddlerwoaroof 5y ago
    I’ve done everything from Haskell to Java and I still strongly prefer Clojure and Common Lisp-style dynamic types.
    - lemper 5y ago
      i have a small side project in clojure [1] and i always miss type checking when working on it. not by much because it's a small project but i am tired of iseq is not a function error. [1] https://dactyl.siskam.link https://dactyl.siskam.link
      - fiddlerwoaroof 5y ago
        I sort of think there are two mindsets behind this debate: people that miss the guard rails of a static type system and people that enjoy the experience of iterating quickly in a dynamically typed language. I don't really want to say everyone should pick one side or the other, just that my experience doesn't bear out the claim that "statically typed languages produce more maintainable code". And, the little bit of empirical evidence for this proposition is largely inconclusive: https://danluu.com/empirical-pl/ https://danluu.com/empirical-pl/
        simion314 5y ago
        >and people that enjoy the experience of iterating quickly in a dynamically typed language Programmers spend more time reading code then writing it. So I personally prefer the devs in the team will spend more time typing the code or use a bit more brain energy to think about types so later we can all read the code and understand it and edit faster. Dynamic works great for write-only scripts.
        fiddlerwoaroof 5y ago
        People always say this about reading code and it’s just never matched my experience working in either sort of codebase: one difference (comparing lisps and, say, Typescript or Java) is that lisps just have fewer lines to read. So, any assistance you get from the types is counteracted by having to read more code. But, additionally, I just don’t find it true to my experience that it’s easier to read and understand a dynamically typed codebase vs. a statically typed one. Especially when you have a lisp-like environment that makes accurate jump-to-definition possible. EDIT: I think I just tend to think about codebases in terms of operations rather than types. And, consequently, when I build a codebase around compositions of functions, the way I think about it isn’t very different in either paradigm.
  - jonny_eh 5y ago
    After using TypeScript for even a little bit I find it painful to go back to JavaScript for anything more complex than white-boarding.
  - wvenable 5y ago
    A lot of people grew up with Java -- especially early Java -- as their primary language. It was taught heavily in schools. I think it ruined a lot of people to static typing and exceptions because Java is/was terrible for both of those things.
- kzrdude 5y ago
  That's not really the debate in Python :) Almost every Python user now has to "deal" with type annotations. It's tempting to gradually add type annotations, it's nice documentation. But it also rubs me the wrong way to have annotations that are never checked(!). In many codebases, you might just have "casual" style type annotations in Python, and nothing ever asserts that they hold. That's nagging on me, a bit.
  - inertiatic 5y ago
    Never checked? They're statically checked. Also, tooling like https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/ can do runtime checking for important parts of your app or you can use this https://github.com/agronholm/typeguard https://github.com/agronholm/typeguard to enforce all types at runtime (although I haven't measured the performance impact, probably something to do in a separate environment than production?).
    - kzrdude 5y ago
      They are statically checked, if you run a type checker. Which many don't.
  - tehnub 5y ago
    That's a good point. If they're never checked, then they're just like incorrect/outdated comments. They sort of get at this idea in the article, and sort of describe a compromise for it. They have a list of files that they've completely annotated, and only those files are checked by mypy. So in their case, they know which annotations to ignore, and which they can rely on.
deleted 5y ago
[deleted]
handrous 5y ago
I want type checking on pretty much anything that will ever exceed about two screenfuls of code. If I can't keep the whole thing in my head at once, I want the computer to do it for me. That's the point, right? Making computers do stuff for us so we don't have to?
- david422 5y ago
  I kindof think of them as a giant set of unit tests. The compiler/linter etc. can check every variable and every function call to check to make sure you didn't mix up your types, which _will_ blow up at runtime if you got them wrong. So rather than write them all by hand, just get your tools to do it.
  - handrous 5y ago
    I see them as documentation of what I think something means or is, which the computer can check for accuracy (more or less), both as I write and as the codebase changes. That legibility to the computer is what makes them much better than documenting the same thing some other way. Are they out of date? Were they wrong to begin with? The computer will tell me, no action needed on my part. I need to look up something in the context of what I'm reading right now—oh, look, the computer just told me exactly what I needed.
matsemann 5y ago
Just wish Python's typing was better. But it's impossible to type hint the crazy "pythonic" code out there. Like the kwargs used to do a Django query.

dado3212 5y ago

Honestly will never go back to languages without type checking, it prevents so many bugs and is a huge help in understanding code you haven’t worked with previously.

SketchySeaBeast 5y ago
> is a huge help in understanding code you haven’t worked with previously This is huge for me. As someone who takes on already completed projects, it's a huge help with debugging and understand what's going on without requiring you to know the whole system forward and backwards. Sure, you still need to build a mental map of the general code flow, but you can look at a single function and clearly see the obvious inputs and outputs. Combine that with a a stack trace and you can debug that method as a single unit and then start to look at where it's called and what its downstream effects are. You don't need to start from the very beginning of the call and then follow it through, keeping mental track of what is available and in what form when and where.
seiferteric 5y ago
It is kind of ridiculous not to have types. I think in the old days handling types felt to heavy for scripting languages, but now with type inference and stuff I don't think it is any longer.
handrous 5y ago
> Honestly will never go back to languages without type checking, it prevents so many bugs and is a huge help in understanding code you haven’t worked with previously. I see static types as one of the most powerful communication tools around, as far as code goes. I can't relate at all to people complaining that they waste time. They must work very differently from how I do, is all I can figure. It's that, or they don't realize how much time they're losing to communication-related tasks, or refactoring, or writing (and maintaining!) extra or more verbose tests, or having even one more bug per year make it to production, or whatever, that'd be saved by static types, so aren't correctly accounting for the time savings. One of the two.
- adrianmonk 5y ago

m12k 5y ago

I've only been in the industry for ~15 years, but it still feels like every year, some ecosystem discovers the value of something that another ecosystem has taken for granted for decades - type-checking, immutability, unidirectional data-flow, AOT-compilation, closures, pure functions, you name it. I'm glad we seem to be converging on a set of best practices as an industry, but sometimes I wish we were spending less time rediscovering the wheel and more time building on top of and adding to the actual state of the art.

axiosgunnar 5y ago
I think the problem is to figure out what the best practices actually are. What we are observing here is „the market fixing it“. The process is messy and redundant, but effective.
- deleted 5y ago
  [deleted]
taeric 5y ago
I've been alive long enough to see that most things are useful, and all things are oversold. More, the nice easy things to build with major restrictions pretty much gets thrown out the window for complicated things that have constraints that most efforts don't have. This isn't just a software thing. Building a little shed outside? Would be silly to use the same rigor that goes into a high rise. Which would be crazy to use the same materials engineering that goes into a little shed.
- nllsh 5y ago
  Not that I disagree, but I feel like you are overselling the simplicity of [building a shed](https://en.wiktionary.org/wiki/bikeshedding https://en.wiktionary.org/wiki/bikeshedding).
  - taeric 5y ago
    My point is that all methods and techniques probably have worked for someone doing something. And I should have leaned in on how much is still left to implementation in terms of "shed." From weather, to what is being stored. It isn't like there is a universal shed design that will make everyone happy. Nor is this saying that some things aren't truly valuable. Just recognize that some places they don't help as much as you would like. This isn't saying they are bad or worthless. Just acknowledging that they are oversold.

mdoms 5y ago

It is extremely funny to me watching Silicon Valley types slowly (very slowly) re-invent everything we knew about programming languages decades ago.

pjmlp 5y ago
Well for the hipster culture what we were doing wasn't cool.

david422 5y ago

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a string - which makes all sorts of convoluted code paths when someone else was looking at "person" and only saw one type so now their function doesn't work on both types etc. I hate having to waste time figuring out the type of every variable and hold it in my head every single time I read a piece of code.

sseagull 5y ago
This is absolutely how I feel. I've mentioned previously taking over a project, and just not knowing the type of anything took me months to overcome. Also, type hints really help your IDE, even catching errors before you even run tests. There's also a visual cue that you are doing something wrong: If a function returns 4 levels of Union[Tuple[List[int]], Optional[str]........ Then you are doing something too complex and the function should be broken up.
jfabre 5y ago
I never understood this argument. In what kind of shop are you working that passing a string named person to a method expecting an object is tolerated. Or even passing different types that don't share a common interface. This would never fly in a code review in any of the companies I've worked for.
- kennywinker 5y ago
  I've seen essentially this code in so many organically grown codebases (when they grew up without types). It's usually close the the UI, because someone had to quickly add an alternate path to support some new user interaction function find_user(person) { if user is string { query_by_name(person) } else { query_by_name(person.name) } } and yeah, we all know it's kinda messy, but also that logic has to live somewhere and we need this feature asap so it passes code review. I wrote a test for it, ship it.

spicyramen 5y ago

I wrote Python code for 4 years, then moved to GoLang I really appreciate the typing languages as prevent so many bugs I just was used to handle

exdsq 5y ago

I can’t understand programming without types - it’s just so weird…

kzrdude 5y ago
Python is not without types, there are dynamic types.
- exdsq 5y ago
  Sorry to be specific I find it so weird to program without defining what I want in and out of a function

chromatin 5y ago

Even though I had previously learned some rudimentary C, C++, and Java, I really came-of-age with Python. Now, having written (and maintained) nontrivial code bases in statically typed languages including D and Rust (and dabbling in others with contributions in C, OCaml, etc.), I am never going back — except perhaps in a few cases when a library like PyTorch or Pandas has no good substitute. (edit: corrected "Linda's" to "Pandas" heh, mobile kbd)

papito 5y ago

Lack of type checking was a hot thing for a while. It made you "move faster". It was actually sold as an advantage. Until we realized that after moving faster you grind to a halt because now you have a massive codebase, with hundreds or thousands of files, and everything takes forever, and every change requires multiple rounds of testing. I believe it really has to do with the size and complexity of modern projects. With a half-decent IDE you could sort of used non-type-checked Python in 2012, but times have changed, and now we are talking about statically checking Python and Ruby. And Javascript, of course, now has it in form of TypeScript.

tabbott 5y ago

I agree with all the benefits of mypy cited in this article. For me, most important thing for the long-term health of a codebase is its readability/maintainability, and mypy static typing makes such a huge difference for that in large Python codebases. I'm really excited to see large libraries doing this migration. I'll add for folks thinking about this transition that we took a pretty different strategy for converting Zulip to be type-checked: https://blog.zulip.com/2016/10/13/static-types-in-python-oh-mypy/ https://blog.zulip.com/2016/10/13/static-types-in-python-oh-... The post is from 2016 and thus a bit stale in terms of the names of mypy options and the like, but the incremental approach we took involved only using mypy's native exclude tooling, and might be useful for some projects thinking about doing this transition. One particular convention that I think many other projects may find useful is how we do `type: ignore` in comments in the Zulip codebase, which is to have a second comment on the line explaining why we needed a `type: ignore`, like so: * # type: ignore[type-var] # https://github.com/python/typeshed/issues/4234 https://github.com/python/typeshed/issues/4234 * # type: ignore[attr-defined] # private member missing from stubs * # type: ignore[assignment] # Apparent mypy bug with Optional[int] setter. * # type: ignore[misc] # This is an undocumented internal API We've find this to be a lot more readable than using the commit message to record why we needed a `type: ignore`, and in particular it makes the work of removing these with time feel a lot more manageable to have the information organized this way. (And we can have a linter enforce that `type: ignore` always comes with such a comment).

lrobinovitch 5y ago
I really like this documented type ignore strategy and will start incorporating it in our codebase. Thanks for sharing.

andybak 5y ago

Having spent a decade with Python and more recently a few years with C# I still can't quite put my feelings into words but here's an attempt: "The benefits of explicit typing are obvious and clear but they downsides are subtle and hard to communicate" I still think typing in general is a net win but I'm not sure whether static typing is. You find yourself writing code that just wouldn't be neccesary in a dynamic language - and I don't just mean the direct code you write to declare and cast types. There are more subtle costs. I need to spend time with a good type inference in a language with modern typing and dynamic features to sort out how I feel about this.

layer8 5y ago
Types are effectively assertions about the values they represent, and statically-typed code constitutes proofs that the assertions actually hold at runtime. The static typing forces you to be sufficiently rigorous in those proofs, which may require additional code as you mention. Without static typing, one has to rely on the "proofs" in one’s head to be correct (which humans aren’t really good at), instead of having the compiler double-check one’s reasoning.
- andybak 5y ago
  I think this falls into the category of "The benefits of explicit typing are obvious and clear". It's the other side of the equation that I'm intrigued by and struggling most to formulate.

LadyCailin 5y ago

Oh look, they're finally discovering that strong typing is actually a benefit, and using a language without it is a huge step in the wrong direction.

KingMachiavelli 5y ago

I think mandatory type hints in method signatures and optional type hints at assignment are a good compromise. But if I had to pick either a language without any type hint/inference or a verbosely strictly typed language - I would must rather use the strictly typed language.

richard_todd 5y ago

I think it's interesting that PEP 484 says ([1]): "the authors have no desire to ever make type hints mandatory, even by convention," while the opening of this article says "type hints have grown from a nice-to-have to an expectation for popular packages." Things don't always work out the way the PEP authors expect. [1]: https://www.python.org/dev/peps/pep-0484/ https://www.python.org/dev/peps/pep-0484/

kkirsche 5y ago
It’s the most demoralizing aspect as even just typing the standard library online documentation and examples (such as Emil autoname example) would be extremely valuable.

kraf 5y ago

It would be really interesting to see some examples of the logic errors that were found that couldn't be found by tests. This seems to have been a very robust library. What kind of problems did you find? From what is mentioned in the article it really doesn't sound like the investment of hundreds of hours from multiple people has been actually worth it.

aitchnyu 5y ago

Tangential: did anybody find success with typed Model.objects methods with Django?