4 ms·

What’s the big deal about type checking? I haven’t worked on large code bases, only run small ML analyses on tabular data. Can someone be concrete with the adv

What’s the big deal about type checking? I haven’t worked on large code bases, only run small ML analyses on tabular data. Can someone be concrete with the advantages and use cases?

nynx 6y ago
When you're writing code in a dynamic language, you're still using a type-checker; it's just in your head instead of the computer. This is a fantastic waste of mental energy and working memory.
- tzs 6y ago
  I'm not so sure about that. When I'm actually using variables in expressions or as arguments to functions, as opposed to when I'm declaring them, the code is essentially the same in a typed language and in an untyped language. Realizing when I'm writing that the expression I'm about to type is wrong because the types do not match still depends on my doing type checking in my head. It seems to me that the case where typed wins is when my in-head type checker fails and I write something that is invalid. The typed language will catch that failure at compile time. The untyped won't catch it until runtime (and maybe not even then). For the typed language, an IDE might be able to catch a mistake immediately so I don't have to wait until compilation to find it, and that immediate feedback might prevent me from writing dozens more lines with the same mistake, but even then I was using my in-head type checker to write that first mistake for the IDE to catch.
  - valenterry 6y ago
    You are only describing a small part. Here's another example. You are using a library and call a function that expects a List[Foo] as argument. Now you check in the same library if there is a function that returns a List[Foo] or at least a Foo. So you find it and are good. Without types, now you have to read the documentation which is much much slower than having your IDE help you out. And yeah, sometimes variable names and function names are sufficient, but in my experience they are very often not. Also, what happens when you update your library to the next version and it changed what it expects and returns? Now you either have a typechecker that checks _all_ the code, even the one you haven't written... or you have to do it yourself. For each version. And each library.
- mixmastamyk 6y ago
  It's in your head, and it works faster because there is less boilerplate to read. This is great until the project reaches a size where this becomes unmanageable.
bobbylarrybobby 6y ago
The big advantages for me are autocomplete, documentation, and simple bug-finding (you passed an Optional[str] to something expecting a str; that’s almost surely a bug). Being able to hover my cursor over methods and get their arguments and doc strings is invaluable. But that’s only possible if the type of the caller is known.
- guywhocodes 6y ago
  This is too true, too often is the real type signature str. And with all the inconsistency we have in the ecosystem and even standard library it saves you so much esoteric crap.
guywhocodes 6y ago
I think the difference in the workflow is very much as described. Often you will have test suites that run in tens of minutes at least due to heavy coupling and many integration tests. While the automatically incremental type checkers run in less than a second and you can run it on ever save.
jameskilton 6y ago
Now that I'm working in a very large code base with many engineers (Stripe), I have a new understanding of where type-checking is actually important: documentation. Yeah you can go in circles for an eternity about which is better and which is worse. I try to stay away from such arguments because they aren't productive. I do now argue though that as an engineering team and code base gets bigger, you will need static typing to be able to continue to grow at a sustainable pace. Static typing provides a very strong safety net letting people make good, safe changes in code they may not be intimately familiar with. If you have a small code base and/or small team, then static typing becomes much more of a personal preference.
brianberns 6y ago
Type checking turns run-time errors into compile-time errors. This results in a huge improvement in code quality.
anentropic 6y ago
It's verified documentation
jerf 6y ago
"I haven’t worked on large code bases, only run small ML analyses on tabular data." While I endorse the other answers, I do want to highlight how the issues only appear at scale. I have no problem using a dynamically typed language for up to low-hundreds of numbers of lines. But that's about where I start to get nervous I've picked the wrong language. (Or, putting it a different way, about day 3 of working on the same code base continuously.) When the whole program fits on the screen, essentially, it's no big deal to not have types. But as the program grows, the problems emerge. I think I have a minority view on what the problem that emerge is, though. I think the first problem a dynamically-typed code base usually encounters is that there is some function that accepts an object of some kind, and you discover that it actually needs to accept a list of that object sometimes instead. In a statically-typed language, you change the type from "MyObject" to "MyObject[]" and immediately change all the call sites. Possibly you even discover the change reverberates back up the program design, and you push it up higher, with the compiler helping all the way. With dynamic languages, you tend instead to do something like: def myOldFunction(obj): if typeof(obj) != typeof([]): obj = [obj] # function continues Well, that's if you're lucky. It seems to be more popular to instead decorate the entire function with if statements every time "obj" is used, but let's take this instead. Now you've taken your first step down a dark road, where you now have a function that accepts an object, or maybe an array of those objects. Then you decide to treat None/nil/whatever as an empty list. Then you realize that sometimes the return value also needs to be a list of whatever it used to return, but you have to add a parameter to the call now to specify you want a list in the return value so you don't break all the old callers. You inevitably head down a road where the function is filled to the brim with entangled concerns from a lot of other code. Then you start getting lots of these functions in a codebase together, and the codebase can never again be refactored because it'll break everything. (Or, rather, it can be, but only in very constrained ways.) By contrast, I prefer even to prototype in static languages now, because when I make a mistake in the signature a function should have, in just a minute or two, I can fix it, and it's gone like the mistake was never there because the compiler made me fix it (and, fortunately, helped). It is much easier to maintain a discipline where every function isn't deeply entangled with another when you're not carrying along the entire history of the function's input and output parameters forever. As you scale up, other problems emerge too, like the difficulty of using static analysis tools and the way documentation and unit tests have to carry a lot more water because the code is so much harder to get a grasp on... but whenever I'm starting a new code base, or even just a new module in an existing code base, it is always the above problem that is the first one I hit in a dynamic language and the first benefit of a static language I notice. A lot of the other scaling issues take months or even years to develop, but this is the one I notice in days, or in the worst cases even mere hours. I sometimes wonder how much of the "prototype one to throw away" comes from people using dynamic languages; usually with not much extra effort, my statically-typed "prototype" comes out production quality by the time I'm done working with it this way. The end result may not much resemble what I first sketched out, but I got there with a clear set of easy steps, and usually I don't even take that large a step backwards since I pair everything with unit tests and between the static types and tests I'm usually always moving forwards even as I make rather substantial changes in the codebase that I wouldn't dream of doing in dynamic languages because I know from experience that it's much harder to avoid breaking things and taking huge steps backwards even as I move some part incrementally forward.
Decabytes 6y ago
A coworker sent me a block of python code that wasn't working, asking if anything looked off. It took me a bit to grok what each line was doing, what values were getting stored in what variables, and what values were being returned from what functions. If everything was typed I would have been able to understand it quicker.
zackees 6y ago
Imagine changing core features of your codebase and then the compiler / type checker telling you all the places that you needed to fix it. Then when it's fixed, the code runs flawlessly. That's happened a LOT in my 10 years of C++ development. I can leverage the compiler to do 90% of the refactoring work. For example, say you want to remove a field from an object in C++. The compiler will tell you every place where the field is accessed and raise a compiler error. In python this won't manifest until the code is run in that particular spot. I can only imagine that in this type of language a well intentioned refactor could easily create 10 bugs that don't get triggered for a very long time.

What’s the big deal about type checking? I haven’t worked on large code bases, only run small ML analyses on tabular data. Can someone be concrete with the advantages and use cases?

nynx 6y ago

When you're writing code in a dynamic language, you're still using a type-checker; it's just in your head instead of the computer. This is a fantastic waste of mental energy and working memory.

tzs 6y ago
I'm not so sure about that. When I'm actually using variables in expressions or as arguments to functions, as opposed to when I'm declaring them, the code is essentially the same in a typed language and in an untyped language. Realizing when I'm writing that the expression I'm about to type is wrong because the types do not match still depends on my doing type checking in my head. It seems to me that the case where typed wins is when my in-head type checker fails and I write something that is invalid. The typed language will catch that failure at compile time. The untyped won't catch it until runtime (and maybe not even then). For the typed language, an IDE might be able to catch a mistake immediately so I don't have to wait until compilation to find it, and that immediate feedback might prevent me from writing dozens more lines with the same mistake, but even then I was using my in-head type checker to write that first mistake for the IDE to catch.
- valenterry 6y ago
  You are only describing a small part. Here's another example. You are using a library and call a function that expects a List[Foo] as argument. Now you check in the same library if there is a function that returns a List[Foo] or at least a Foo. So you find it and are good. Without types, now you have to read the documentation which is much much slower than having your IDE help you out. And yeah, sometimes variable names and function names are sufficient, but in my experience they are very often not. Also, what happens when you update your library to the next version and it changed what it expects and returns? Now you either have a typechecker that checks _all_ the code, even the one you haven't written... or you have to do it yourself. For each version. And each library.
mixmastamyk 6y ago
It's in your head, and it works faster because there is less boilerplate to read. This is great until the project reaches a size where this becomes unmanageable.

bobbylarrybobby 6y ago

The big advantages for me are autocomplete, documentation, and simple bug-finding (you passed an Optional[str] to something expecting a str; that’s almost surely a bug). Being able to hover my cursor over methods and get their arguments and doc strings is invaluable. But that’s only possible if the type of the caller is known.

guywhocodes 6y ago
This is too true, too often is the real type signature str. And with all the inconsistency we have in the ecosystem and even standard library it saves you so much esoteric crap.

guywhocodes 6y ago

I think the difference in the workflow is very much as described. Often you will have test suites that run in tens of minutes at least due to heavy coupling and many integration tests. While the automatically incremental type checkers run in less than a second and you can run it on ever save.

jameskilton 6y ago

Now that I'm working in a very large code base with many engineers (Stripe), I have a new understanding of where type-checking is actually important: documentation. Yeah you can go in circles for an eternity about which is better and which is worse. I try to stay away from such arguments because they aren't productive. I do now argue though that as an engineering team and code base gets bigger, you will need static typing to be able to continue to grow at a sustainable pace. Static typing provides a very strong safety net letting people make good, safe changes in code they may not be intimately familiar with. If you have a small code base and/or small team, then static typing becomes much more of a personal preference.

brianberns 6y ago

Type checking turns run-time errors into compile-time errors. This results in a huge improvement in code quality.

anentropic 6y ago

It's verified documentation

jerf 6y ago

"I haven’t worked on large code bases, only run small ML analyses on tabular data." While I endorse the other answers, I do want to highlight how the issues only appear at scale. I have no problem using a dynamically typed language for up to low-hundreds of numbers of lines. But that's about where I start to get nervous I've picked the wrong language. (Or, putting it a different way, about day 3 of working on the same code base continuously.) When the whole program fits on the screen, essentially, it's no big deal to not have types. But as the program grows, the problems emerge. I think I have a minority view on what the problem that emerge is, though. I think the first problem a dynamically-typed code base usually encounters is that there is some function that accepts an object of some kind, and you discover that it actually needs to accept a list of that object sometimes instead. In a statically-typed language, you change the type from "MyObject" to "MyObject[]" and immediately change all the call sites. Possibly you even discover the change reverberates back up the program design, and you push it up higher, with the compiler helping all the way. With dynamic languages, you tend instead to do something like: def myOldFunction(obj): if typeof(obj) != typeof([]): obj = [obj] # function continues Well, that's if you're lucky. It seems to be more popular to instead decorate the entire function with if statements every time "obj" is used, but let's take this instead. Now you've taken your first step down a dark road, where you now have a function that accepts an object, or maybe an array of those objects. Then you decide to treat None/nil/whatever as an empty list. Then you realize that sometimes the return value also needs to be a list of whatever it used to return, but you have to add a parameter to the call now to specify you want a list in the return value so you don't break all the old callers. You inevitably head down a road where the function is filled to the brim with entangled concerns from a lot of other code. Then you start getting lots of these functions in a codebase together, and the codebase can never again be refactored because it'll break everything. (Or, rather, it can be, but only in very constrained ways.) By contrast, I prefer even to prototype in static languages now, because when I make a mistake in the signature a function should have, in just a minute or two, I can fix it, and it's gone like the mistake was never there because the compiler made me fix it (and, fortunately, helped). It is much easier to maintain a discipline where every function isn't deeply entangled with another when you're not carrying along the entire history of the function's input and output parameters forever. As you scale up, other problems emerge too, like the difficulty of using static analysis tools and the way documentation and unit tests have to carry a lot more water because the code is so much harder to get a grasp on... but whenever I'm starting a new code base, or even just a new module in an existing code base, it is always the above problem that is the first one I hit in a dynamic language and the first benefit of a static language I notice. A lot of the other scaling issues take months or even years to develop, but this is the one I notice in days, or in the worst cases even mere hours. I sometimes wonder how much of the "prototype one to throw away" comes from people using dynamic languages; usually with not much extra effort, my statically-typed "prototype" comes out production quality by the time I'm done working with it this way. The end result may not much resemble what I first sketched out, but I got there with a clear set of easy steps, and usually I don't even take that large a step backwards since I pair everything with unit tests and between the static types and tests I'm usually always moving forwards even as I make rather substantial changes in the codebase that I wouldn't dream of doing in dynamic languages because I know from experience that it's much harder to avoid breaking things and taking huge steps backwards even as I move some part incrementally forward.

Decabytes 6y ago

A coworker sent me a block of python code that wasn't working, asking if anything looked off. It took me a bit to grok what each line was doing, what values were getting stored in what variables, and what values were being returned from what functions. If everything was typed I would have been able to understand it quicker.

zackees 6y ago

Imagine changing core features of your codebase and then the compiler / type checker telling you all the places that you needed to fix it. Then when it's fixed, the code runs flawlessly. That's happened a LOT in my 10 years of C++ development. I can leverage the compiler to do 90% of the refactoring work. For example, say you want to remove a field from an object in C++. The compiler will tell you every place where the field is accessed and raise a compiler error. In python this won't manifest until the code is run in that particular spot. I can only imagine that in this type of language a well intentioned refactor could easily create 10 bugs that don't get triggered for a very long time.