4 ms·

>A key question arises: why are so few repositories type-correct? The authors don't seem to ever discuss the fact that mypy version changes frequently make pre

>A key question arises: why are so few repositories type-correct? The authors don't seem to ever discuss the fact that mypy version changes frequently make previously-passing code fail type checks. I don't think I have ever upgraded mypy and not found new errors. Usually they're correct, sometimes incorrect, but it's a fact of being a mypy user. Between mypy itself and typeshed changes, most mypy upgrades are going to involve corresponding code changes. The larger your code base and the more complicated your types are, the worse it'll be, but it's basically an ever-present issue for any program interesting enough to really benefit from a type checker. How many of those repositories were "type-correct" but only on particular versions of mypy? I bet it's a lot!

nerdponx 5y ago
I don't have this experience. Can you give an example of a previously-valid codebase that failed typechecking unexpectedly on a recent MyPy update, that wasn't a result of a false negative bug in MyPy?
- dharmab 5y ago
  I obviously can't share it here, but my primary codebase at my job needs a few dozen changes every time we change mypy versions. There's a few places of false positives where I've had to comment `# shut up, mypy` and a `type: ignore`. Usually when I'm being clever with the subprocess module.
  - emptysea 5y ago
    I wonder if that's a mypy issue or more that the typeshed types are bugged, since type shed versions also get shipped (used to?) with new type checker versions. https://github.com/python/typeshed https://github.com/python/typeshed
- a_cool_username 5y ago
  I can't give you any links because it's not open source code, but there was a bug fix in 0.790 named "Don't simplify away Any when joining union types" that caused me some problems with bad annotations in the existing code: The annotations implied that Any was possible, but it wasn't, but it got dropped from the final Union by the bug so we never had to handle Any. Dataclasses have had some backwards-incompatible improvements as well. But the big culprit is typeshed. Something will get new/fixed annotations and suddenly you aren't handling all possible return types in your callers, or whatever.
- calpaterson 5y ago
  I've had this problem a few times., For example it happened with the 0.800+ versions. Mypy got stricter, and was more aggressive in finding code (eg small scripts in not in the proper python hierarchy). I can't show any of my professional work (no publicly available src) but this side project of mine is locked to 0.790 until I can find time to sort the issues: https://github.com/calpaterson/quarchive/tree/master/src/server https://github.com/calpaterson/quarchive/tree/master/src/ser... It's hard to classify anything as a "false negative" with mypy since it is very liberal (often unexpectedly so, which I think is one of the sharp edges of gradual typing).
ledauphin 5y ago
in a codebase with tens of thousands of lines of typed code, I see maybe a few new type errors with a new mypy release. They've always been fixable within a few minutes. I think mypy has some problems, but this isn't one of the bigger ones for me.
- a_cool_username 5y ago
  I don't think it's a "problem" with mypy, I just think it's likely the cause of a lot of the programs that the authors think don't type check. Though I will say, while most have been fixable in a few minutes, some have been a real chore to fix. Sometimes an innocuous looking error balloons into several hours of reconciling obscure type system behavior errors once you start fixing it. Regardless, it's a small price to pay for proper type checking in Python. I've more than made up the lost time in detecting bugs before they ship.
tcbasche 5y ago
I’ve found the opposite with pyright. Code that seems right but is failing checks is fixed with the next release :)
izgzhen 5y ago
Even for fully statically typed languages like C++, it is very common that some old code can’t compile with latest compiler. Shrugs.
- IshKebab 5y ago
  No it isn't. C++ has extremely good backwards compatibility.
  - nyanpasu64 5y ago
    Even aside from deliberate backwards-compatibility breaks in the standard, compilers sometimes break compatibility. Both MSVC and GCC 11 have changed their header file transitive includes within the past few years, causing projects (like doctest and Qt5) to stop compiling because they forgot to include headers, which built fine in the past but not anymore. IDK if it's "very common", but it's definitely happening in the wild. MSVC: https://github.com/onqtam/doctest/issues/183 https://github.com/onqtam/doctest/issues/183 GCC: - https://invent.kde.org/qt/qt/qtbase/-/commit/8252ef5fc6d043004ddf7085e1c1fe1bf2ca39f7 https://invent.kde.org/qt/qt/qtbase/-/commit/8252ef5fc6d0430... - https://invent.kde.org/qt/qt/qtbase/-/commit/cb2da673f53815a5cfe15f50df49b98032429f9e https://invent.kde.org/qt/qt/qtbase/-/commit/cb2da673f53815a...
  - volta83 5y ago
    I disagree. C++ often removes language features, standard library features (std::random_shuffle), etc. Also object files compiled with different standard versions often are ABI incompatible with each other, which means you can't just pick new C++ for new code, but its rather all or nothing. You can argue that when it removes features it provides a replacement (sometimes), but that does not change the fact that if you have any reasonably large project (>1 million LOC), every standard upgrade will break your app. One of the main reasons we write all new code in Rust and are migrating the C++ code base step by step to Rust is because Rust offers infinitely better backward compatiblity guarantees than C++. Rust never ever breaks your code, and you can opt-in to newer Rust editions for new code only, and these are ABI compatible with Rust code written using older editions.
bigbillheck 5y ago
In the c and c++ worlds, this is the same line of thinking that kept new warnings out of "-Wall" for so long.

nerdponx 5y ago

I don't have this experience. Can you give an example of a previously-valid codebase that failed typechecking unexpectedly on a recent MyPy update, that wasn't a result of a false negative bug in MyPy?

dharmab 5y ago
I obviously can't share it here, but my primary codebase at my job needs a few dozen changes every time we change mypy versions. There's a few places of false positives where I've had to comment `# shut up, mypy` and a `type: ignore`. Usually when I'm being clever with the subprocess module.
- emptysea 5y ago
  I wonder if that's a mypy issue or more that the typeshed types are bugged, since type shed versions also get shipped (used to?) with new type checker versions. https://github.com/python/typeshed https://github.com/python/typeshed
a_cool_username 5y ago
I can't give you any links because it's not open source code, but there was a bug fix in 0.790 named "Don't simplify away Any when joining union types" that caused me some problems with bad annotations in the existing code: The annotations implied that Any was possible, but it wasn't, but it got dropped from the final Union by the bug so we never had to handle Any. Dataclasses have had some backwards-incompatible improvements as well. But the big culprit is typeshed. Something will get new/fixed annotations and suddenly you aren't handling all possible return types in your callers, or whatever.
calpaterson 5y ago
I've had this problem a few times., For example it happened with the 0.800+ versions. Mypy got stricter, and was more aggressive in finding code (eg small scripts in not in the proper python hierarchy). I can't show any of my professional work (no publicly available src) but this side project of mine is locked to 0.790 until I can find time to sort the issues: https://github.com/calpaterson/quarchive/tree/master/src/server https://github.com/calpaterson/quarchive/tree/master/src/ser... It's hard to classify anything as a "false negative" with mypy since it is very liberal (often unexpectedly so, which I think is one of the sharp edges of gradual typing).

ledauphin 5y ago

in a codebase with tens of thousands of lines of typed code, I see maybe a few new type errors with a new mypy release. They've always been fixable within a few minutes. I think mypy has some problems, but this isn't one of the bigger ones for me.

a_cool_username 5y ago
I don't think it's a "problem" with mypy, I just think it's likely the cause of a lot of the programs that the authors think don't type check. Though I will say, while most have been fixable in a few minutes, some have been a real chore to fix. Sometimes an innocuous looking error balloons into several hours of reconciling obscure type system behavior errors once you start fixing it. Regardless, it's a small price to pay for proper type checking in Python. I've more than made up the lost time in detecting bugs before they ship.

tcbasche 5y ago

I’ve found the opposite with pyright. Code that seems right but is failing checks is fixed with the next release :)

izgzhen 5y ago

Even for fully statically typed languages like C++, it is very common that some old code can’t compile with latest compiler. Shrugs.

IshKebab 5y ago
No it isn't. C++ has extremely good backwards compatibility.
- nyanpasu64 5y ago
  Even aside from deliberate backwards-compatibility breaks in the standard, compilers sometimes break compatibility. Both MSVC and GCC 11 have changed their header file transitive includes within the past few years, causing projects (like doctest and Qt5) to stop compiling because they forgot to include headers, which built fine in the past but not anymore. IDK if it's "very common", but it's definitely happening in the wild. MSVC: https://github.com/onqtam/doctest/issues/183 https://github.com/onqtam/doctest/issues/183 GCC: - https://invent.kde.org/qt/qt/qtbase/-/commit/8252ef5fc6d043004ddf7085e1c1fe1bf2ca39f7 https://invent.kde.org/qt/qt/qtbase/-/commit/8252ef5fc6d0430... - https://invent.kde.org/qt/qt/qtbase/-/commit/cb2da673f53815a5cfe15f50df49b98032429f9e https://invent.kde.org/qt/qt/qtbase/-/commit/cb2da673f53815a...
- volta83 5y ago
  I disagree. C++ often removes language features, standard library features (std::random_shuffle), etc. Also object files compiled with different standard versions often are ABI incompatible with each other, which means you can't just pick new C++ for new code, but its rather all or nothing. You can argue that when it removes features it provides a replacement (sometimes), but that does not change the fact that if you have any reasonably large project (>1 million LOC), every standard upgrade will break your app. One of the main reasons we write all new code in Rust and are migrating the C++ code base step by step to Rust is because Rust offers infinitely better backward compatiblity guarantees than C++. Rust never ever breaks your code, and you can opt-in to newer Rust editions for new code only, and these are ABI compatible with Rust code written using older editions.

bigbillheck 5y ago

In the c and c++ worlds, this is the same line of thinking that kept new warnings out of "-Wall" for so long.