3 ms·

Nice. How do you force types just in the new code?

by neves 5y ago

jsmeaton 5y ago
You can pipe the diff into specific flake8 checkers. Disable the check by default so it's not run over the entire codebase, and have a separate line for running specific checks on just the diff. Eg: git diff -U0 --relative origin/master... | flake8 --diff --select <custom-codes> I'll leave the flake8 plugin as an exercise to the reader as ours is intertwined with code I can't share right now.
Too 5y ago
Mypy can be configured to ignore modules based on their names. Though honestly I think it’s worth it to just bite the bullet and spend a full day or two to go through and fix every single error. After this you will have a much easier time navigating code base. I mean, your “new code” is probably modifications to your old code or calling your old code right? So you want to have at least the boundary between new and old typed. It’s quite satisfying anyway, as you will likely learn and discover tons of bugs on the way.
- LtWorf_ 5y ago
  > day or two Your code must be quite small.
- necovek 5y ago
  With any large legacy Python codebase, that's usually a month-long undertaking for one engineer, if not longer! Python by design encourages duck typing, which means that you'll have plenty of places that simply take multiple different types, and just slapping a union is usually non-trival and not very beneficial either (eg. you'd be randomly patching things with fakes and mocks in tests). If you think it's a single day job, I am sure my company (and plenty others) would pay you gladly your single daily rate to get our codebase migrated in a day :-D
- icedchai 5y ago
  For any large code base, it'll go on for months, since it gets done between higher priority work. You'll discover some problems, sure. But if they were really important, you'd have hit them earlier. My experience is these sorts of efforts create a lot of busy work and not much more.
orf 5y ago
The same (and only) way you’d reasonably be able to do it in any other language that uses a series of text files for an input. File by file.
chombier 5y ago
I generally force it on a per-file basis by adding # mypy: disallow-untyped-defs on top of new python files.
Mehdi2277 5y ago
My team started with an untyped codebase a year ago. I added a rule to CI that every pr that touches files with type errors must decrease the type errors in touched files by at least 5. When we started we had like 30k type errors. A year later and it’s about 5k left. This was a small wrapper script over mypy/pyright that just called them twice once on master and once on your branch to compare error counts. I did occasionally get pushback on it but stricter checks will be inconvenience at first and I argued it’d help over time. Now that we’ve had it a year it’s pretty well accepted in same way many would accept run formatter.
- icedchai 5y ago
  The problem with these sorts of "rules" is they create needless changes outside of the PR's primary focus. The PR's purpose is to do X: add a feature, fix a bug, whatever. Now because I've done X, you want me to do Y, a set of totally unrelated changes that ultimately are a distraction.
  - ok_dad 5y ago
    Why wouldn’t you submit two different branches to review? One where you fix errors for types and another for the feature with the fixes: (git main) > git checkout -b abc-1234/type-fixes <do work on types and commit it> (git abc-1234/type-fixes) > git checkout -b abc-1234/feature-branch <do work on feature and commit it> Then you can submit the feature branch as a PR on top of the type fixing branch and a PR for that branch to main. You only see the type fixing changes on that PR and only see the feature changes on its PR.
    - icedchai 5y ago
      Seems fine to me, but in my experience these “rules” are often tied to a single PR and various automated CI checks.
nyuszika7h 5y ago
You can use flakehell for that.

Nice. How do you force types just in the new code?

by neves 5y ago

Nice. How do you force types just in the new code?

You can pipe the diff into specific flake8 checkers. Disable the check by default so it's not run over the entire codebase, and have a separate line for running specific checks on just the diff. Eg: git diff -U0 --relative origin/master... | flake8 --diff --select <custom-codes> I'll leave the flake8 plugin as an exercise to the reader as ours is intertwined with code I can't share right now.

Too 5y ago

Mypy can be configured to ignore modules based on their names. Though honestly I think it’s worth it to just bite the bullet and spend a full day or two to go through and fix every single error. After this you will have a much easier time navigating code base. I mean, your “new code” is probably modifications to your old code or calling your old code right? So you want to have at least the boundary between new and old typed. It’s quite satisfying anyway, as you will likely learn and discover tons of bugs on the way.

LtWorf_ 5y ago
> day or two Your code must be quite small.
necovek 5y ago
With any large legacy Python codebase, that's usually a month-long undertaking for one engineer, if not longer! Python by design encourages duck typing, which means that you'll have plenty of places that simply take multiple different types, and just slapping a union is usually non-trival and not very beneficial either (eg. you'd be randomly patching things with fakes and mocks in tests). If you think it's a single day job, I am sure my company (and plenty others) would pay you gladly your single daily rate to get our codebase migrated in a day :-D
icedchai 5y ago
For any large code base, it'll go on for months, since it gets done between higher priority work. You'll discover some problems, sure. But if they were really important, you'd have hit them earlier. My experience is these sorts of efforts create a lot of busy work and not much more.

orf 5y ago

The same (and only) way you’d reasonably be able to do it in any other language that uses a series of text files for an input. File by file.

chombier 5y ago

I generally force it on a per-file basis by adding # mypy: disallow-untyped-defs on top of new python files.

Mehdi2277 5y ago

My team started with an untyped codebase a year ago. I added a rule to CI that every pr that touches files with type errors must decrease the type errors in touched files by at least 5. When we started we had like 30k type errors. A year later and it’s about 5k left. This was a small wrapper script over mypy/pyright that just called them twice once on master and once on your branch to compare error counts. I did occasionally get pushback on it but stricter checks will be inconvenience at first and I argued it’d help over time. Now that we’ve had it a year it’s pretty well accepted in same way many would accept run formatter.

icedchai 5y ago
The problem with these sorts of "rules" is they create needless changes outside of the PR's primary focus. The PR's purpose is to do X: add a feature, fix a bug, whatever. Now because I've done X, you want me to do Y, a set of totally unrelated changes that ultimately are a distraction.
- ok_dad 5y ago
  Why wouldn’t you submit two different branches to review? One where you fix errors for types and another for the feature with the fixes: (git main) > git checkout -b abc-1234/type-fixes <do work on types and commit it> (git abc-1234/type-fixes) > git checkout -b abc-1234/feature-branch <do work on feature and commit it> Then you can submit the feature branch as a PR on top of the type fixing branch and a PR for that branch to main. You only see the type fixing changes on that PR and only see the feature changes on its PR.
  - icedchai 5y ago
    Seems fine to me, but in my experience these “rules” are often tied to a single PR and various automated CI checks.

nyuszika7h 5y ago

You can use flakehell for that.