4 ms·

We use Flake8, PyLint and PyTest on an internal engineering tool / internal automation libraries with about 200,000 lines of code, including test code. I would

We use Flake8, PyLint and PyTest on an internal engineering tool / internal automation libraries with about 200,000 lines of code, including test code. I would love to use Git hooks, but long before the project grew this big the time it took to run our code checks was unacceptable to enforce with hooks. The fastest I've seen them run recently is 3 minutes. With about 800 tests, PyTest take a minimum of 45 seconds just to do test discovery--the tests actually run faster. We started with Flake8 and have been slowly adding PyLint checks, but each check (so far) has added a little more time. We diff the local "feature" branch against the remote dev branch and only run PyLint against the differences, but it still takes 1 min on average to run--experiments to run against a smaller set of files still result in coffee break worthy execution times most of the time. Judging from history, if we enforced the use of Git hooks to run these tools it would result in mutiny with teams "secretly"--their manager(s) being onboard--working around and supplanting the tool. Small things like this result in a lot of frustration and a loss of goodwill.

mmmrk 7y ago
Interesting! How exactly do you run the tests and the linters then? Pytest with flake8+pylint plugins? Separately? Serially? In parallel? Why flake8 and pylint at the same time, did you find significant differences?
- wisemanwillhear 7y ago
  We ask developers to make it part of their workflow, but the project is shared across a 50+ engineers and few people do that. We enforce it via Jenkins pipeline on every commit that get's pushed to our internal BitBucket Git server. The Jenkins BitBucket plugin is used to enforce a successful build before pull requests can be merged. It works, but it can be a little frustrating at times to see an endless steam of alternating failed then passed builds because few people run the checks before pushing. We use a combination of Python Invoke and the flake8 & pylint CLI commands. At some point we plan to reinvestigate our use of Python Invoke as it doesn't run tasks in parallel--we were new to Python when we started this project two years ago and didn't understand the limitations of the tools we were using. (I should say, Python Invoke has worked out well for most of our developer centric tasks.) We run both Flake8 and Pylint as we started out with Flake8 and later tried to use Pylint. The workload to fix the code base was too much for us as most developers are QA people learning good software design and how to share a large code base with their "extra", "volunteer" time. We've been slowly turning on more Pylint checks and cleaning up the code base. When we did try to move more quickly, we were deluged with complains about the cognitive load it placed on people and backed off. We did find some differences, but I don't remember what they were and couldn't comment on the significance.
  - mmmrk 7y ago
    Thanks. It's a tricky topic, as Google has found as well: https://static.googleusercontent.com/media/research.google.com/en/pubs/archive/43322.pdf https://static.googleusercontent.com/media/research.google.c....

mmmrk 7y ago

Interesting! How exactly do you run the tests and the linters then? Pytest with flake8+pylint plugins? Separately? Serially? In parallel? Why flake8 and pylint at the same time, did you find significant differences?

wisemanwillhear 7y ago
We ask developers to make it part of their workflow, but the project is shared across a 50+ engineers and few people do that. We enforce it via Jenkins pipeline on every commit that get's pushed to our internal BitBucket Git server. The Jenkins BitBucket plugin is used to enforce a successful build before pull requests can be merged. It works, but it can be a little frustrating at times to see an endless steam of alternating failed then passed builds because few people run the checks before pushing. We use a combination of Python Invoke and the flake8 & pylint CLI commands. At some point we plan to reinvestigate our use of Python Invoke as it doesn't run tasks in parallel--we were new to Python when we started this project two years ago and didn't understand the limitations of the tools we were using. (I should say, Python Invoke has worked out well for most of our developer centric tasks.) We run both Flake8 and Pylint as we started out with Flake8 and later tried to use Pylint. The workload to fix the code base was too much for us as most developers are QA people learning good software design and how to share a large code base with their "extra", "volunteer" time. We've been slowly turning on more Pylint checks and cleaning up the code base. When we did try to move more quickly, we were deluged with complains about the cognitive load it placed on people and backed off. We did find some differences, but I don't remember what they were and couldn't comment on the significance.
- mmmrk 7y ago
  Thanks. It's a tricky topic, as Google has found as well: https://static.googleusercontent.com/media/research.google.com/en/pubs/archive/43322.pdf https://static.googleusercontent.com/media/research.google.c....