4 ms·

Prospector: Python Static Analysis for Humans

12ifrb 12y ago
GPLv2 License - https://github.com/landscapeio/prospector/blob/master/LICENSE https://github.com/landscapeio/prospector/blob/master/LICENS... Warning for those who don't use that license.
- davvid 12y ago
  If you're using it as an external tool then the license is an irrelevant detail. Folks don't typically do "import pylint" in their code.
- muyuu 12y ago
  What's the problem in GPLv2 for a code analysis tool?
sbochins 12y ago
I think a better alternative is to use python mode for vim and get all the static analysis checks as you're coding. I tend to do half my development using an IDE (java/scala), but I don't really miss any of those features using this great plugin and vim.
- MaxGabriel 12y ago
  I don't think the author's tool is in opposition to IDE/plugin based static analysis. For one thing, tools like python-mode for vim (https://github.com/klen/python-mode https://github.com/klen/python-mode) call out to libraries like pylint to implement their functionality. Prospector also supports machine readable outputs that IDEs/plugins can use to determine how to display linting issues. Even if you prefer not to use a command-line based linter, a non-GUI option is essential if you want to do linting with CI. If you run an open source Python project, you can't assume that everyone will use a linter to check for warnings before submitting a PR (and even then, can't assume their linter settings match yours)—but you can check easily lint all PRs using Travis CI or the author's company Landscape.
- afarrell 12y ago
  I should take another crack at installing python-mode for vim. Last time I did, I spent an hour at it and it ended up taking 10s to register any entered text. I didn't know how to debug stick a debugger or logger into vim plugins, so I just gave up. Sometime when I have more patience, I'll give it another try and maybe produce an actual bug report. Anyone happen to know a good way to debug vimscript?
of 12y ago
I haven't tried PyLint -- I use flake8 for static Python analysis: https://pypi.python.org/pypi/flake8 https://pypi.python.org/pypi/flake8
the_mitsuhiko 12y ago
This "for humans" trend really needs to stop :)
- afarrell 12y ago
  Why? So long as it doesn't get watered down and actually meaningfully indicates "I've made a serious effort to think about the user interaction and engineering psychology issues involved in my design.", I think its a trend that advances the industry.
  - sitkack 12y ago
    It is pretentious in the same way 'with love' is tacked onto startup tag lines. It implies that all the other libraries are for Asperger's wielding non-persons.
    - afarrell 12y ago
      No, it implies that other libraries are written primarily for computers to execute and secondarily for humans to use. It is only pretentious if it is...actually pretending.
joelthelion 12y ago
Is there a way to use it with YouCompleteMe/Syntastic?
- carlio 12y ago
  There is not right now, although I hope to add some plugins for common IDEs and editors. You may have success using https://github.com/davidhalter/jedi-vim https://github.com/davidhalter/jedi-vim though - the guy who writes it is adding static analysis.
jdimov 12y ago
This is an awesome tool with a lot of potential and I can't wait to see it evolve. Even though all the hard work is actually done by the 3rd party checkers, THIS ties all these other tools together and makes them an order of magnitude more accessible. Well done!
ekimekim 12y ago
"you will consider most of the warnings that come from tools like pylint or pep8 or pyflakes to be a bit picky." I use pyflakes, and out of the box it doesn't give ANY stylistic warnings, only basic "I don't think this variable exists" and common symptoms of typos like unused variables and overwriting a local var with a loop variable (I suppose you could argue that that could be a stylistic choice, but the intention is to catch errors, not correct your code style).
- abecedarius 12y ago
  I've tried all three, and pyflakes is the only one I'd keep using regularly. I agree that the OP misrepresents pyflakes.
chair6 12y ago
"The whole purpose of this project is to get people using static analysis. I think they're an excellent resource for projects and can help you find problems in your code and even learn new things." -- agreed! I'm looking forward to having a play with Prospector. For those who are interested in this, check out the security-focused Python static analysis tool that I built an initial PoC for and we've been building out within the OpenStack Security Group. The Bandit project page is at https://wiki.openstack.org/wiki/Security/Projects/Bandit https://wiki.openstack.org/wiki/Security/Projects/Bandit and the code at https://github.com/stackforge/bandit https://github.com/stackforge/bandit.
- carlio 12y ago
  Bandit looks like a really great addition to the various Python checking tools. Expect some pull requests soon!
president 12y ago
How do all these libraries check comments? As far as I know, the Python ast library does not parse comments?
- rcfox 12y ago
  They check for docstrings. From PEP 257[0]: "A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. Such a docstring becomes the __doc__ special attribute of that object." [0] http://legacy.python.org/dev/peps/pep-0257/ http://legacy.python.org/dev/peps/pep-0257/
- carlio 12y ago
  Comments are available after tokenisation (https://docs.python.org/2/library/tokenize.html https://docs.python.org/2/library/tokenize.html). They're indeed not in the AST but you can get at them for analysis.

GPLv2 License - https://github.com/landscapeio/prospector/blob/master/LICENSE https://github.com/landscapeio/prospector/blob/master/LICENS... Warning for those who don't use that license.

davvid 12y ago
If you're using it as an external tool then the license is an irrelevant detail. Folks don't typically do "import pylint" in their code.
muyuu 12y ago
What's the problem in GPLv2 for a code analysis tool?

sbochins 12y ago

I think a better alternative is to use python mode for vim and get all the static analysis checks as you're coding. I tend to do half my development using an IDE (java/scala), but I don't really miss any of those features using this great plugin and vim.

MaxGabriel 12y ago
I don't think the author's tool is in opposition to IDE/plugin based static analysis. For one thing, tools like python-mode for vim (https://github.com/klen/python-mode https://github.com/klen/python-mode) call out to libraries like pylint to implement their functionality. Prospector also supports machine readable outputs that IDEs/plugins can use to determine how to display linting issues. Even if you prefer not to use a command-line based linter, a non-GUI option is essential if you want to do linting with CI. If you run an open source Python project, you can't assume that everyone will use a linter to check for warnings before submitting a PR (and even then, can't assume their linter settings match yours)—but you can check easily lint all PRs using Travis CI or the author's company Landscape.
afarrell 12y ago
I should take another crack at installing python-mode for vim. Last time I did, I spent an hour at it and it ended up taking 10s to register any entered text. I didn't know how to debug stick a debugger or logger into vim plugins, so I just gave up. Sometime when I have more patience, I'll give it another try and maybe produce an actual bug report. Anyone happen to know a good way to debug vimscript?

of 12y ago

I haven't tried PyLint -- I use flake8 for static Python analysis: https://pypi.python.org/pypi/flake8 https://pypi.python.org/pypi/flake8

the_mitsuhiko 12y ago

This "for humans" trend really needs to stop :)

afarrell 12y ago
Why? So long as it doesn't get watered down and actually meaningfully indicates "I've made a serious effort to think about the user interaction and engineering psychology issues involved in my design.", I think its a trend that advances the industry.
- sitkack 12y ago
  It is pretentious in the same way 'with love' is tacked onto startup tag lines. It implies that all the other libraries are for Asperger's wielding non-persons.
  - afarrell 12y ago
    No, it implies that other libraries are written primarily for computers to execute and secondarily for humans to use. It is only pretentious if it is...actually pretending.

joelthelion 12y ago

Is there a way to use it with YouCompleteMe/Syntastic?

carlio 12y ago
There is not right now, although I hope to add some plugins for common IDEs and editors. You may have success using https://github.com/davidhalter/jedi-vim https://github.com/davidhalter/jedi-vim though - the guy who writes it is adding static analysis.

jdimov 12y ago

This is an awesome tool with a lot of potential and I can't wait to see it evolve. Even though all the hard work is actually done by the 3rd party checkers, THIS ties all these other tools together and makes them an order of magnitude more accessible. Well done!

ekimekim 12y ago

"you will consider most of the warnings that come from tools like pylint or pep8 or pyflakes to be a bit picky." I use pyflakes, and out of the box it doesn't give ANY stylistic warnings, only basic "I don't think this variable exists" and common symptoms of typos like unused variables and overwriting a local var with a loop variable (I suppose you could argue that that could be a stylistic choice, but the intention is to catch errors, not correct your code style).

abecedarius 12y ago
I've tried all three, and pyflakes is the only one I'd keep using regularly. I agree that the OP misrepresents pyflakes.

chair6 12y ago

"The whole purpose of this project is to get people using static analysis. I think they're an excellent resource for projects and can help you find problems in your code and even learn new things." -- agreed! I'm looking forward to having a play with Prospector. For those who are interested in this, check out the security-focused Python static analysis tool that I built an initial PoC for and we've been building out within the OpenStack Security Group. The Bandit project page is at https://wiki.openstack.org/wiki/Security/Projects/Bandit https://wiki.openstack.org/wiki/Security/Projects/Bandit and the code at https://github.com/stackforge/bandit https://github.com/stackforge/bandit.

carlio 12y ago
Bandit looks like a really great addition to the various Python checking tools. Expect some pull requests soon!

president 12y ago

How do all these libraries check comments? As far as I know, the Python ast library does not parse comments?

rcfox 12y ago
They check for docstrings. From PEP 257[0]: "A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. Such a docstring becomes the __doc__ special attribute of that object." [0] http://legacy.python.org/dev/peps/pep-0257/ http://legacy.python.org/dev/peps/pep-0257/
carlio 12y ago
Comments are available after tokenisation (https://docs.python.org/2/library/tokenize.html https://docs.python.org/2/library/tokenize.html). They're indeed not in the AST but you can get at them for analysis.