9 ms·

Phan by Etsy – A Static Analyzer for PHP 7+

113 points by nikolay 11y ago

debacle 11y ago
This has been written exclusively with PHP 7 in mind and must have taken a reasonable amount of thought and a lot of hard work. The source code looks great, and I could see this supplanting some of the other tools out there for PHP static analysis.
asm 11y ago
Co-author here. Happy to answer any questions folks have about Phan.
- cweagans 11y ago
  This is awesome. Any plans to add autocomplete functionality similar to HHVM's typechecker?
  - asm 11y ago
    We'd like to get there eventually. Phan populates a sqlite database when analyzing code (if run with the -s flag). This allows fast runs on single file changes or patches. We're hoping to use this for things like auto-complete and finding callers for a class/method/whatever eventually. For now, we have a primordial Syntastic vim plugin (https://github.com/etsy/phan/blob/master/plugins/vim/bundle/syntastic/syntax_checkers/php/phan.vim https://github.com/etsy/phan/blob/master/plugins/vim/bundle/...) for getting errors into the quickfix view.
    - cweagans 11y ago
      That's awesome. If a PR was opened that added autocomplete and/or finding callers and/or refactoring (and assuming that the code is good), would you guys be open to merging it? Or would it be delayed so that the core functionality can be focused on?
      - asm 11y ago
        Definitely. If you want to start working on that, feel free to open up an issue (https://github.com/etsy/phan/issues/new https://github.com/etsy/phan/issues/new) to start a conversation about it. We currently store method/function callers in the sqlite database (see https://github.com/etsy/phan/blob/master/src/Phan/Analyze/BreadthFirstVisitor.php#L922-L929 https://github.com/etsy/phan/blob/master/src/Phan/Analyze/Br...), but aren't yet exposing an interface for getting callers. We'll want to store callers for constants, classes, properties and global variables as well.
  - allan_s 11y ago
    Meanwhile, have you checked padawan ? https://github.com/mkusher/padawan.php https://github.com/mkusher/padawan.php it's a server made to be used by editors for autocompletion of PHP code (like jedi for python) it works pretty well with vim AFAIK
    - cweagans 11y ago
      Yep. https://github.com/mkusher/padawan.php/issues/1 https://github.com/mkusher/padawan.php/issues/1 Project started after mine. https://github.com/cweagans/theforce https://github.com/cweagans/theforce Just ran out of time, so mkusher did his own thing. I disagree with his direction and I think it's overly complex from a user's standpoint, but he disagrees and isn't willing to change it, so I lost interest in that project too. Using the actual AST from the PHP interpreter is going to be a) much faster and b) much more accurate.
- gravypod 11y ago
  How Phan treat SIMD in PHP7? Does it pick up what is going on with functions as variables?
  - asm 11y ago
    Phan does not attempt to make sense of things like `$class_name->$function_name()` or `$$v`. I'd be happy to see a pull request that could handle simple strings, but otherwise I imagine the best we can do is emit a warning. I'm not sure what SIMD refers to. Can you clarify that?
    - merlincorey 11y ago
      > I'm not sure what SIMD refers to. Can you clarify that? With a name like "asm" that comes as a surprise... Single Instruction Multiple Data[0] refers to low level processor instructions that are able to work across a range of data, e.g. video cards working on changing the values in a framebuffer with one instruction mapping across an entire buffer in parallel. [0] https://en.wikipedia.org/wiki/SIMD https://en.wikipedia.org/wiki/SIMD
      - giaour 11y ago
        That doesn't clarify what the OP meant. PHP does not allow you to issue processor instructions directly.
      - psaintla 11y ago
        I'm going to take a crazy leap here based on the commits. asm might not stand for what you think.
    - StefanKarpinski 11y ago
      What about `$$$v`? Does it handle `$$$v`?
    - gravypod 11y ago
      I wrote this when I had just woke up as I was excited, so I had to get a question in. I think what I meant was passing functions into other functions. I do this for things like: $user_requirements = [filter1, filter2]; foreach ($user_requirements as $test) { if (!$test($some_data)) { return false; } return true; } Is there any way to tell the analyzer to make sure that afunction of n arguments is passed in?
- durich 11y ago
  Awesome project, Do you know where I can download a php_ast.dll for the latest php 7 for windows?
  - asm 11y ago
    I'm not sure. PHP7 is pretty new so package maintainers may not have had a chance to put binary releases together. You may need to compile it yourself from https://github.com/nikic/php-ast https://github.com/nikic/php-ast.
- cdnsteve 11y ago
  How is performance of Phan? Nested looping checks in phase 2 scans sound intensive and bottleneck-ish.
  - asm 11y ago
    Analysis takes a lot of CPU and we're not doing any parallel processing. Phan itself is about 30k lines of code and takes about 1.5s for a clean run. That being said, you should take a look at the '-s' flag for saving state to a sqlite database. After the initial run the stored state can be used to only scan changed files. For a patch with a few files changed it should be about 0.5s to run the analysis.
- mcfunley 11y ago
  Am I right in assuming that the name is a tribute to John Allspaw's rec.music.phish posts from the 90s? https://groups.google.com/forum/#!search/allspaw$20phriend%7Csort:relevance%7Cspell:false/rec.music.phish/8xedeCvJSAM/U9rSD7a0K54J https://groups.google.com/forum/#!search/allspaw$20phriend%7...
VeejayRampay 11y ago
PHP gets so much love and money shoved at it, the Ruby programmer in me finds it unfair. That being said, I'm really glad that this helps millions of PHP devs around the world so bravo to the people working on this.
- dr_zoidberg 11y ago
  PHP gets "so much love and money shoved at it" because, for a lot of problems, it's the right tool. You can always ask facebook about it.
  - lmm 11y ago
    It wasn't the right tool for Facebook - Python would have been a better alternative, at least once they graduated from shared hosting. It was good enough that migration was never worthwhile, because the technology wasn't the innovative part of Facebook, but that doesn't make it good.
    - dr_zoidberg 11y ago
      Could you explain your point? Given it's success, I can't see why PHP wasn't the right tool for Facebook.
      - lmm 11y ago
        Just that Python is a better language for solving those kind of problems. I wrote a vaguely-facebook-like webapp in the same timeframe; later I worked (briefly) at last.fm (which uses, or at least used, the facebook stack) and spent most of my time working on better replacements for PHP code and issues that were caused primarily by PHP. If nothing else look at all the tools facebook produces - thrift and HHVM being the most prominent ones - to work around their PHP problems. PHP was the "right tool" in that it was the language Zuck knew when it was a one-man project. And facebook's success is a reminder that second-tier technologies can be perfectly adequate to build a successful business on, that business concerns often have a lot more impact than technical ones. It shouldn't be taken as an indication that all their technical decisions were right.
pka 11y ago
I really don't want to sound negative, I'm sure a lot of thought and work went into this, but one has to wonder - would it not be easier to just use static languages instead of writing static analyzers from scratch for every dynamic language out there?
- Bulk70 11y ago
  Often the problem being solved by this sort of thing isn't directly about making it easier to work with a language, but making it easier to hire developers. The hiring pool for developers who are very good with PHP is much much larger than a lot of other languages simply by virtue of it being an easy language to get started with. This is also why Facebook has spent so much time and effort on things like HHVM and Hack.
  - pka 11y ago
    2 counterpoints: * Would you hire somebody who wasn't able to learn a new language in a reasonable amount of time? * Would you let somebody who just started with PHP (by virtue of it being an easy language to start with) work on your backend?
    - FranOntanaya 11y ago
      Not counterpoints. The comment above states that the pool of very good PHP developers is larger. This implies easier hiring (capacity to learn a new language is not relevant, because it works both ways), which, among other reasons, would be because they'll be applying first to PHP positions; and secondly, a very good PHP developer is far from somebody who just started.
      - pka 11y ago
        Let me rephrase. I am proposing that (barring very specific requirements which webdev doesn't justify) there's no such thing as a "good PHP developer". A good developer is a good developer, period. A good developer will learn a new language in a reasonable amount of time and will be able to be productive and write good code in it. A PHP-only (or X-only for that matter) developer is, by the definition above, not a good developer. So why restrict oneself to some random pool of X-only people?
asm 11y ago
Clarification on the title of this post; Phan is written in PHP7, but will analyze code written in any version of PHP.
- debacle 11y ago
  As long as that code can run on PHP7.
pbreit 11y ago
I'm guessing PHP still dominates shared hosting but would anyone ever use it on AWS, DO, Heroku, own servers, etc?
- sarciszewski 11y ago
  Yes.
- jon-wood 11y ago
  Yes. Facebook is PHP, as are Etsy, and there's no way they're the only ones. Heroku, DO, and AWS all fully support PHP, which they wouldn't bother doing if there weren't a market.
- jdhawk 11y ago
  Sure, why not? You can still write good applications using PHP. It not just for legacy servers and pre-installed applications (wordpress/drupal). You don't even have to use [framework of the month]. Its just a tool. Good architecture, good practices, and good code are not syntax dependent.

113 points by nikolay 11y ago

This has been written exclusively with PHP 7 in mind and must have taken a reasonable amount of thought and a lot of hard work. The source code looks great, and I could see this supplanting some of the other tools out there for PHP static analysis.

asm 11y ago

Co-author here. Happy to answer any questions folks have about Phan.

cweagans 11y ago
This is awesome. Any plans to add autocomplete functionality similar to HHVM's typechecker?
- asm 11y ago
  We'd like to get there eventually. Phan populates a sqlite database when analyzing code (if run with the -s flag). This allows fast runs on single file changes or patches. We're hoping to use this for things like auto-complete and finding callers for a class/method/whatever eventually. For now, we have a primordial Syntastic vim plugin (https://github.com/etsy/phan/blob/master/plugins/vim/bundle/syntastic/syntax_checkers/php/phan.vim https://github.com/etsy/phan/blob/master/plugins/vim/bundle/...) for getting errors into the quickfix view.
  - cweagans 11y ago
    That's awesome. If a PR was opened that added autocomplete and/or finding callers and/or refactoring (and assuming that the code is good), would you guys be open to merging it? Or would it be delayed so that the core functionality can be focused on?
    - asm 11y ago
      Definitely. If you want to start working on that, feel free to open up an issue (https://github.com/etsy/phan/issues/new https://github.com/etsy/phan/issues/new) to start a conversation about it. We currently store method/function callers in the sqlite database (see https://github.com/etsy/phan/blob/master/src/Phan/Analyze/BreadthFirstVisitor.php#L922-L929 https://github.com/etsy/phan/blob/master/src/Phan/Analyze/Br...), but aren't yet exposing an interface for getting callers. We'll want to store callers for constants, classes, properties and global variables as well.
- allan_s 11y ago
  Meanwhile, have you checked padawan ? https://github.com/mkusher/padawan.php https://github.com/mkusher/padawan.php it's a server made to be used by editors for autocompletion of PHP code (like jedi for python) it works pretty well with vim AFAIK
  - cweagans 11y ago
    Yep. https://github.com/mkusher/padawan.php/issues/1 https://github.com/mkusher/padawan.php/issues/1 Project started after mine. https://github.com/cweagans/theforce https://github.com/cweagans/theforce Just ran out of time, so mkusher did his own thing. I disagree with his direction and I think it's overly complex from a user's standpoint, but he disagrees and isn't willing to change it, so I lost interest in that project too. Using the actual AST from the PHP interpreter is going to be a) much faster and b) much more accurate.
gravypod 11y ago
How Phan treat SIMD in PHP7? Does it pick up what is going on with functions as variables?
- asm 11y ago
  Phan does not attempt to make sense of things like `$class_name->$function_name()` or `$$v`. I'd be happy to see a pull request that could handle simple strings, but otherwise I imagine the best we can do is emit a warning. I'm not sure what SIMD refers to. Can you clarify that?
  - merlincorey 11y ago
    > I'm not sure what SIMD refers to. Can you clarify that? With a name like "asm" that comes as a surprise... Single Instruction Multiple Data[0] refers to low level processor instructions that are able to work across a range of data, e.g. video cards working on changing the values in a framebuffer with one instruction mapping across an entire buffer in parallel. [0] https://en.wikipedia.org/wiki/SIMD https://en.wikipedia.org/wiki/SIMD
    - giaour 11y ago
      That doesn't clarify what the OP meant. PHP does not allow you to issue processor instructions directly.
    - psaintla 11y ago
      I'm going to take a crazy leap here based on the commits. asm might not stand for what you think.
  - StefanKarpinski 11y ago
    What about `$$$v`? Does it handle `$$$v`?
  - gravypod 11y ago
    I wrote this when I had just woke up as I was excited, so I had to get a question in. I think what I meant was passing functions into other functions. I do this for things like: $user_requirements = [filter1, filter2]; foreach ($user_requirements as $test) { if (!$test($some_data)) { return false; } return true; } Is there any way to tell the analyzer to make sure that afunction of n arguments is passed in?
durich 11y ago
Awesome project, Do you know where I can download a php_ast.dll for the latest php 7 for windows?
- asm 11y ago
  I'm not sure. PHP7 is pretty new so package maintainers may not have had a chance to put binary releases together. You may need to compile it yourself from https://github.com/nikic/php-ast https://github.com/nikic/php-ast.
cdnsteve 11y ago
How is performance of Phan? Nested looping checks in phase 2 scans sound intensive and bottleneck-ish.
- asm 11y ago
  Analysis takes a lot of CPU and we're not doing any parallel processing. Phan itself is about 30k lines of code and takes about 1.5s for a clean run. That being said, you should take a look at the '-s' flag for saving state to a sqlite database. After the initial run the stored state can be used to only scan changed files. For a patch with a few files changed it should be about 0.5s to run the analysis.
mcfunley 11y ago
Am I right in assuming that the name is a tribute to John Allspaw's rec.music.phish posts from the 90s? https://groups.google.com/forum/#!search/allspaw$20phriend%7Csort:relevance%7Cspell:false/rec.music.phish/8xedeCvJSAM/U9rSD7a0K54J https://groups.google.com/forum/#!search/allspaw$20phriend%7...

VeejayRampay 11y ago

PHP gets so much love and money shoved at it, the Ruby programmer in me finds it unfair. That being said, I'm really glad that this helps millions of PHP devs around the world so bravo to the people working on this.

dr_zoidberg 11y ago
PHP gets "so much love and money shoved at it" because, for a lot of problems, it's the right tool. You can always ask facebook about it.
- lmm 11y ago
  It wasn't the right tool for Facebook - Python would have been a better alternative, at least once they graduated from shared hosting. It was good enough that migration was never worthwhile, because the technology wasn't the innovative part of Facebook, but that doesn't make it good.
  - dr_zoidberg 11y ago
    Could you explain your point? Given it's success, I can't see why PHP wasn't the right tool for Facebook.
    - lmm 11y ago
      Just that Python is a better language for solving those kind of problems. I wrote a vaguely-facebook-like webapp in the same timeframe; later I worked (briefly) at last.fm (which uses, or at least used, the facebook stack) and spent most of my time working on better replacements for PHP code and issues that were caused primarily by PHP. If nothing else look at all the tools facebook produces - thrift and HHVM being the most prominent ones - to work around their PHP problems. PHP was the "right tool" in that it was the language Zuck knew when it was a one-man project. And facebook's success is a reminder that second-tier technologies can be perfectly adequate to build a successful business on, that business concerns often have a lot more impact than technical ones. It shouldn't be taken as an indication that all their technical decisions were right.

pka 11y ago

I really don't want to sound negative, I'm sure a lot of thought and work went into this, but one has to wonder - would it not be easier to just use static languages instead of writing static analyzers from scratch for every dynamic language out there?

Bulk70 11y ago
Often the problem being solved by this sort of thing isn't directly about making it easier to work with a language, but making it easier to hire developers. The hiring pool for developers who are very good with PHP is much much larger than a lot of other languages simply by virtue of it being an easy language to get started with. This is also why Facebook has spent so much time and effort on things like HHVM and Hack.
- pka 11y ago
  2 counterpoints: * Would you hire somebody who wasn't able to learn a new language in a reasonable amount of time? * Would you let somebody who just started with PHP (by virtue of it being an easy language to start with) work on your backend?
  - FranOntanaya 11y ago
    Not counterpoints. The comment above states that the pool of very good PHP developers is larger. This implies easier hiring (capacity to learn a new language is not relevant, because it works both ways), which, among other reasons, would be because they'll be applying first to PHP positions; and secondly, a very good PHP developer is far from somebody who just started.
    - pka 11y ago
      Let me rephrase. I am proposing that (barring very specific requirements which webdev doesn't justify) there's no such thing as a "good PHP developer". A good developer is a good developer, period. A good developer will learn a new language in a reasonable amount of time and will be able to be productive and write good code in it. A PHP-only (or X-only for that matter) developer is, by the definition above, not a good developer. So why restrict oneself to some random pool of X-only people?

asm 11y ago

Clarification on the title of this post; Phan is written in PHP7, but will analyze code written in any version of PHP.

debacle 11y ago
As long as that code can run on PHP7.

pbreit 11y ago

I'm guessing PHP still dominates shared hosting but would anyone ever use it on AWS, DO, Heroku, own servers, etc?

sarciszewski 11y ago
Yes.
jon-wood 11y ago
Yes. Facebook is PHP, as are Etsy, and there's no way they're the only ones. Heroku, DO, and AWS all fully support PHP, which they wouldn't bother doing if there weren't a market.
jdhawk 11y ago
Sure, why not? You can still write good applications using PHP. It not just for legacy servers and pre-installed applications (wordpress/drupal). You don't even have to use [framework of the month]. Its just a tool. Good architecture, good practices, and good code are not syntax dependent.