30 ms·

Is this meant to compliment or compete with semgrep?

by beardedwizard 3y ago

andrewshadura 3y ago
Well, it is semgrep (hence sg).
- beardedwizard 3y ago
  yeah I had this feeling a bit, I guess im curious what problems they solve differently (if any). My sense it that semgrep is an enterprise managed solution of the same kind (and btw, is still itself OSS)
ekidd 3y ago
Well, when I seach for "semgrep", I get a very nice corporate landing page with a "Book Demo" button. Which is a level of hassle that just isn't worth it for smaller teams, because "Book Demo" usually means "We're going to do a dance to see how much money we can extract from you." Which smaller teams may only want to do for a handful of key tools. (4 years ago, I was more willing to put up with enterprise licensing. But in the last two years, I've seen way too many enterprise vendors try to squeeze every penny they can get from existing clients. An enterprise sales process now often means "Expect 30% annual price hikes once you're in too deep to back out." The lack of easy VC money seems to have made some enterprise vendors pretty desperate.) There's also an open source "semgrep" project here: https://github.com/semgrep/semgrep https://github.com/semgrep/semgrep. But this seems to be basically a vulernability scanner, going by the README. Whereas AST-grep seems to focus heavily on things like: 1. One-off searching: "Search my tree for this pattern." 2. Refactoring: "Replace this pattern with this other pattern." AST-grep also includes a vulnerability scanning mode like semgrep. It's possible that semgrep also has nice support for (1) and (2), but it isn't clearly visible on their corporate landing page or the first open source README I found.
- icholy 3y ago
  Semgrep is capable of one-off searching and refactoring. I agreed that the docs are a little hard to navigate.
- herrington_d 3y ago
  Thank ekidd for your kind words! ast-grep author here. This is a hobby project and mainly focuses on developers' daily job like search and linting. Appreciate you like it! Semgrep's vulnerability scanning is much more advanced, mostly for enterprise security usage.
icholy 3y ago
Looks like a competitor to me.
herrington_d 3y ago
Hi, ast-grep author here. This is a great question and I asked this in the first place before I started the hobby project. TLDR; I designed ast-grep to be on different tracks than semgrep. Semgrep is for security and ast-grep is for development. First and foremost, I have always been in awe of semgrep. Semgrep's documentation, product sites and Padioleau's podcast all gave me a lot of inspiration. Using code to find code is such a cool idea that I never need to craft an intricate regex or write a lengthy AST program. sgrep and patch from https://github.com/facebookarchive/pfff/wiki/Sgrep https://github.com/facebookarchive/pfff/wiki/Sgrep have helped me a lot in real large codebases. When I used semgrep as a software engineer, instead of a security researcher, I found semgrep has not touched too much on routine development works. I can use `semgrep -e PATTERN` but the Python wrapper is not too fast compared to grep. While pattern is cool, it cannot precisely match some syntax nodes. (example, selecting generator expression in Semgrep is very hard). It also does not have API to find code programmatically. I have also a short summary for tool comparison. https://ast-grep.github.io/advanced/tool-comparison.html https://ast-grep.github.io/advanced/tool-comparison.html
- herrington_d 3y ago
  Why I think semgrep is a security tool different from ast-grep: * Semgrep is security focused. It has many advanced static analysis features in its core product, such as dataflow analysis, symbolic propagation, and semantic equivalence, all of which are useful for security analysis. They are not available in ast-grep. * Semgrep’s pattern syntax also prefers matching more potentially vulnerable semantics than matching precise syntax. Semantic level information is the better level of abstraction for security model. ast-grep, on the other hand, sticks to faithfully translating users' queries syntactically. * Semgrep has a one-off search and rewrite feature, but it is not its primary focus. The CLI is a bit slow compared to other tools. ast-grep strives to be a fast CLI tool. * Semgrep has a product matrix for vulnerability detection: detecting secrets, supply chain vulnerabilities, and cross-file detection. It also has a plethora of security rules in the registry. These features will not be included in ast-grep.

Is this meant to compliment or compete with semgrep?

andrewshadura 3y ago

Well, it is semgrep (hence sg).

beardedwizard 3y ago
yeah I had this feeling a bit, I guess im curious what problems they solve differently (if any). My sense it that semgrep is an enterprise managed solution of the same kind (and btw, is still itself OSS)

ekidd 3y ago

Well, when I seach for "semgrep", I get a very nice corporate landing page with a "Book Demo" button. Which is a level of hassle that just isn't worth it for smaller teams, because "Book Demo" usually means "We're going to do a dance to see how much money we can extract from you." Which smaller teams may only want to do for a handful of key tools. (4 years ago, I was more willing to put up with enterprise licensing. But in the last two years, I've seen way too many enterprise vendors try to squeeze every penny they can get from existing clients. An enterprise sales process now often means "Expect 30% annual price hikes once you're in too deep to back out." The lack of easy VC money seems to have made some enterprise vendors pretty desperate.) There's also an open source "semgrep" project here: https://github.com/semgrep/semgrep https://github.com/semgrep/semgrep. But this seems to be basically a vulernability scanner, going by the README. Whereas AST-grep seems to focus heavily on things like: 1. One-off searching: "Search my tree for this pattern." 2. Refactoring: "Replace this pattern with this other pattern." AST-grep also includes a vulnerability scanning mode like semgrep. It's possible that semgrep also has nice support for (1) and (2), but it isn't clearly visible on their corporate landing page or the first open source README I found.

icholy 3y ago
Semgrep is capable of one-off searching and refactoring. I agreed that the docs are a little hard to navigate.
herrington_d 3y ago
Thank ekidd for your kind words! ast-grep author here. This is a hobby project and mainly focuses on developers' daily job like search and linting. Appreciate you like it! Semgrep's vulnerability scanning is much more advanced, mostly for enterprise security usage.

icholy 3y ago

Looks like a competitor to me.

herrington_d 3y ago

Hi, ast-grep author here. This is a great question and I asked this in the first place before I started the hobby project. TLDR; I designed ast-grep to be on different tracks than semgrep. Semgrep is for security and ast-grep is for development. First and foremost, I have always been in awe of semgrep. Semgrep's documentation, product sites and Padioleau's podcast all gave me a lot of inspiration. Using code to find code is such a cool idea that I never need to craft an intricate regex or write a lengthy AST program. sgrep and patch from https://github.com/facebookarchive/pfff/wiki/Sgrep https://github.com/facebookarchive/pfff/wiki/Sgrep have helped me a lot in real large codebases. When I used semgrep as a software engineer, instead of a security researcher, I found semgrep has not touched too much on routine development works. I can use `semgrep -e PATTERN` but the Python wrapper is not too fast compared to grep. While pattern is cool, it cannot precisely match some syntax nodes. (example, selecting generator expression in Semgrep is very hard). It also does not have API to find code programmatically. I have also a short summary for tool comparison. https://ast-grep.github.io/advanced/tool-comparison.html https://ast-grep.github.io/advanced/tool-comparison.html

herrington_d 3y ago
Why I think semgrep is a security tool different from ast-grep: * Semgrep is security focused. It has many advanced static analysis features in its core product, such as dataflow analysis, symbolic propagation, and semantic equivalence, all of which are useful for security analysis. They are not available in ast-grep. * Semgrep’s pattern syntax also prefers matching more potentially vulnerable semantics than matching precise syntax. Semantic level information is the better level of abstraction for security model. ast-grep, on the other hand, sticks to faithfully translating users' queries syntactically. * Semgrep has a one-off search and rewrite feature, but it is not its primary focus. The CLI is a bit slow compared to other tools. ast-grep strives to be a fast CLI tool. * Semgrep has a product matrix for vulnerability detection: detecting secrets, supply chain vulnerabilities, and cross-file detection. It also has a plethora of security rules in the registry. These features will not be included in ast-grep.