10 ms·

Vulnerability Management for Go

365 points by mfrw 4y ago

elgolem89 4y ago
What's the difference between this a https://github.com/securego/gosec https://github.com/securego/gosec?
goalieca 4y ago
So shall we have a chat about go.sum files yet? Several scanners seem to choke on the fact that go.sum includes several versions of each module, some of which are vulnerable, but basically never actually packaged. Am i missing a reason why go did not go with a lockfile as nearly every other modern language?
- ainar-g 4y ago
  It's not a lockfile. It contains hashes of packages so that you could fetch them from any proxy and not worry about proxies doing funny business.
  - jrockway 4y ago
    Even ignoring proxies, it ensures that you have the same code that you started developing with. If you're depending on a tagged Git repository, for example, someone can just force push the tags and change what commit go.mod is actually pointing at. With go.sum, you're guaranteed to be notified that that happened.
    - svnpenn 4y ago
      great point. I am glad they took this into consideration.
- shp0ngle 4y ago
  go.mod IS a lockfile. go.mod will exactly tell you what version you have, always. * * that does not apply if your code is a library; then, other dependency of the "main" app can "update" your dependency. Which sometimes does break things. However, the go.mod of the main app will deterministically tell what version is used, always. I never did understand what packages go to go.sum though and what is the logic there. But it is not that important.
  - preseinger 4y ago
    This is only true for go 1.17 and above. Prior to that, go.mod wouldn't necessarily list all transitive dependencies. And I don't think this property is actually guaranteed by the relevant specification. The only reliable way to get a "lock file" is to run go list. Unfortunately.
  - indil 4y ago
    go.mod lists minimum versions. Minimum Version Selection may increase the versions used as required by other packages in the build. go.mod isn't a lock file.
verst 4y ago
This is super helpful for me as a Dapr maintainer (we have a ton of third party integrations we compile into our binary). As others mentioned - other tools can generate a lot of noise. Found and upgraded a vulnerable dependency and then quickly added this check to our CI/CD workflow. https://github.com/dapr/components-contrib/pull/2054 https://github.com/dapr/components-contrib/pull/2054
waihtis 4y ago
Attackers now scrambling onto Github to run this against open projects
- deleted 4y ago
  [deleted]
- tgv 4y ago
  It's checks against a list of known vulnerabilities. If you search the go.mod or .sum files on github for them, you have an more powerful route, I think.
  - waihtis 4y ago
    Hmm, thought the problem with that was poor fidelity.. from the top comment: > go.sum… ends up reporting a lot of vulnerabilities that you don't actually care about because your code isn't actually linking to the optional transitive dependency that has the vulnerability in it. Not a Go dev myself though.
  - artitars 4y ago
    Even though a search based on go.mod or .sum files should definitely result in more hits, the new vulnerability management should be way more efficient, because it is already filtered for more relevant hits and should thus contain more signal vs noise as it only returns at least transitively used vulns. Am I missing something here?
  - preseinger 4y ago
    Neither of those files reliably represents the actual dep graph of your program.
  - smoyer 4y ago
    The vulnerability list is open source and you can get a list of public projects that use each dependency from pkg.go.dev - if you really want a list, there are at least a few ways that are more efficient than scanning GitHub! (As you pointed out above)
tzahifadida 4y ago
Thanks. Looks good!
topicseed 4y ago
This is amazing! And only raises relevant issues so we avoid alert fatigue and can actually solve whatever real problems are flagged.
BeefWellington 4y ago
It's a great theoretical idea but in terms of practice unless all third-party scanners adopt the same process it won't be relevant I suspect. It also places all of your trust in the accuracy of the data submitted to govulncheck.
Arbortheus 4y ago
This is a great feature! The company I work for has spent a lot of money trying to use Snyk for this purpose, but it really sucks, and chokes on most of our Go repos go.mod files meaning some of our most important repositories are blind for vulnerability scanning. We don't get the static code analysis feature from Snyk (where vulnerabilities are raised only when they affect us), because this is an optional paid extra. Now we get it for free! Snyk really need to up their game with their lacklustre Go support to compete with free.
- freeqaz 4y ago
  Note: I run a vuln scanning company in this space (our GitHub[0]), so I can add some context to this. GitHub has this for Dependabot via CodeQL which is a part of their "Advanced Security" package. ($$$, ie "contact us", but I've heard ~$1-1.5k per dev per year roughly) Other big ones are SonaType, FOSSA, and Snyk (which OP mentioned). There are some smaller vendors too which I can add if anybody is curious. GitHub CodeQL is by far the best and their tech comes from their acquisition of Semmle/LGTM a few years ago. When I was at Uber, we used Semmle to augment the efforts of the bug bounty and to "scale" our AppSec team to keep up with an ever growing engineering team. The lack of flexibility with CodeQL and the other proprietary scanning tools is actually the reason why, when I decided to start a security company, I decided to center the company around building publicly on GitHub. It's harder to make money, at least initially, but it's also the "right" choice to actually push this industry forward. (The thought on how to make money is a hosted SaaS, of course.) Anyway, I'm happy to answer any questions around this stuff! 0: https://github.com/lunasec-io/lunasec https://github.com/lunasec-io/lunasec
brainphreeze 4y ago
Has anyone tested this? I'm testing it on a project with known vulnerabilities and am getting positive matches when testing against a go binary (govulncheck binary), but not when ran within a directory using: govulncheck .
- ainar-g 4y ago
  The pattern to check the package and all subpackages for pretty much all Go tools is "./...". So: govulncheck ./...
synergy20 4y ago
Modules, Generics, and now Govulncheck are all enterprise-welcome features, hope this can help Golang to gain more 'market share', at present it remains at around 1%(Python 15%, C|C++|Java each about 10%) in programming language TIOBE ranking, it remains to be an uphill battle to move up. I spent quite sometime learning golang while using c++ at job, now I pressed the pause button on golang and decided to focus more on modern c++ instead, yeah its tooling is lacking comparing to golang, but it's just so widely used and too good on performance, will revisit Golang later.
- eliben 4y ago
  Please, please don't use tiobe for programming language comparisons. https://blog.nindalf.com/posts/stop-citing-tiobe/ https://blog.nindalf.com/posts/stop-citing-tiobe/, etc.
  - synergy20 4y ago
    well google does give multiple rankings, the exact number differ, but the basic order remains. some top hits: https://spectrum.ieee.org/top-programming-languages/ https://www.cleveroad.com/blog/programming-languages-ranking/ https://distantjob.com/blog/programming-languagesrank/ https://statisticstimes.com/tech/top-computer-languages.php
    - eliben 4y ago
      2 of these refer to TIOBE and another is a 404 The IEEE Spectrum one seems independent
bert2002 4y ago
I hope they will release a rss feed for the database
psanford 4y ago
> Govulncheck analyzes your codebase and only surfaces vulnerabilities that actually affect you, based on which functions in your code are transitively calling vulnerable functions. This is huge! Every existing vulnerability scanner that I've worked with have just looked at go.sum, which ends up reporting a lot of vulnerabilities that you don't actually care about because your code isn't actually linking to the optional transitive dependency that has the vulnerability in it. Thank you, Go team!
- auspex 4y ago
  - psanford 4y ago
    What? How does a piece of code that isn't compiled into your binary make you vulnerable?
    - auspex 4y ago
      I should have been more clear that I was referring to external dependencies that are included. I did some research and there’s so much extra stuff included in the go.sum. You’re right that would result in a lot of false positives.
  - umvi 4y ago
    Well if RCE is possible, then why is it going to matter if you have uncalled vulnerable functions or not? Attacker can execute anything they want.
    - Someone 4y ago
      Not necessarily. RCE may start with being able to run ‘something’ that has restrictions such as “can’t inject code that has zero bytes in it” or “injected code can only be X bytes long”. In such cases, having another vulnerability available may be the easiest way to get rid of those restrictions. Also, the second vulnerability may be complementary. For example, the first may get you onto the machine, but not out of the sandbox, while the second won’t get you on the machine, but will get you out of the sandbox. In this case, I think the go linker won’t include the never-called vulnerable function in the executable (it only would if the vulnerability checker were smarter than the linker in detecting never-called code. That’s theoretically possible, but highly unlikely)

365 points by mfrw 4y ago

What's the difference between this a https://github.com/securego/gosec https://github.com/securego/gosec?

So shall we have a chat about go.sum files yet? Several scanners seem to choke on the fact that go.sum includes several versions of each module, some of which are vulnerable, but basically never actually packaged. Am i missing a reason why go did not go with a lockfile as nearly every other modern language?

ainar-g 4y ago
It's not a lockfile. It contains hashes of packages so that you could fetch them from any proxy and not worry about proxies doing funny business.
- jrockway 4y ago
  Even ignoring proxies, it ensures that you have the same code that you started developing with. If you're depending on a tagged Git repository, for example, someone can just force push the tags and change what commit go.mod is actually pointing at. With go.sum, you're guaranteed to be notified that that happened.
  - svnpenn 4y ago
    great point. I am glad they took this into consideration.
shp0ngle 4y ago
go.mod IS a lockfile. go.mod will exactly tell you what version you have, always. * * that does not apply if your code is a library; then, other dependency of the "main" app can "update" your dependency. Which sometimes does break things. However, the go.mod of the main app will deterministically tell what version is used, always. I never did understand what packages go to go.sum though and what is the logic there. But it is not that important.
- preseinger 4y ago
  This is only true for go 1.17 and above. Prior to that, go.mod wouldn't necessarily list all transitive dependencies. And I don't think this property is actually guaranteed by the relevant specification. The only reliable way to get a "lock file" is to run go list. Unfortunately.
- indil 4y ago
  go.mod lists minimum versions. Minimum Version Selection may increase the versions used as required by other packages in the build. go.mod isn't a lock file.

verst 4y ago

This is super helpful for me as a Dapr maintainer (we have a ton of third party integrations we compile into our binary). As others mentioned - other tools can generate a lot of noise. Found and upgraded a vulnerable dependency and then quickly added this check to our CI/CD workflow. https://github.com/dapr/components-contrib/pull/2054 https://github.com/dapr/components-contrib/pull/2054

waihtis 4y ago

Attackers now scrambling onto Github to run this against open projects

deleted 4y ago
[deleted]
tgv 4y ago
It's checks against a list of known vulnerabilities. If you search the go.mod or .sum files on github for them, you have an more powerful route, I think.
- waihtis 4y ago
  Hmm, thought the problem with that was poor fidelity.. from the top comment: > go.sum… ends up reporting a lot of vulnerabilities that you don't actually care about because your code isn't actually linking to the optional transitive dependency that has the vulnerability in it. Not a Go dev myself though.
- artitars 4y ago
  Even though a search based on go.mod or .sum files should definitely result in more hits, the new vulnerability management should be way more efficient, because it is already filtered for more relevant hits and should thus contain more signal vs noise as it only returns at least transitively used vulns. Am I missing something here?
- preseinger 4y ago
  Neither of those files reliably represents the actual dep graph of your program.
- smoyer 4y ago
  The vulnerability list is open source and you can get a list of public projects that use each dependency from pkg.go.dev - if you really want a list, there are at least a few ways that are more efficient than scanning GitHub! (As you pointed out above)

tzahifadida 4y ago

Thanks. Looks good!

topicseed 4y ago

This is amazing! And only raises relevant issues so we avoid alert fatigue and can actually solve whatever real problems are flagged.

BeefWellington 4y ago

It's a great theoretical idea but in terms of practice unless all third-party scanners adopt the same process it won't be relevant I suspect. It also places all of your trust in the accuracy of the data submitted to govulncheck.

Arbortheus 4y ago

This is a great feature! The company I work for has spent a lot of money trying to use Snyk for this purpose, but it really sucks, and chokes on most of our Go repos go.mod files meaning some of our most important repositories are blind for vulnerability scanning. We don't get the static code analysis feature from Snyk (where vulnerabilities are raised only when they affect us), because this is an optional paid extra. Now we get it for free! Snyk really need to up their game with their lacklustre Go support to compete with free.

freeqaz 4y ago
Note: I run a vuln scanning company in this space (our GitHub[0]), so I can add some context to this. GitHub has this for Dependabot via CodeQL which is a part of their "Advanced Security" package. ($$$, ie "contact us", but I've heard ~$1-1.5k per dev per year roughly) Other big ones are SonaType, FOSSA, and Snyk (which OP mentioned). There are some smaller vendors too which I can add if anybody is curious. GitHub CodeQL is by far the best and their tech comes from their acquisition of Semmle/LGTM a few years ago. When I was at Uber, we used Semmle to augment the efforts of the bug bounty and to "scale" our AppSec team to keep up with an ever growing engineering team. The lack of flexibility with CodeQL and the other proprietary scanning tools is actually the reason why, when I decided to start a security company, I decided to center the company around building publicly on GitHub. It's harder to make money, at least initially, but it's also the "right" choice to actually push this industry forward. (The thought on how to make money is a hosted SaaS, of course.) Anyway, I'm happy to answer any questions around this stuff! 0: https://github.com/lunasec-io/lunasec https://github.com/lunasec-io/lunasec

brainphreeze 4y ago

Has anyone tested this? I'm testing it on a project with known vulnerabilities and am getting positive matches when testing against a go binary (govulncheck binary), but not when ran within a directory using: govulncheck .

ainar-g 4y ago
The pattern to check the package and all subpackages for pretty much all Go tools is "./...". So: govulncheck ./...

synergy20 4y ago

Modules, Generics, and now Govulncheck are all enterprise-welcome features, hope this can help Golang to gain more 'market share', at present it remains at around 1%(Python 15%, C|C++|Java each about 10%) in programming language TIOBE ranking, it remains to be an uphill battle to move up. I spent quite sometime learning golang while using c++ at job, now I pressed the pause button on golang and decided to focus more on modern c++ instead, yeah its tooling is lacking comparing to golang, but it's just so widely used and too good on performance, will revisit Golang later.

eliben 4y ago
Please, please don't use tiobe for programming language comparisons. https://blog.nindalf.com/posts/stop-citing-tiobe/ https://blog.nindalf.com/posts/stop-citing-tiobe/, etc.
- synergy20 4y ago
  well google does give multiple rankings, the exact number differ, but the basic order remains. some top hits: https://spectrum.ieee.org/top-programming-languages/ https://www.cleveroad.com/blog/programming-languages-ranking/ https://distantjob.com/blog/programming-languagesrank/ https://statisticstimes.com/tech/top-computer-languages.php
  - eliben 4y ago
    2 of these refer to TIOBE and another is a 404 The IEEE Spectrum one seems independent

bert2002 4y ago

I hope they will release a rss feed for the database

psanford 4y ago

> Govulncheck analyzes your codebase and only surfaces vulnerabilities that actually affect you, based on which functions in your code are transitively calling vulnerable functions. This is huge! Every existing vulnerability scanner that I've worked with have just looked at go.sum, which ends up reporting a lot of vulnerabilities that you don't actually care about because your code isn't actually linking to the optional transitive dependency that has the vulnerability in it. Thank you, Go team!

auspex 4y ago
- psanford 4y ago
  What? How does a piece of code that isn't compiled into your binary make you vulnerable?
  - auspex 4y ago
    I should have been more clear that I was referring to external dependencies that are included. I did some research and there’s so much extra stuff included in the go.sum. You’re right that would result in a lot of false positives.
- umvi 4y ago
  Well if RCE is possible, then why is it going to matter if you have uncalled vulnerable functions or not? Attacker can execute anything they want.
  - Someone 4y ago
    Not necessarily. RCE may start with being able to run ‘something’ that has restrictions such as “can’t inject code that has zero bytes in it” or “injected code can only be X bytes long”. In such cases, having another vulnerability available may be the easiest way to get rid of those restrictions. Also, the second vulnerability may be complementary. For example, the first may get you onto the machine, but not out of the sandbox, while the second won’t get you on the machine, but will get you out of the sandbox. In this case, I think the go linker won’t include the never-called vulnerable function in the executable (it only would if the vulnerability checker were smarter than the linker in detecting never-called code. That’s theoretically possible, but highly unlikely)