4 ms·

Whats vulnerable for one company, may not be for another. It needs lot of context. Low hanging thing is to update dependencies but code fix is very tricky. Gene

Whats vulnerable for one company, may not be for another. It needs lot of context. Low hanging thing is to update dependencies but code fix is very tricky. Generic best practices is not very valuable as it might conflict with the code context as well as generate code might create confusion. Historically, a generated code and human driven code is often segregated. With copilot, it is still assisting. But here, it generating and replacing code over the existing code therefore impacting the ownership of the code as well as value is not so much here. I think the PR that it would generate would endup not being merged. Having said that, it is a great hook to get attention but i think you might fail in delivering meaningful value.

asadeddin 3y ago
For your first point, we're not responsible for detecting what is vulnerable (or even exploitable). Today there are 4 categories of vulnerabilities in software: 1 - CVE's which you've mentioned around updating dependencies. This requires the least amount of context to detect, and it's easiest to change but requires a lot of context to fix downstream dependent code. 2 - CWE's which are common weaknesses in software. This requires a medium amount of context for detection and a small to medium amount of context to fix. 3 - Business and code logic flaws. This is currently unserved by most tools, and this is where the wide variance between code bases is. This requires a lot of context to both detect and fix, and it's what most people think of when it comes to your first point. 4 - Misconfiguration of environments. Currently, we've focused on CWE's because of how focused and isolated some of the fixes are in relation to items #1, #3 and #4. We've run thousands of tests and see a very high accuracy in results. We do have plans to support #1 after we feel accomplished with #2. This requires more sophisticated tools and logic to handle upgrade changes safely. At the moment, the responsibility is on the SAST tools at the moment to perform these detections. We've heard a lot of complaints about false positives and it's probably one of the biggest problems in the industry. We have future plans to tackle both detection and prioritization, but that's a separate thing. To comment on your second point, Dependabot and other code-gen products like ourselves, code ownership will be impacted. I believe our understanding of code ownership will change fundamentally as more of these tools come out and get adopted. One clarification point, Corgea doesn't auto issue PR's like Dependabot does. Someone needs to look at the fix, before issuing a PR. Thanks for commenting, and it's definitely a different perspective on meaning value. For other code-gen tools like copilot, where do you think the value is then?

asadeddin 3y ago

For your first point, we're not responsible for detecting what is vulnerable (or even exploitable). Today there are 4 categories of vulnerabilities in software: 1 - CVE's which you've mentioned around updating dependencies. This requires the least amount of context to detect, and it's easiest to change but requires a lot of context to fix downstream dependent code. 2 - CWE's which are common weaknesses in software. This requires a medium amount of context for detection and a small to medium amount of context to fix. 3 - Business and code logic flaws. This is currently unserved by most tools, and this is where the wide variance between code bases is. This requires a lot of context to both detect and fix, and it's what most people think of when it comes to your first point. 4 - Misconfiguration of environments. Currently, we've focused on CWE's because of how focused and isolated some of the fixes are in relation to items #1, #3 and #4. We've run thousands of tests and see a very high accuracy in results. We do have plans to support #1 after we feel accomplished with #2. This requires more sophisticated tools and logic to handle upgrade changes safely. At the moment, the responsibility is on the SAST tools at the moment to perform these detections. We've heard a lot of complaints about false positives and it's probably one of the biggest problems in the industry. We have future plans to tackle both detection and prioritization, but that's a separate thing. To comment on your second point, Dependabot and other code-gen products like ourselves, code ownership will be impacted. I believe our understanding of code ownership will change fundamentally as more of these tools come out and get adopted. One clarification point, Corgea doesn't auto issue PR's like Dependabot does. Someone needs to look at the fix, before issuing a PR. Thanks for commenting, and it's definitely a different perspective on meaning value. For other code-gen tools like copilot, where do you think the value is then?