4 ms·
Thanks for commenting, and it is a great point. Keep in mind, we're not touting that Corgea is the fixer-all for every vulnerability. CWE's typically have simpl
by asadeddin 3y ago
Thanks for commenting, and it is a great point. Keep in mind, we're not touting that Corgea is the fixer-all for every vulnerability. CWE's typically have simpler and more standardized fixes. You can see examples of that here: https://cwe.mitre.org/ https://cwe.mitre.org/
A few things we're doing to combat this:
1) We've given the entire corpus of CWE's to Corgea, and how to fix things safely. This we've found from our testing and users does a really good job. I personally QA a lot of results (in the thousands) and we've not seen that to be a common problem.
2) Corgea is designed to require two sets of eye balls. The first being the security engineer and the second being the developer that reviews the PR. We hope, that things should be caught. Additionally, we believe our fixes will be better than what a developer does. There are over 900 CWE's, and it's really hard for engineers to know how to fix every issue. Googling answers and asking ChatGPT can lead to them introducing issues.
3) We provide in the product AI generated explanations on the fix and how it was appropriate. This is to educate non-experts on the topic.
4) We already have checks in place to make sure things aren't misbehaving, but we're rolling out soon a more advanced fix checker to make sure we didn't introduce any new vulnerabilities. Based on our testing, and
5) Finally we QA a lot every week, and run reports on the areas we're good at or not so good at to help us iterate.
- 8organicbits 3y agoI guess I'm not convinced. This is a demo that should have been chosen to show the product in the best light, but it doesn't fix the problem. Was the demo reviewed by QA? I really like the idea and I think that you're right about the goal being to fix bugs "better than the average engineer". I don't think you've reached that bar.