5 ms·

While I agree in general, some forms of code reviewing require less context than others. They say "We review within your tools to catch security threats, stop c

While I agree in general, some forms of code reviewing require less context than others. They say "We review within your tools to catch security threats, stop crashes, and fix performance issues before they reach production.", and it does seem to me that these things are less a matter of style/context than just noticing potential issues. Not sure though, I had the exact same thoughts as you when first seeing this. Edit: to be clear, I personally don't see the value here, just playing devil's advocate.

jermaustin1 5y ago
But almost any good SA can catch all of those issues, a person doesn't need to look through the code for that.
- senko 5y ago
  This is very close to saying Halting problem is not, in fact, a problem: https://brilliant.org/wiki/halting-problem/ https://brilliant.org/wiki/halting-problem/
  - jermaustin1 5y ago
    Not at all, it is saying that if the only things this service is providing is a human doing SA (since they dont have any deep-knowledge of the codebase), then you could just use an SA. Code review in the real world misses the Halting problem, the only way you can really see it is if you know the codebase well enough to SEE it, and any 3rd party that is given a pull request to complete in a timely manner will not have time to fully learn your code base or even the modules you are submitting.
  - adgjlsfhk1 5y ago
    This isn't quite true because in most real world scenarios, one of the requirements for code to be correct is that it provably halts in a fairly limited amount of time.
  - reuben364 5y ago
    A human being doesn't solve the halting problem either.
  - samatman 5y ago
    Ironically, it very seldom is. It's Rice's Theorem which tends to get me in trouble!
HPsquared 5y ago
What will actually happen, is people will use these kinds of "context-free code review" services, then say a code review has been done. Technically correct in a narrow way, but not what most people would expect.
malux85 5y ago
I suspect that there’s only a tiny, tiny fraction of issues that are above the complexity that a linter and test framework can’t identify, but are below the complexity that requires deep codebase knowledge, and I would say that the overwhelming majority of value of code reviews is inside the issues that require deep codebase knowledge. Maybe this service could be good to point out superficial security bugs though, but the infrequency of these coupled with the effort of external human engagement I think would be a barrier. Also a few times in my career I’ve seen a team with no senior devs on it, they usually fail due to inexperience, maybe this service could help a team like this - who are producing a lot of really obvious mistakes …
alkonaut 5y ago
Code the falls between between "I need an experienced team member to see the issue" and "A few static analysis tools would no doubt have seen it" must be vanishingly small. For specific areas, I can see myself paying for external code review. For example some open source library authors sell it as a service, so they can review the specific parts of the code where the library is used. Rob Menschings FireGiant is one such example. Security reviews (Audits) I can also see a good use for. The current practice here is already to have external non-domain-experts review the code. So making that simpler or more frequent is a net win. The reviews I see little use for would be normal feature code, (pull request reviews) which need a lot of context and domain knowledge to even begin to review. Not audits, or reviews of specific areas or aspects of the code.

jermaustin1 5y ago

But almost any good SA can catch all of those issues, a person doesn't need to look through the code for that.

senko 5y ago
This is very close to saying Halting problem is not, in fact, a problem: https://brilliant.org/wiki/halting-problem/ https://brilliant.org/wiki/halting-problem/
- jermaustin1 5y ago
  Not at all, it is saying that if the only things this service is providing is a human doing SA (since they dont have any deep-knowledge of the codebase), then you could just use an SA. Code review in the real world misses the Halting problem, the only way you can really see it is if you know the codebase well enough to SEE it, and any 3rd party that is given a pull request to complete in a timely manner will not have time to fully learn your code base or even the modules you are submitting.
- adgjlsfhk1 5y ago
  This isn't quite true because in most real world scenarios, one of the requirements for code to be correct is that it provably halts in a fairly limited amount of time.
- reuben364 5y ago
  A human being doesn't solve the halting problem either.
- samatman 5y ago
  Ironically, it very seldom is. It's Rice's Theorem which tends to get me in trouble!

HPsquared 5y ago

What will actually happen, is people will use these kinds of "context-free code review" services, then say a code review has been done. Technically correct in a narrow way, but not what most people would expect.

malux85 5y ago

I suspect that there’s only a tiny, tiny fraction of issues that are above the complexity that a linter and test framework can’t identify, but are below the complexity that requires deep codebase knowledge, and I would say that the overwhelming majority of value of code reviews is inside the issues that require deep codebase knowledge. Maybe this service could be good to point out superficial security bugs though, but the infrequency of these coupled with the effort of external human engagement I think would be a barrier. Also a few times in my career I’ve seen a team with no senior devs on it, they usually fail due to inexperience, maybe this service could help a team like this - who are producing a lot of really obvious mistakes …

alkonaut 5y ago

Code the falls between between "I need an experienced team member to see the issue" and "A few static analysis tools would no doubt have seen it" must be vanishingly small. For specific areas, I can see myself paying for external code review. For example some open source library authors sell it as a service, so they can review the specific parts of the code where the library is used. Rob Menschings FireGiant is one such example. Security reviews (Audits) I can also see a good use for. The current practice here is already to have external non-domain-experts review the code. So making that simpler or more frequent is a net win. The reviews I see little use for would be normal feature code, (pull request reviews) which need a lot of context and domain knowledge to even begin to review. Not audits, or reviews of specific areas or aspects of the code.