5 ms·

* Build farm consisting of (tens?) of thousands of servers * Automated presubmit/bisect queues * Well built code review ui and not like github On this topic,

* Build farm consisting of (tens?) of thousands of servers * Automated presubmit/bisect queues * Well built code review ui and not like github On this topic, most companies will give up on “rigorous code reviews”.

brightball 4y ago
I’m curious if you can think of any tools with a well built code review UI?
- rockwotj 4y ago
  Our company uses codeapprove.com to get a critique (Google's internal review tool) like experience. It's not all the way there, but miles ahead of GitHub
  - brightball 4y ago
    Thanks, I’ll check it out!
    - codeapprove 4y ago
      CodeApprove founder here! If you want to chat, my email is in my HN profile.
      - brightball 4y ago
        Thanks!
  - gruez 4y ago
    Based on the screenshots on codeapprove.com it looks similar to gerrit[1]. It also looks like gerrit is used for several google projects (eg. android, chromium), so overall it looks like a potential replacement as well. Are there any features that are in codeapprove.com/critique that aren't in gerrit? [1] https://en.wikipedia.org/wiki/Gerrit_(software) https://en.wikipedia.org/wiki/Gerrit_(software)
    - rockwotj 4y ago
      I have never used Gerrit, so I can't speak to it directly. My memory from folk who have worked on both seemed to prefer critique, but I am not sure if that's because of the tight integration with other tools like codesearch, linters, etc or something more fundamental to the review tool itself.
      - surajrmal 4y ago
        Gerrit UX has historically been subpar compared to critique, however it's seen a lot of investment in recent years afaict. A lot of the remaining benefits of critique are largely due to its tight integrations with other Google tools. Critique is fairly malleable so it's possible with enough effort to get a similar experience by integrating with your own tools, but it's definitely something that needs to be staffed to work that well.
- dilyevsky 4y ago
  Gerrit was a rewrite/partial opensourcing of previous generation of Google’s code review tool (Mondrian) so imo it’s the closest thing to the real deal.
- bvrmn 4y ago
  I can highly recommend gerrit. It can lack of eye candy, but from UX perspective it's the most effective tool I've tried. Also it forces patch-oriented workflow from a GIT side and you get a nice linear history in the end.
0x20cowboy 4y ago
I’d be curious what these super secret tools do. Most “rigorous code reviews” I’ve seen are junior devs fighting over “tabs vs spaces” or “doing a for loop vs a for each” - or something else borderline meaningless. I’d love a tool that helped with context in a larger system design way. Coming in cold on some project and expecting to know all the design decisions, and if the code structure is going away from some higher level goal is where most problems that I’ve seen come in. I don’t see how any of the current tools help with that, and I somewhat doubt these secret tools do either - but would love to seen them if they do.
- rwiggins 4y ago
  One random thing I've been impressed with (that I know is public) is mutant testing: https://testing.googleblog.com/2021/04/mutation-testing.html https://testing.googleblog.com/2021/04/mutation-testing.html , https://research.google/pubs/pub46584/ https://research.google/pubs/pub46584/ To pull an example from the paper, if your line of code says if (a == b || b == 1) you might get a comment that says something like > "Changing this line to if (a != b || b == 1) > does not cause any test to fail." Page 4 lists other mutations, like replacing logical ANDs/ORs with just `true` or `false`, switching arithmetic plus to minus, etc. This isn't a larger system design context thing, though, just a testing one. Disclosure: I work at Google.
  - technion 4y ago
    There's a pretty good Ruby gem I've used for this before: https://github.com/mbj/mutant https://github.com/mbj/mutant
  - YZF 4y ago
    This is for test coverage? I get it technically but not clear to me if this adds value. Covering all the internal states/code paths in tests seems very hard and potentially of diminishing returns? Isn't time better invested in other things? I'm also guessing this ties into Google's tooling of being able to tell which line of code is hit by which tests and running those as part of the mutation test? EDIT: The blog has some discussion about the correlation between some of these mutation failures and real bugs. I remain a little suspicious to be honest. Also "I also looked into the developer behavior changes after using mutation testing on a project for longer periods of time, and discovered that projects that use mutation testing get more tests over time, as developers get exposed to more and more mutants. Not only do developers write more test cases, but those test cases are more effective in killing mutants: less and less mutants get reported over time. I noticed this from personal experience too: when writing unit tests, I would see where I cut some corners in the tests, and anticipated the mutant. Now I just add the missing test cases, rather than facing a mutant in my Code review, and I rarely see mutants these days, as I’ve learned to anticipate and preempt them." Now this one is expected, if you build automated tooling that comments on your code review you expect people to try and avoid that. However there's still the question of what's the quality improvement and what's the productivity impact.

brightball 4y ago

I’m curious if you can think of any tools with a well built code review UI?

rockwotj 4y ago
Our company uses codeapprove.com to get a critique (Google's internal review tool) like experience. It's not all the way there, but miles ahead of GitHub
- brightball 4y ago
  Thanks, I’ll check it out!
  - codeapprove 4y ago
    CodeApprove founder here! If you want to chat, my email is in my HN profile.
    - brightball 4y ago
      Thanks!
- gruez 4y ago
  Based on the screenshots on codeapprove.com it looks similar to gerrit[1]. It also looks like gerrit is used for several google projects (eg. android, chromium), so overall it looks like a potential replacement as well. Are there any features that are in codeapprove.com/critique that aren't in gerrit? [1] https://en.wikipedia.org/wiki/Gerrit_(software) https://en.wikipedia.org/wiki/Gerrit_(software)
  - rockwotj 4y ago
    I have never used Gerrit, so I can't speak to it directly. My memory from folk who have worked on both seemed to prefer critique, but I am not sure if that's because of the tight integration with other tools like codesearch, linters, etc or something more fundamental to the review tool itself.
    - surajrmal 4y ago
      Gerrit UX has historically been subpar compared to critique, however it's seen a lot of investment in recent years afaict. A lot of the remaining benefits of critique are largely due to its tight integrations with other Google tools. Critique is fairly malleable so it's possible with enough effort to get a similar experience by integrating with your own tools, but it's definitely something that needs to be staffed to work that well.
dilyevsky 4y ago
Gerrit was a rewrite/partial opensourcing of previous generation of Google’s code review tool (Mondrian) so imo it’s the closest thing to the real deal.
bvrmn 4y ago
I can highly recommend gerrit. It can lack of eye candy, but from UX perspective it's the most effective tool I've tried. Also it forces patch-oriented workflow from a GIT side and you get a nice linear history in the end.

0x20cowboy 4y ago

I’d be curious what these super secret tools do. Most “rigorous code reviews” I’ve seen are junior devs fighting over “tabs vs spaces” or “doing a for loop vs a for each” - or something else borderline meaningless. I’d love a tool that helped with context in a larger system design way. Coming in cold on some project and expecting to know all the design decisions, and if the code structure is going away from some higher level goal is where most problems that I’ve seen come in. I don’t see how any of the current tools help with that, and I somewhat doubt these secret tools do either - but would love to seen them if they do.

rwiggins 4y ago
One random thing I've been impressed with (that I know is public) is mutant testing: https://testing.googleblog.com/2021/04/mutation-testing.html https://testing.googleblog.com/2021/04/mutation-testing.html , https://research.google/pubs/pub46584/ https://research.google/pubs/pub46584/ To pull an example from the paper, if your line of code says if (a == b || b == 1) you might get a comment that says something like > "Changing this line to if (a != b || b == 1) > does not cause any test to fail." Page 4 lists other mutations, like replacing logical ANDs/ORs with just `true` or `false`, switching arithmetic plus to minus, etc. This isn't a larger system design context thing, though, just a testing one. Disclosure: I work at Google.
- technion 4y ago
  There's a pretty good Ruby gem I've used for this before: https://github.com/mbj/mutant https://github.com/mbj/mutant
- YZF 4y ago
  This is for test coverage? I get it technically but not clear to me if this adds value. Covering all the internal states/code paths in tests seems very hard and potentially of diminishing returns? Isn't time better invested in other things? I'm also guessing this ties into Google's tooling of being able to tell which line of code is hit by which tests and running those as part of the mutation test? EDIT: The blog has some discussion about the correlation between some of these mutation failures and real bugs. I remain a little suspicious to be honest. Also "I also looked into the developer behavior changes after using mutation testing on a project for longer periods of time, and discovered that projects that use mutation testing get more tests over time, as developers get exposed to more and more mutants. Not only do developers write more test cases, but those test cases are more effective in killing mutants: less and less mutants get reported over time. I noticed this from personal experience too: when writing unit tests, I would see where I cut some corners in the tests, and anticipated the mutant. Now I just add the missing test cases, rather than facing a mutant in my Code review, and I rarely see mutants these days, as I’ve learned to anticipate and preempt them." Now this one is expected, if you build automated tooling that comments on your code review you expect people to try and avoid that. However there's still the question of what's the quality improvement and what's the productivity impact.