3 ms·

I found Luna and even 5.4-mini to be quite good at code review provided a few things: 1. Run it in multiple cycles, only on the diff, and only emit a few findi

I found Luna and even 5.4-mini to be quite good at code review provided a few things: 1. Run it in multiple cycles, only on the diff, and only emit a few findings at a time. 2. Give it a memory so each cycle, it knows the previous finding to check if it's been fixed. 3. Give it access to canonical docs that encode your human reviewer heuristics. I exposed these as tool calls so they could be tracked via telemetry. 4. Run multiple reviewers, each with a tight focus. Security, performance, structural, database, etc. Each a separate prompt and persona. Additionally, we had file activation filters so the FE React reviewer didn't activate on BE only changes. Luna and 5.4-mini with no reasoning were exceptionally fast and almost always found issues with code produced by Opus and Fable. Default prompts for the curious (these are templates deployed by default, but customizable). Performance: https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Zeeq.Core.Models/CodeReviews/CodeReviewerAgentTemplateLibrary.PerformanceEngineer.cs https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Ze... Structural: https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Zeeq.Core.Models/CodeReviews/CodeReviewerAgentTemplateLibrary.StructuralReviewer.cs https://github.com/zeeq-ai/zeeq-app/blob/main/src/backend/Ze... (Keep in mind each agent also has tools to access and reference external docs.)

therealdrag0 12d ago
Only on the diff? Why? I found AI reviews garbage until they stopped being only on the diff and were actually able to query real context.
- CharlieDigital 11d ago
  Because your local agent can already see the full codebase; the code review only needs to see what's changing and evaluate the change.
  - therealdrag0 11d ago
    That’s strange. So your policy is “trust me bro my local agent knows best”?
    - threecheese 11d ago
      This tactic has somewhat broad acceptance in the AI code review sphere, but looks to me like it might trade off accuracy for context reduction/speed.
      - therealdrag0 11d ago
        Interesting. For example, GitHub’s built in Copilot review for example will checkout the whole repo and do a broad review. I found it to be WAY better than our prior CI reviews that just looked at diff and had TONs of false positives. (We have also built our own review tools like Copilot that check out repo and they are similarly league ahead IMO than diff reviewers.) I found diff reviewers so heavy in false positives, all my coworkers just ignored them.
        CharlieDigital 10d ago
        This is only true if you only run the code reviewer in the final PR. The way we set it up was that the code review responded to two signals: 1. GH PR webhook 2. The exact same review agents running as an MCP tool that the local agent can invoke before pushing. Practically, what this means is that it's OK to have a false positive because the local agent can make the check with the full context. This would be the same as if the entire team used Codex and, for example, had a sub-agent configured to run code reviews using a smaller model. In this case, the benefit to this tool-based approach is that the exact same agent works for all harnesses across a team and also works in the PR itself.
        therealdrag0 10d ago
        Thanks for sharing.

therealdrag0 12d ago

Only on the diff? Why? I found AI reviews garbage until they stopped being only on the diff and were actually able to query real context.

CharlieDigital 11d ago
Because your local agent can already see the full codebase; the code review only needs to see what's changing and evaluate the change.
- therealdrag0 11d ago
  That’s strange. So your policy is “trust me bro my local agent knows best”?
  - threecheese 11d ago
    This tactic has somewhat broad acceptance in the AI code review sphere, but looks to me like it might trade off accuracy for context reduction/speed.
    - therealdrag0 11d ago
      Interesting. For example, GitHub’s built in Copilot review for example will checkout the whole repo and do a broad review. I found it to be WAY better than our prior CI reviews that just looked at diff and had TONs of false positives. (We have also built our own review tools like Copilot that check out repo and they are similarly league ahead IMO than diff reviewers.) I found diff reviewers so heavy in false positives, all my coworkers just ignored them.
      - CharlieDigital 10d ago
        This is only true if you only run the code reviewer in the final PR. The way we set it up was that the code review responded to two signals: 1. GH PR webhook 2. The exact same review agents running as an MCP tool that the local agent can invoke before pushing. Practically, what this means is that it's OK to have a false positive because the local agent can make the check with the full context. This would be the same as if the entire team used Codex and, for example, had a sub-agent configured to run code reviews using a smaller model. In this case, the benefit to this tool-based approach is that the exact same agent works for all harnesses across a team and also works in the PR itself.
        therealdrag0 10d ago
        Thanks for sharing.