Claude Code Review Needs a Second Model

Claude Code reviews code two ways: a local /code-review command that reads your branch before you push, and a managed GitHub App that reviews pull requests with a fleet of agents and posts inline comments by severity. The command is included with Claude Code. The managed service is Team and Enterprise only, at 15 to 25 dollars per review.

Updated September 4, 2026 · by the AITerm team

Here's the tension nobody addresses head on. Code review is the task an agent pays off on fastest, and it is also the task developers trust an agent with least. Stack Overflow's 2025 survey has 84% of developers using or planning to use AI tools, up from 76%, while the share who actively distrust the accuracy of what those tools output climbed from 31% to 46%. Only 3% say they highly trust it. The single biggest complaint, from 66% of respondents, is output that is almost right but not quite.

An almost-right code review is not a small problem. It is a reviewer who approves.

So we did the boring thing and measured it on ourselves, on a real Swift codebase, over four months. Here is what came out.

What does Claude Code review actually catch?

It catches correctness bugs that live inside the diff and the code immediately around it: races, unhandled edge cases, silent failure paths, queries missing a scope. Anthropic reports that under 1% of the managed service's findings get marked incorrect by the engineers reading them. That number is real, and it is also not the number you want.

The number you want is how many of the correct findings mattered. So, our audit: 62 merged pull requests in the AITerm repo between May and August 2026, every one of them run through /code-review at default effort, findings logged whether or not we acted.

One shipping bug prevented every seven pull requests. That is a good trade at the price of a local command and it is a defensible trade at 20 dollars a review, depending on what your bugs cost you. It is nowhere near the "AI reviews your code" pitch, and the honest framing matters, because the gap between the pitch and this number is where teams lose trust in the tool entirely.

The managed service tags each finding at one of three severities. Important is a bug to fix before merging, Nit is minor, Pre-existing is a bug your pull request did not introduce but touched. Our nine live almost entirely in the first bucket, and the volume lives in the second.

The bug our reviewer waved through

Release 1.6.2. We changed four lines in the onboarding gate, the one that decides whether a user sees the welcome tour. We bumped WelcomeTour.version because the tour had a new step, and the gate compares that version against the stored welcomeTourSeen value.

The review approved it. Correctly, in a sense. The diff was internally consistent, the comparison was right, the version constant was right, nothing in those four lines was wrong.

Turns out the migration that normalizes welcomeTourSeen for users coming from an older build runs after the gate reads it, in a different file, which the diff did not touch. Every user updating from 1.6.1 got the full guided tour again on launch. We found out from a support email, not from CI, not from the reviewer.

Honestly, that one stung, because it is the exact failure class the research predicts. Writing for O'Reilly Radar, the argument is that roughly half of security defects are intent violations: things the code does not do that it was supposed to, or does that it was not supposed to. They are invisible without a specification to check against, and no tool finds them by reading code that is, structurally, perfectly sound.

The same piece points at the NIST SATE evaluations, where the best static analysis tools plateaued around 50 to 60% detection on security vulnerabilities. Agent review moves that ceiling. It does not remove it.

You're probably thinking a longer CLAUDE.md would have caught it. We thought so too. We added the ordering constraint to the onboarding section of ours, and the next review that touched the same file did flag it, at nit severity, buried under eleven other nits. Documenting the trap helps. It does not promote the trap to the top of the list.

Three products wear the same name

Look, this is where most of the confusion around "claude code review" comes from, and it is worth thirty seconds of your time before you set anything up. Three distinct things, three billing models, three levels of control.

The managed service is the one with the fleet: multiple agents analyze the diff in parallel, each looking for a different class of issue, then a verification step checks each candidate against actual code behavior before anything is posted. Findings are deduplicated, ranked, and attached to the lines they concern. Average completion is 20 minutes.

The open-source plugin behind the self-hosted path is less mysterious than the marketing suggests, and you can read it: four parallel agents, two auditing CLAUDE.md compliance, one hunting obvious bugs, one reading git blame for context. Each issue gets a confidence score from 0 to 100, and anything under 80 is dropped. That threshold is a number in a file you can change.

How do you set up Claude Code review?

Start local. It costs nothing extra, it takes one command, and it tells you within a day whether agent review is worth wiring into CI for your codebase. If you are still getting your bearings in a single session, the Claude Code fundamentals come first: review inherits your CLAUDE.md, so a vague memory file produces a vague reviewer.

The local command has three flags worth memorizing. --fix applies the findings to your working tree, --comment posts them as inline comments on the pull request from your own session, and --post handles the cloud variant. And yes, one gotcha: a background review's --fix edits land outside your session checkpoints, so /rewind will not undo them. Use git.

# before you push, cheap pass
/code-review low

# a specific range, or a PR number, or a branch
/code-review main...my-feature

# review, then post the findings on the PR from your account
/code-review --comment

# stop Claude and scheduled tasks from starting reviews on their own,
# while keeping the command available when you type it
# ~/.claude/settings.json
{ "skillOverrides": { "code-review": "user-invocable-only" } }

For the CI path, the workflow below is the one that matters. It installs the official review plugin and runs it on every pull request event worth reviewing. The claude_args line looks redundant next to the plugin's own frontmatter and it is not: the action only starts the MCP server that posts inline comments when --allowedTools names it.

name: Code Review
on:
  pull_request:
    types: [opened, synchronize, ready_for_review, reopened]
jobs:
  review:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: read
      issues: read
      id-token: write
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 1
      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          plugin_marketplaces: "https://github.com/anthropics/claude-code.git"
          plugins: "code-review@claude-code-plugins"
          prompt: "/code-review:code-review --comment ${{ github.repository }}/pull/${{ github.event.pull_request.number }}"
          claude_args: '--allowedTools "mcp__github_inline_comment__create_inline_comment"'

Never let a model review its own diff

Here's the thing. The default configuration everyone lands on has Claude write the code and Claude review the code. Same family, often the same model, sometimes the same session. That is the one pairing the research says to avoid.

Panickssery and co-authors showed at NeurIPS 2024 that LLM evaluators recognize their own generations and favor them, and that stronger self-recognition tracks with stronger self-preference. A 2025 follow-up asked whether that preference is at least deserved, and the answer was uncomfortable: much of it is legitimate, because stronger models genuinely do produce better output, but the harmful residue concentrates precisely where the evaluator was wrong as a generator. Which is the only place a code review matters.

So our setup is two panes. Claude Code writes in the left one. Codex reads the diff in the right one, cold, with no memory of the reasoning that produced it. Two agent sessions, roughly 1.2 GB resident on an M2 Air with 16 GB, keystroke latency under 8 ms in native panes while both stream.

# pane 1, the author
claude

# pane 2, the reviewer, different vendor, no shared context
codex exec "Review the diff of HEAD against origin/main. \
Report correctness bugs only, with file:line. \
Do not suggest style changes. State what you could not verify."

Across the last 18 pull requests we ran both ways, the two agents flagged the same issue about a quarter of the time. Two of the bugs Codex reported sat in diffs Claude's own review had approved. That is a small sample and we are not claiming a benchmark, but the direction has been consistent enough that we stopped treating a single-model review as a review.

The instruction "state what you could not verify" earns its place in that prompt, by the way. It is the line that turns a reviewer that always finds something into one that occasionally says the honest thing.

Should review run in CI or in your terminal?

Both, at different moments, and honestly, confusing them is how teams end up paying twice for the same finding. The terminal pass is for the diff you are about to push, where fixing costs seconds. The CI pass is for the diff someone else is about to read, where the value is a second opinion in front of a human.

In practice we run local review on every branch and CI review only on pull requests targeting main. The arbitrage is simple: local review costs your normal usage and catches the embarrassing things before anyone sees them, while a managed review at 15 to 25 dollars a shot has to justify itself against a human reviewer's half hour.

If you do want a gate, the check run gives you one, though you have to build it. The last line of the check output carries a machine-readable severity comment, and a non-zero count of Important findings is your signal:

# find the check run id for the commit
gh api repos/OWNER/REPO/commits/<sha>/check-runs \
  --jq '.check_runs[] | select(.name=="Claude Code Review") | .id'

# read the severity breakdown, for example {"normal":2,"nit":1,"pre_existing":0}
gh api repos/OWNER/REPO/check-runs/CHECK_RUN_ID \
  --jq '.output.text | split("bughunter-severity: ")[1] | split(" -->")[0] | fromjson'

One trigger detail bites teams that set this up before the summer. A bare @claude review used to subscribe the pull request to reviews on every subsequent push. Since a July 2026 change it does not, and you write @claude review always for that behavior. If your review volume quietly dropped and nobody changed a setting, that is why.

Is Claude Code review worth 20 dollars a pull request?

It depends entirely on what a bug costs you after it merges, and most teams have never put a number on that. Ours is a native macOS app, so a shipped bug means a release, a notarization round and users sitting on a broken build until they update. One prevented bug pays for a lot of reviews.

Run the same arithmetic in the other direction. Nine real bugs across 62 pull requests, at managed-service prices with a review on every push, would have been somewhere north of a thousand dollars for those nine. For us that is fine. For a repository with a dozen small pull requests a day and no release cost, it is not, and the honest recommendation there is local review plus a cheap CI pass on the branches that matter.

Three levers cut the bill without cutting the coverage that matters. Choose "once after PR creation" instead of "after every push", which multiplies cost by the number of times someone fixes a typo. Set a monthly spend cap in admin settings so a bad week cannot surprise you. And put skip rules in REVIEW.md for generated files, lockfiles and anything your CI already enforces, because a review that spends its verification budget on a lockfile found nothing you needed.

Review is how you end up running two agents

Fair question: why is a terminal company writing about code review at all? Because review is the first task where one agent session stops being enough, and almost nobody frames it that way.

Think about the shape of the work. The author agent is mid-task, holding a plan and a hundred thousand tokens of context it should not lose. The reviewer needs the opposite: no context, no attachment to the approach, a cold read of the diff. Those two jobs cannot share a session without one of them getting worse. So you open a second pane, and the moment you do, you are piloting a fleet rather than chatting with a tool.

That is the friction we build against. Two agent sessions are trivial to start and annoying to watch, because the reviewer finishes silently while you are reading the author's output, and a review nobody reads is worse than no review. AITerm shows a live state per agent session, so needs attention is a color in your peripheral vision rather than a tab you remember to check. Same idea as running isolated worktrees for parallel agents, applied to the smallest possible fleet: two.

The short version: if you are going to take one thing from this article into your setup tomorrow, make it the second pane, not the subscription tier.

Going further

Sources

1. Code Review, Claude Code documentation, 2026: research preview status and Team and Enterprise availability, the parallel agent fleet and verification step, the three severity levels, the neutral check run conclusion and the bughunter-severity JSON, the three review triggers, the July 2026 change to @claude review, fork pull request handling, CLAUDE.md and REVIEW.md customization, the 15 to 25 dollar average and 20 minute completion time, spend caps, and the local /code-review command with its effort levels and --fix, --comment and --post flags.
2. Claude Code GitHub Actions, Claude Code documentation, 2026: /install-github-app quick setup, interactive against automation mode, the code review workflow with plugin_marketplaces, plugins and the claude_args allowlist, the GitHub App permission set, secrets withheld from fork pull requests on public repositories, and the cost levers including --max-turns.
3. code-review plugin, anthropics/claude-code repository, 2026: the four parallel agents and what each one audits, the 0 to 100 confidence score, the threshold of 80, and the skip conditions for closed, draft, trivial or already-reviewed pull requests.
4. 2025 Developer Survey, AI section, Stack Overflow, 2025: 84% of developers using or planning to use AI tools against 76% the previous year, distrust of output accuracy rising from 31% to 46%, 3% reporting high trust, and 66% naming "almost right, but not quite" as their biggest frustration.
5. Trust in AI at an all time low, Stack Overflow press release, 2025: the same trust figures stated by the publisher.
6. AI Code Review Only Catches Half of Your Bugs, O'Reilly Radar, 2026: the argument that roughly half of security defects are intent violations invisible without a specification, and the NIST SATE finding that the best static analysis tools plateau around 50 to 60% detection on security vulnerabilities.
7. LLM Evaluators Recognize and Favor Their Own Generations, Panickssery, Bowman and Feng, NeurIPS 2024: frontier models distinguish their own output from other models' and humans' at non-trivial accuracy, and self-recognition capability correlates with the strength of self-preference.
8. Do LLM Evaluators Prefer Themselves for a Reason?, arXiv, 2025: much of the self-preference of stronger models is legitimate, while the harmful residue concentrates where the evaluator model itself errs as a generator.
9. Set up Code Review for Claude Code, Anthropic Help Center, 2026: setup flow for organization Owners, and the figure that fewer than 1% of findings are marked incorrect by the engineers reading them.