Do Two AI Reviewers Find the Same Bugs? 49 Findings

Two AI reviewers almost never find the same bug

"There is an AI code review bubble" hit 351 points and 249 comments on Hacker News in January 2026, and the argument inside it is whether a second AI reviewer adds signal or just more to read. I counted mine. Across five changes reviewed by two to four AI reviewers each, 49 findings, and not a single one was reported by two of them. That count comes from the notes I wrote during the reviews rather than from surviving reviewer output, which is a real weakness and I go into it below.

TL;DR: 49 findings across five changes, zero found by two reviewers, and 36 of them came from a reviewer that was not the first one to look. What predicts overlap is not which model you pick. It is whether the two reviewers read the same version of the code, and whether they can see the same things. Differ in either and they tend to find different defects.

Three questions people actually type, and where each one is answered here. Is a second AI reviewer worth it? Yes, but only if it differs in a specific way. Are the models not too similar to be independent? Four of them read the identical diff and reported four different things. Should reviewers have to agree before I merge? No, and that design fails hardest where you need it.

The count, change by change

Say the awkward part first. Not one of these five rows is auditable by a reader: four live in private repositories and the fifth is an unmerged branch. Exactly one run in this post sits in a public repository, and it is not in this table. I lean on that one hardest for exactly that reason. What I can do here is give the pull request, the date and the commit headlines, so the account is at least specific enough to be wrong in a checkable way.

Forty-nine findings. Nothing appeared on two lists.

The merge dates and titles above I re-read out of the repositories while writing this post. The per-reviewer finding counts did not survive that treatment. My reviewers write to temporary directories that are long gone, so those counts come from notes I wrote at the time, and the original finding lists no longer exist at all. That second part matters more than the first: without the lists, "none was reported by two reviewers" is a claim I can no longer re-derive, only remember. Read the 49 as a careful contemporaneous record, not as a measurement you or I could repeat.

The reviewers are four kinds of thing, and the difference between them turns out to matter more than the model names. grok-build from xAI gets file paths and can read and run the repository. fugu from Sakana AI and deepseek get a text packet and nothing else, so they reason about what they are shown. Claude Code's own /code-review runs inside the repository, on the machine, and can execute the test suite. One row below also lists codex, which is a packet reader like the other two and was pointed specifically at the tests rather than at the code. I drive the non-Claude ones through a single wrapper so I cannot quietly run only the cheap one.

I will call that difference the reviewer's aperture: what it is able to look at. A packet reader has a narrow one and cannot open a file it was not handed. A reviewer that can read and run the repository has a wide one. Aperture turns out to matter more than the model behind it, which is the whole argument of this post.

The one published benchmark, and the number it does not have

There is a real, quantified reviewer benchmark, and I want to put it up front because my whole argument would look sloppier without it.

CodeRabbit published Opus 5 for code review on 24 July 2026 (read 30 August 2026). They ran each configuration three times against roughly 100 common error patterns "drawn from verified issues in real open-source pull requests" and compared averages against their production model mix. Opus 5 at x-high effort caught 55.2% of the known issues against the baseline's 61.1%, with cleaner actionable comments (39.3% precision versus 35.2%) and about four times the nitpicks, 92 against 23.

Those are recall and precision figures for each configuration against a shared ground truth. They are not overlap. Two reviewers at 55% and 61% could have the 55% sitting entirely inside the 61%, in which case the second one adds nothing at all, or they could barely intersect. The benchmark cannot distinguish those two worlds, and neither can any other published reviewer comparison I could find.

CodeRabbit themselves reach for the ensemble anyway. Their verdict is that Opus 5 "may become a useful precision-oriented lane inside a routed ensemble" and that teams should "pair Opus 5 with a recall-oriented reviewer." On the same chart, GPT-5.6 Sol catches 69.7% of known issues while only 31.6% of its comments are worth keeping. So the recommendation to pair reviewers is already out there, published by people running this at scale, and the quantity that would justify it is still missing.

That is the gap my numbers sit in. Mine are n=1 developer and nowhere near a benchmark, but they are the intersection, and the intersection is what decides whether a second reviewer is worth its cost.

Four reviewers, one diff, four different findings

The cleanest of the five is PR #597, because all four reviewers read the same version of the same change.

The code fix was already correct after round 1 and all four agreed on that. Every round-2 finding was about the test rather than the code, and each reviewer attacked a different property of it:

grok-build   enumerated all 21 filesystem writes in the file and confirmed
             no second uncovered write site existed
fugu         the mutation evidence was ONE-SIDED - reverting the write
             tripped leg 2 before leg 3 was ever evaluated, so leg 3 had
             never been shown capable of failing
deepseek     the test always set the env var, so only the redirected
             branch was exercised; the branch every real session takes
             was uncovered
/code-review ran the suite and hit a live failure: the assertion
             fingerprinted a globally shared file any concurrent session
             writes, so the test was reading the blast radius instead of
             preventing it

Three of those four turned into commits whose headlines name the finding: "prove the second direction of the new test", "cover the unset branch, which the redirected test cannot", "stop fingerprinting a globally-shared file in the assertion". I read those out of the repository for this post rather than off my notes, which is the closest I can get to showing my work on a private repo.

Same diff, same day, four reviewers, four disjoint findings. If reviewer similarity were the binding constraint, this is the run where it should have shown up.

That matters because the sharpest objection in the Hacker News thread is exactly that. One commenter: "Independence is ridiculous - the underlying llm models are too similar on their training days and methodologies to be anything like independent. Trying different models may somewhat reduce the dependency, but all have read stack overflow, Reddit, and GitHub in their training."

The reasoning is sound and the prediction did not hold here. Being trained on the same corpus does not make two reviewers report the same defect, because reporting is not recall. What each one reports is shaped by what it was pointed at, what it can execute, and what angle it was asked to attack from. Similar priors, different apertures.

I should be precise about what I have and have not shown, because there is an easy overstatement here that I made in my own first draft. Zero overlap in what the reviewers reported is not proof of statistical independence, and it is not proof that a second reviewer catches things a first one could not have. It is also not 49 separate wins: if the first reviewer found 6 and the second found 4 with no overlap, the second reviewer bought me 4 findings, not 10. The added yield is the later reviewers' sets: the 49 minus whatever the first reviewer on each change found, which is 49 minus 13, so 36. That is the number that pays for the second reviewer, and it is the one I would defend.

The run where overlap was total

Then there is the run that breaks the pattern, and it is the most informative one in the set.

Kairn PR #30 (merged 25 Aug 2026) changed how a ranking function orders results. This repository is public, so this is the one run you can open and check line by line. Three external lenses attacked it and found four defects. I fixed them. Then Claude Code's /code-review ran, and it independently reproduced all four, said so, and did not re-list them. Overlap was 4 of 4, complete.

It then added four more that no external lens had raised, including the serious one. That is not my summary of a transcript, it is what the commit body says: "The internal review ran against the same commit the three external lenses had already worked over. It independently reproduced all four of their findings and then added four they had all missed."

Two things happened in that same run that complicate the tidy version, and I only went back and read them because a reviewer of this article told me to check my own source. Both make the picture better rather than worse.

The two packet-only reviewers overlapped with each other. The first fix commit says it plainly: "Two lenses independently reduced to the same first finding, and one added a fourth." So in the run I am holding up as the overlap case, there was also external-to-external overlap, between exactly the two reviewers that share an aperture. My draft did not mention it.

And the two packet-only reviewers were then refuted by the one that can read the tree. A separate commit: "A third external lens, the only one able to read the tree, refuted a claim the two content-only lenses had let stand." The defect was that a Python slice and a SQL LIMIT disagree on negative values. That is a fifth external finding, and an earlier draft of this post wrongly filed it under the internal reviewer's four.

The highest-severity finding was invisible to every external reviewer for a structural reason. The defective function had shipped in a different pull request, #26, which merged two hours and twenty minutes before this one. The change under review only made it load-bearing, by sorting on its output for the first time. A reviewer scoped to the diff sees the sort. A reviewer scoped to the repository sees the function the sort now depends on.

Two hours, not two weeks, and that is the sharper version. Age is not what hid it. Diff scope is. Anything that shipped in a previous pull request is outside this pull request's diff, whether it landed last month or before lunch, and a reviewer that only reads the diff cannot reach it. (My own notes said "weeks earlier" and my own review round caught that, which is the third number in this post that changed when I opened the source.)

So overlap was not random, and this one run contains three of the four pairings I have data for:

Three of those four rows come out of one public pull request, which is why I keep pointing at it.

The pattern I take from this is about aperture and version rather than model choice. Reviewers that read the same version through the same aperture tend to converge. Reviewers that differ in either tend to partition. Convergence is worth something, but it is confirmation rather than new information, and it costs the same as the run that would have produced new information.

I want to be careful with the word "tend", because one of my own runs does not fit. On PR #597 all four read the same version, and fugu and deepseek share an aperture exactly, and they still reported different things. So a shared aperture does not force agreement, it only makes agreement more likely. Something else is moving as well, most likely which angle each reviewer was pointed at, and I have not isolated it.

What this does to consensus gating

The same Hacker News thread contains a design that this measurement argues against, and it is a design a lot of people are reaching for. One commenter's risk-triage ladder puts it plainly: for high-risk changes, "Three models have to reach consensus before a human even looks at it" (posted 6 February 2026, read 30 August 2026).

Run that against my numbers. If three reviewers produce 15 findings and none of them is reported by two, a consensus gate passes zero findings to the human. The gate does not filter noise, it filters everything, and the changes it filters hardest are the high-risk ones you built it for. Requiring agreement is only safe when agreement is common, and agreement was common in exactly one of my six runs: the one where the reviewers read the same version with the same access.

Consensus is still worth having, just not as a gate. When two reviewers with genuinely different apertures land on the same defect, that is the strongest signal available, because it is the one thing neither of them could have produced alone. On one change in a different repository, grok-build reading the files and Claude Code reading the rendered behaviour independently found the same broken canvas dimension. I fixed that one first and without arguing about it.

The rule I actually use: convergence raises priority, divergence raises coverage, and neither is allowed to veto the other. A reviewer is an instrument, not an authority, and when two instruments disagree I run the experiment the disagreement implies rather than weighing the two claims. I refuted a lens exactly that way while writing the review-rounds post: a content-only reviewer ruled on the contents of a web page it had no way to open, and the fix was to open the page, not to average the opinions.

Sequence the reviewers, do not fan them out

This is the actionable half, and it costs nothing to adopt.

Point the second reviewer at the corrected version, not the original. Both times I measured it deliberately, the second reviewer got code that had already absorbed the first reviewer's fixes, and both times it returned a fresh set instead of duplicates. On PR #412 that ordering is what surfaced a partial fail-open boundary, a defect that only became visible once the obvious ones were out of the way.

Fanning the same diff to N models in parallel buys overlap. Sequencing them buys depth. Same token cost, same wall-clock if you were going to fix the first round's findings anyway.

Order matters in one more way, and I learned it the expensive way this week. Run the external reviewers first, apply their fixes, and run the repository-aware reviewer last. On the review-rounds article, three external lenses correctly refuted one sentence. My fix made that sentence stronger and false in a new way. Claude Code's /code-review then caught it, and it caught it because the lens round is what had created it. A reviewer that proposed a fix is the worst possible reader of that fix, and the reviewer reading the artifact as it stands now is the only one positioned to see what the last round broke.

Reversed, that defect ships. This is the same trap I keep hitting one level up: the instrument carries the defect it is looking for.

Count surfaces, not reviewers

Here is where I have to narrow something I published myself.

I have a post titled Claude Code code review needs a non-Claude model, and its rule reads "both models, always, never one." That is still right for the reason it was written: a Claude reviewer grading a Claude-authored design is the same distribution twice. But the count in it is loose, and the loose part is the thing this article is about.

fugu and deepseek both get a text packet and neither has a filesystem. Running both is two reviewers and one surface. They share a blind spot exactly, and their agreement about code neither of them can open is not corroboration, it is two reasoners guessing in the same direction. An article arguing for a second reviewer that miscounts its own reviewers argues against itself, so: count surfaces.

The narrower rule I run now:

question                          the right set
what the code DOES at runtime     the reviewer with a filesystem, plus one packet reader
a claim inside the packet         the packet readers - the packet IS the evidence
completeness over a large file    the reviewer with a filesystem, ALONE

That third row is the one that surprised me. For a completeness question over a 1,900-line document, adding packet-only reviewers does not add a surface. It adds two confident opinions about a file they cannot open, and their agreement reads as corroboration when it is not. On one run, a packet-only reviewer returned zero findings and called a file consistent while the file-reading reviewer found eight things wrong with it.

So one reviewer is sometimes the correct instrument rather than a degraded three, and the test is whether the evidence fits in a packet. That does not license skipping reviewers because building the packet is annoying. It is a property of the question, not of your effort. And when only one answers, say so: "checked by one reviewer" is honest, "cross-checked" is not.

The other side of that ledger is worth stating too, because I once had it backwards. On a two-file documentation change I ran six internal review rounds. Each one found real defects. Not one of them asked whether the central claim was true, because the internal reviewer and I share the frame. One external reviewer refuted it in a single pass, and a sandbox test confirmed the external reviewer: the guard I had shipped could be walked straight through. Neither direction substitutes for the other.

What I do now

1. Ask what each reviewer can see before adding one. A second reviewer with the same access to the same version is a confirmation, not a check. If it cannot run the code and your question is about runtime, it is guessing.
2. Run them in sequence, each on the previous one's corrected version. Same cost as parallel, and it buys defects that only become visible once the obvious ones are gone.
3. Put the repository-aware reviewer last. It is the only one reading the artifact as it now stands, which is where the previous round's fix lives.
4. Never use agreement as a gate. With near-zero overlap a consensus requirement passes nothing. Use convergence to rank what to fix first, not to decide what counts.
5. Report the honest count. Two packet readers are one surface. If one reviewer answered, say one.

If you want the surrounding workflow rather than the reviewer question specifically, six Claude Code failure modes covers what goes wrong before review, how I orchestrate parallel sessions covers running several of these at once, running several Claude Code sessions side by side covers the mechanics of that, and why done is not done covers what has to be true before you believe a green result. If your reviewer suddenly feels worse than it did last month, that question has its own post. The lens-per-frame idea that this grew out of is in multi-agent analysis with cognitive lenses, and the agent composition underneath it is in trait-based agent composition. There is a walkthrough of the whole setup in the guide.

Honest scope

n=1 developer, six changes across three repositories, my own reviewer setup and my own prompts. The changes are not independent of each other, because I wrote every one of them and I wrote the fixes the later reviewers then read.

What "zero overlap" means here is narrow: no finding appeared on two reviewers' lists in the same round. It does not mean the reviewers could not have found each other's defects under different prompting, and it is not a measurement of recall against a known bug set. CodeRabbit's numbers are that; mine are not, and the two answer different questions.

The strongest caveat is about provenance. What I re-read from the repositories for this post is the pull request numbers, merge dates, titles and commit sequences, and those all check out. The per-reviewer finding counts come from notes I wrote during the reviews, because the reviewers write to temporary directories that no longer exist. So the count is as good as my note-taking, and my note-taking has been wrong before: the PR #95 date in the callout above is one example, and the previous post in this pair had to publish a per-round sequence instead of a total because the stored total did not reconcile.

The counting has other soft edges I would rather name than hide. On PR #475 the internal review returned 9 findings of which 7 were confirmed, and I counted 7. On PR #412 fugu returned an empty stream after zero seconds, which is a failure and not a clean bill of health, so I excluded it rather than scoring it as "found nothing". The permission-gate branch is not merged; I have included it because its 15-finding split is the largest single-change sample I have, and I excluded nothing else on those grounds. Two smaller illustrations, the 1,900-line document and the broken canvas dimension, come from my notes with no independent check in this post, and neither carries a number the argument rests on.

The aperture-and-version explanation comes from one public run and five private ones, and PR #597 already does not fit it. That is a hypothesis with a mechanism, not an established result. The clean test would be running the same reviewer twice, once on the original and once on the corrected version, and comparing the two lists directly. I have not run it.

There is also a circularity I should name rather than let a reader find. Two of the five rows are sequential by construction: the second reviewer was deliberately pointed at the already-corrected code, which is the practice this post recommends. Their partition is therefore partly something my method produced, not only something I observed. PR #597 is the row that escapes this, because all four reviewers read the same version, and it is the one I would build on if I ran this properly.

There are Reddit threads on this question and none of them is cited here. Reddit blocks the automated readers I have, which I confirmed on several routes earlier the same day while working on the companion post, so rather than cite titles I could not open I left them out entirely. This post quotes only Hacker News and vendor material, all of which I read.

This post got the same treatment it describes: external reviewers first, then fixes, then the repository-aware review last. That round is why several paragraphs above read the way they do. The reviewers caught that I was calling a notes-derived number a measurement, that "49 times out of 49" does not follow from disjoint lists, that all five table rows are unauditable rather than four, and that my draft had filed an external reviewer's finding under the internal one in the Kairn run. The last of those was a plain factual error and it is corrected above.

Two of their findings I refuted rather than accepted, and both took a source check. One argued the "reproduced all four" claim is unsupported by the public pull request; the commit body states it in those words, and the reviewer had read two other commits in that pull request but not that one. The other said the consensus quote could not be found on Hacker News; it is comment 46915031 on that thread, and it is linked above.

Then the repository-aware review ran on the corrected draft, and it did the thing this post argues it does. Five findings, none of them raised by any of the three reviewers before it. The one that mattered: "shipped weeks earlier" was two hours and twenty minutes, which it settled by running git log -S against the public repository rather than by reasoning about my sentence. It also found that this post's hero image does not exist while all 49 other posts have theirs, that my reviewer paragraph says "three external" while the table lists four, and that the reading time I had just committed as "the site's own rate" implied 203 words a minute against a corpus median of 186. That last one is the small comedy of the thing: I fixed a number, wrote a commit message claiming I had matched a measured rate, and had not measured it.