Fighting Fire with Fire: How AI Can Help Sustain the Future of Peer Review

Image

Quick Facts 

• Research forthcoming in Management Science argues that journals should replace hidden, ad hoc AI use with governed AI workflows that keep humans in control.
   

• Co-authors include Hemant K. Bhargava and Pantelis Loupos of UC Davis; Sarah H. Bana of Chapman University; Zhe Zhang of UC San Diego; Laura Brandimarte of the University of Arizona; Vidyanand Choudhary of UC Irvine; J. Frank Li of the University of British Columbia, and Daniel Zantedeschi of the University of South Florida.

  ---

Academic publishing has an AI problem. 

The same tools that help scholars draft, revise, replicate, and submit papers faster are pushing more manuscripts into a review system built around scarce expert time. Journals now face a basic operations bottleneck: more submissions, a largely fixed reviewer pool, and growing pressure on turnaround time and decision quality.

Our forthcoming paper in Management Science, “Fighting Fire with Fire: Infusing AI into Peer Review to Sustain Quality Scholarship,” argues that the answer is not to pretend AI can be kept out of peer review. It cannot. The better question is whether journals will govern AI use directly, or allow it to continue through fragmented and largely invisible private use by reviewers.

Our paper proposes a journal-controlled AI layer at the front of the review process. The goal is not to replace human reviewers. It is to redesign the workflow so expert reviewers spend less time on mechanical checks and more time on the judgments that matter most: contribution, novelty, rigor, and impact.

A Governed AI Workflow

The proposed workflow has two steps.

First, when a manuscript is submitted, a journal-specific AI reviewer generates a structured diagnostic. The AI can flag clarity problems, missing reporting elements, methodological gaps, internal inconsistencies, or other issues the journal has explicitly authorized it to assess. The AI does not make an accept-or-reject recommendation.

Second, authors receive the AI review and can provide a short response before the paper goes to human reviewers. Reviewers then evaluate three materials: the manuscript, the AI assessment, and the author response. The paper remains fixed at this stage; the response is a clarification document, not a revision.

That distinction matters. The AI review is an input into human judgment, not a substitute for it. It gives reviewers a common diagnostic starting point while preserving editorial authority and reviewer responsibility.

Why Governance Beats Shadow AI

We identify a second problem that is already here: shadow AI. Even when journals restrict AI use, reviewers may still use general-purpose tools privately to summarize manuscripts, generate critiques, or draft reports. That creates confidentiality risks, inconsistent prompts, unknown model behavior, and unmeasured dependence on the same kinds of AI-generated signals.

A journal-led system does not eliminate those risks, but it makes them visible. If the AI review is generated inside the journal workflow, editors can standardize the tool, tune it to the journal’s standards, log how it is used, and measure whether it changes reviewer behavior. In other words, AI becomes governed infrastructure rather than an unmanaged workaround.

The Tradeoff: Speed, Accuracy and Independent Judgment

Our paper models peer review as a constrained estimation problem. Manuscripts arrive, reviewers have limited time, and editors must estimate manuscript quality accurately enough to make sound decisions. As submissions rise, journals approach a capacity cliff: small increases in load can create large delays, thinner reviews, or both.

AI can help by making reviewer time more productive. But it also creates a risk. If all reviewers see the same AI-generated assessment and lean on it too heavily, their judgments may become more correlated. The review process may become faster, but less independent.

This is the central insight of the model: AI is not automatically good or bad for peer review. Its value depends on design. AI helps when the AI assessment is reliable enough, when it reduces time spent on routine verification, and when the workflow limits overreliance or anchoring. Poorly governed AI can instead create shared errors, false confidence, or homogenized evaluations.

One important result is counterintuitive. AI does not need to be better than a human reviewer to improve the system. Even a noisier AI signal can help if it allows journals to reallocate scarce human effort more effectively, for example, by reducing routine review burden and giving remaining reviewers more time for deeper evaluation.

What Journals Should Do Next

We do not present a one-size-fits-all solution. Different journals will need different AI scopes, prompts, safeguards, and audit systems. Our paper instead offers a design principle: move from hidden AI use to transparent, contestable, journal-specific AI support.
That means AI reviews should be structured, auditable, and limited to tasks the journal can validate. 

Authors should have a right to respond. Human reviewers should retain responsibility for final judgments. Journals should run pilot experiments to measure whether AI improves turnaround time, decision accuracy, reviewer effort, and the treatment of novel or risky work.

The broader point is institutional. Peer review is no longer only a scholarly norm; it is an information-processing system under resource pressure. AI is changing the volume of research entering that system. The question is whether academic journals will redesign the system carefully enough to preserve rigor at scale.

For the UC Davis Graduate School of Management, the study reflects a broader research focus on how analytical systems reshape organizational decision-making. In this case, the organization is academic publishing itself—and the decision is one of the most consequential in scholarship: what knowledge gets certified, improved and shared.