AI Review Rules Need a Data-Handling Test

A reviewer receives a dense manuscript on a Friday evening. The methods are long, the statistical appendix is awkward, and the editor has asked for a report in two weeks. The reviewer has a university-approved AI assistant that can summarize text, check code comments, and flag contradictions. The journal instructions say, in one paragraph, that peer review is confidential and that reviewers are responsible for their own reports. They say little about the tool itself.

That gap is becoming harder to defend. On June 17, 2026, the American Physical Society announced updated AI guidance for APS journals, saying the policy allows authors and reviewers to use AI tools while setting boundaries for human oversight, accountability, and confidentiality: https://www.aps.org/about/news/2026/06/releases-updated-ai-policy-journals. The APS policy page says it governs AI use by authors during manuscript preparation and by reviewers during peer review, and that sharing peer review materials with unrestricted AI tools is not permitted: https://journals.aps.org/authors/ai-based-writing-tools.

The European Commission is pulling on the same thread from another direction. Its updated ERA Living Guidelines on the responsible use of generative AI in research, published May 8, 2026, keep the focus on accountability, transparency, responsibility, third-party interactions, and hidden prompts: https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/updated-era-living-guidelines-responsible-use-generative-ai-research-2026-05-08_en. In the full guidance, researchers remain responsible for outputs produced with AI support, and organizations are urged to consider quality, transparency, integrity, data protection, confidentiality, intellectual property, and environmental impact when sensitive or protected information is shared with AI tools: https://research-and-innovation.ec.europa.eu/document/download/2b6cf7e5-36ac-41cb-aab5-0d32050143dc_en?filename=ec_rtd_ai-guidelines.pdf.

The operational lesson for journals is not that APS has solved the question or that European guidance should be pasted into every reviewer form. It is that "AI allowed" and "AI banned" are becoming too blunt for real peer review work. The more useful question is what information leaves the journal workflow, where it goes, what the tool may do with it, and who can prove the answer later.

Two Policies, One Practical Problem

Journal leaders are hearing two reasonable arguments at once. The first says peer review is already under strain, and reviewers will use AI quietly if journals give them only blanket prohibitions. A June 30 Scholarly Kitchen essay by Elena Vicario, Director of Research Integrity at Frontiers, argued that hidden and inconsistent AI use in peer review is a governance risk, not a sustainable compromise: https://scholarlykitchen.sspnet.org/2026/06/30/guest-post-now-is-the-time-for-ai-in-peer-review-and-publishing-policies-need-to-recognize-this/.

The second argument says unpublished manuscripts, reviewer identities, confidential editorial correspondence, patient information, proprietary methods, and embargoed findings cannot be treated as ordinary text pasted into a public tool. That concern is not conservative nostalgia. It is a basic data-handling problem.

A policy that answers only "may a reviewer use AI?" skips the harder question. A reviewer might use AI to correct grammar in a report, summarize a manuscript section, generate a draft critique, check a statistical claim, identify possible image manipulation, or search for related work. Those are not the same action. They carry different confidentiality, accuracy, attribution, and accountability risks.

The Missing Test: What Leaves The Journal?

Every reviewer AI rule should start with a data movement test. Does the reviewer upload the manuscript, figures, supplementary files, raw data, code, review report, decision letter, author response, or editorial notes? Does the tool retain prompts or outputs? Are inputs used for model training? Is the tool governed by the reviewer institution, the publisher, a vendor contract, or nobody the journal can audit?

This sounds technical, but editors can understand it without becoming security specialists. If the manuscript or review material is copied into a tool outside a governed environment, the journal has lost control over confidential peer-review material. If the tool is contractually restricted, access-controlled, logging usage, and not training on the content, the risk profile is different.

That distinction should appear in reviewer instructions. "Do not use AI" is simple but often unenforced. "Use AI responsibly" is flexible but vague. "Do not upload confidential manuscript or peer-review material to unrestricted tools; disclose permitted tool-assisted tasks; remain accountable for the report" gives reviewers a workable boundary.

A Four-Lane Model For Reviewer AI

Lane 1: Personal drafting help that never sees the manuscript

Some AI use never touches confidential article content. A reviewer may ask for help turning rough notes into clearer prose, checking tone, or shortening a paragraph after the substantive assessment has already been written. The risk is lower if no manuscript text, author data, figures, tables, unpublished results, or editorial correspondence is supplied to the tool. The reviewer still owns the report and should check that the final language has not softened, exaggerated, or invented criticism.

Lane 2: Tool use inside a governed environment

A journal, publisher, society, or university may provide an AI environment with contractual controls. That can support summarization, consistency checks, reference triage, statistical prompts, code explanation, or reviewer workflow assistance without sending material into consumer systems. The governance question becomes evidence: what tool was used, what data was processed, who had access, what was logged, and what assurances exist about retention and training.

Lane 3: Substantive assessment support

When AI is used to evaluate novelty, significance, methods, image integrity, statistics, or literature coverage, the journal has moved from writing support to assessment support. That may be useful, but it needs a higher standard. The reviewer should not outsource judgment, and the editor should know when an AI-generated prompt, score, summary, or critique influenced the report. A quiet AI-generated methods critique that the reviewer does not understand is not peer review; it is an unexamined intermediary.

Lane 4: Prohibited or exceptional use

Some uses should remain prohibited unless the journal has explicit infrastructure and consent. Uploading a confidential manuscript to an unrestricted public tool is the obvious case. So is using AI to produce a review with no expert reading, sending identifiable patient or participant material outside approved controls, or using hidden prompts to manipulate an automated editorial process. These are policy boundaries, not mere preferences.

Hidden Prompts Make This More Than A Reviewer Problem

The updated ERA guidance is notable because it calls attention to hidden prompts: instructions embedded in content in ways humans may not notice but AI systems may process. For journals, that turns the manuscript itself into a possible adversarial input. An author could hide text that attempts to steer a tool toward a favorable assessment. A reviewer could paste a manuscript into a tool and receive output shaped by instructions the reviewer never saw.

This is not a reason to panic. It is a reason to stop treating peer-review AI rules as a paragraph in the ethics policy. If journals plan to use AI-supported triage, reviewer assistance, or integrity checks, they need safeguards for invisible text, metadata, white-on-white instructions, suspicious formatting, and document conversions that preserve content a human editor would not read.

The risk is also cultural. Once reviewers believe that journals run manuscripts through AI checks before review, authors may begin writing for the tool as well as for the editor. A journal that cannot explain which AI checks exist, what they are allowed to decide, and how human oversight works will invite suspicion from both sides.

What To Change In Reviewer Instructions

• Separate confidential-content rules from general AI-use rules so reviewers know which materials may never leave approved systems.
• Define permitted low-risk uses, such as language polishing of reviewer-written text that contains no manuscript content.
• Name any journal-approved or institution-approved AI environments, and state whether reviewers may use alternatives.
• Require disclosure when AI materially supports summarization, technical checking, literature comparison, or drafting of review text.
• Tell reviewers that human accountability is not transferable: they must be able to defend every substantive claim in the report.
• Create an editor-facing escalation path for suspected hidden prompts, tool misuse, or AI-generated reviews that lack expert engagement.

These instructions should live where reviewers actually work: invitation emails, reviewer agreement pages, manuscript-system prompts, and report forms. A PDF policy page is still useful, but it is not enough. The decision point happens when a reviewer is tired, the deadline is close, and the tool is one browser tab away.

Practical Takeaway For Journal Leaders

The practical takeaway is to replace yes-or-no AI rules with a data-handling test. Before approving any reviewer AI use, classify the task, the material, the tool, the environment, the disclosure requirement, and the audit record. If the journal cannot answer those six points, it is not ready to rely on the tool in peer review.

Reviewer AI use is moving from an ethics debate to an operations problem. APS has shown one path by permitting use with boundaries. The ERA guidance shows why confidentiality, hidden prompts, and accountability need to be built into those boundaries from the start. Journal leaders do not need perfect sector consensus before they act. They need reviewer rules that are specific enough to follow, strict enough to protect unpublished work, and honest enough to reflect how scholarly review is already changing.