How to Build a Codebase-Aware AI PR Reviewer

Set up a codebase-aware AI PR reviewer that catches team-specific review mistakes before humans do.

This guide is for tech leads and senior developers who are drowning in clean-looking pull requests and need a practical way to move team memory into the review flow. By the end, you’ll have a repeatable setup for a codebase-aware AI reviewer that checks your project rules, reads the right files, and surfaces the mistakes your team keeps seeing.

The approach works whether you use Claude Code, Cursor, Cline, GitHub Copilot, or a mix. The key outcome is not a shinier model, but a review system that can actually see your architecture, conventions, and migration rules before a human gets pulled in.

Before you start

• A GitHub account with access to the repository you want to review.
• One AI coding tool account, such as Claude Code, Cursor, Cline, or GitHub Copilot.
• Node 20+ if you plan to script review commands locally.
• Git 2.40+ installed on your machine.
• A repo with at least one existing convention doc, ADR, or architecture note.
• Permission to add repo-level documentation files like AGENTS.md or CLAUDE.md.

Step 1: Map the review misses

Your first goal is to capture the recurring mistakes that humans keep catching late, because those are the rules your reviewer must learn first. Look for patterns such as old middleware paths, duplicate components, layer violations, or literal strings where enums should be used.

Write each miss as a short rule in plain language, then group them by area: auth, UI, backend layering, naming, or migration behavior. This becomes the source material for your reviewer instructions.

Verification: you should have a short list of five to ten review rules that describe real mistakes from your own codebase, not generic style advice.

Step 2: Add repo-level memory files

Your goal here is to move team knowledge into files the agent can read before it reviews code. Start with Claude Code docs and the Claude Code GitHub repo if you use Claude, then create a root-level AGENTS.md or CLAUDE.md that explains the rules in concise bullets.

# AGENTS.md

- New API endpoints must use v2 auth middleware.
- Do not duplicate shared hooks from /hooks.
- Controllers must not import repo functions directly.
- Check /design-system before creating a new UI component.
- Use enums instead of string literals for status checks.

Keep the language specific and testable. If a rule cannot produce a clear yes-or-no review comment, rewrite it until it can.

Verification: you should be able to open the file and point to each rule as something the agent can check during review.

Step 3: Add service-level instruction files

Your goal is to make the reviewer aware of local exceptions and per-service conventions without forcing it to infer them from the whole repo. Create service-specific files beside the code they govern, such as docs in a backend service folder or component notes in a UI package.

For example, add a short file in a service directory that says which auth path is canonical, which layer owns orchestration, or which shared component directory must be checked first. This is where migration rules and architecture boundaries belong.

Verification: you should be able to open one service folder and find a local instruction file that explains the rules unique to that area.

Step 4: Build the review command

Your goal is to give the AI one repeatable command that performs a read-only, codebase-aware review. The command should load the repo rules, inspect the diff, and ask for findings only where the rules are violated or where the change conflicts with existing patterns.

If you use a local script, keep it simple: pass the diff, include the relevant memory files, and ask for a structured output with file, line, issue, and rationale. Avoid letting the model rewrite code during review.

node scripts/review-pr.js --base origin/main --head HEAD

Verification: you should get a review output that names concrete files and points to specific rule violations instead of giving generic praise or vague suggestions.

Step 5: Run the reviewer on a real pull request

Your goal is to test the setup against a real PR that touches a known sensitive area, such as auth, shared UI, or backend layering. Use a recent change that a human reviewer already understands well, so you can compare the AI output with the actual team rule.

Check whether the reviewer catches the same issues a senior engineer would catch from memory. If it misses something important, add that missing rule to AGENTS.md or the relevant service file and rerun the review.

Verification: you should see at least one meaningful, team-specific comment that a generic reviewer would likely miss.

Step 6: Tighten the loop with human feedback

Your goal is to make the reviewer improve every time a human edits its output. After each real PR, record false positives, missed issues, and any rule that was too vague to help.

Then update the memory files so the next review is better. The compounding effect is the real win: each human correction becomes a permanent part of the reviewer’s context.

Verification: you should notice fewer repeated review comments and more first-pass catches on the same class of mistakes.

Common mistakes

• Writing rules that are too broad. Fix: rewrite them as testable statements, such as “controllers must not import repo functions directly.”
• Hiding important guidance in chat threads. Fix: move it into AGENTS.md, CLAUDE.md, or a service-level instruction file that lives in the repo.
• Letting the reviewer edit code during review. Fix: keep the review command read-only so it only reports findings and does not drift into implementation.

What's next

Once the reviewer is working on one repository, extend the same pattern to other services, then add deeper follow-ups like PR templates, architecture decision records, and automated checks for the rules that never should have been tribal knowledge in the first place.