A Claude API Code Review Pipeline: Hook to CI

A three-layer Claude review pipeline: pre-commit catches local issues, GitHub Actions reviews PR diffs, a nightly job scans the repo. Full code and cost math.

Lint handles formatting, type checkers handle interfaces, tests handle behaviour. None of them notice "this write loses data under concurrency" or "this catch block swallows the exception". That class of problem used to require a human reader. It can now go to a model — provided you don't build a bot that only produces noise.

This pipeline has three layers: pre-commit catches what's visible locally, the PR stage reviews the full diff, and a nightly job sweeps code nobody has touched. All three share one review.py; only the diff source and severity threshold change.

Why not an off-the-shelf review bot

Ready-made PR bots have two recurring flaws: they stuff the whole diff into context, so long PRs get truncated, and they report everything, including "consider adding a comment" that nobody will ever act on. Two weeks in, the team learns to ignore it.

Building your own buys control over three things: only changed lines, only six categories of problem, and only high blocks. The price is about 150 lines of code — which is what follows.

Step 1: build a cheap diff

How you build the diff determines the cost. Three rules:

  • Use --unified=3 — enough context for the model, not enough to drag in whole files
  • Exclude lock files, build output and minified assets; they double the token count and carry no review value
  • Cap a single request at 60,000 characters; beyond that, split per file
Shell
# staged changes only, generated files excluded
git diff --cached --unified=3 --no-color -- . \
  ':(exclude)*.lock' ':(exclude)package-lock.json' \
  ':(exclude)dist/*' ':(exclude)*.min.js'

A real PR is usually 200–800 diff lines, around 3,000 tokens. That's light work for Claude Sonnet 5 — a few cents per run (the maths is below).

Step 2: force structure with tool use

The Anthropic Messages API has no response_format. The reliable way to get stable JSON is to define a tool whose only job is input_schema, then force it with tool_choice. The model has no other exit; it must fill the schema.

JSON
{
  "name": "report_findings",
  "description": "Submit code review findings",
  "input_schema": {
    "type": "object",
    "properties": {
      "findings": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "file":       {"type": "string"},
            "line":       {"type": "integer"},
            "severity":   {"type": "string", "enum": ["high", "medium", "low"]},
            "issue":      {"type": "string"},
            "suggestion": {"type": "string"}
          },
          "required": ["file", "line", "severity", "issue", "suggestion"]
        }
      }
    },
    "required": ["findings"]
  }
}

tool_choice: {"type": "tool", "name": "report_findings"} is the load-bearing line. With auto, the model can decline to call anything when it has nothing to say, and you get an empty content array to special-case. Forced, you always get either findings or findings: [].

The complete review.py

Python
#!/usr/bin/env python3
"""ai-review: review a git diff with the Claude Messages API."""
import argparse, os, subprocess, sys, time
import requests

BASE  = os.environ.get("ZAIMOD_BASE", "https://api.zaimod.com").rstrip("/")
MODEL = os.environ.get("REVIEW_MODEL", "claude-sonnet-5")
KEY   = os.environ["ZAIMOD_API_KEY"]
MAX_CHARS = 60_000
IGNORE = ("*.lock", "package-lock.json", "dist/*", "*.min.js", "*.svg")

RULES = """You are a senior code reviewer. Rules:
1. Comment only on lines added or modified in the diff, never on untouched legacy code.
2. Do not report style preferences (indentation, naming, missing comments, import order).
3. Report exactly six categories: correctness bugs, edge cases, concurrency and transactions,
   security, resource leaks, missing error handling.
4. Every finding must land on a specific line; if you cannot state the concrete consequence, skip it.
5. severity=high only for issues that cause data corruption, production outage or a security hole.
6. Return an empty array when there is nothing to report. Never pad the list.
Known false positives (do not report): log wording, TODO comments, hardcoded values in test files,
placeholder secrets in example code.
Team conventions: every outbound call needs a timeout; every database write needs an explicit
transaction; errors must be re-raised with context."""

TOOL = {
    "name": "report_findings",
    "description": "Submit code review findings",
    "input_schema": {
        "type": "object",
        "properties": {"findings": {"type": "array", "items": {
            "type": "object",
            "properties": {
                "file": {"type": "string"}, "line": {"type": "integer"},
                "severity": {"type": "string", "enum": ["high", "medium", "low"]},
                "issue": {"type": "string"}, "suggestion": {"type": "string"}},
            "required": ["file", "line", "severity", "issue", "suggestion"]}}},
        "required": ["findings"]},
}

def collect_diff(staged=False, path=None):
    if path:
        with open(path, encoding="utf-8", errors="replace") as fh:
            return fh.read()
    cmd = ["git", "diff", "--unified=3", "--no-color"]
    if staged:
        cmd.append("--cached")
    cmd += ["--", "."] + [f":(exclude){p}" for p in IGNORE]
    return subprocess.run(cmd, capture_output=True, text=True, check=True).stdout

def review(diff_text):
    if not diff_text.strip():
        return []
    body = {
        "model": MODEL, "max_tokens": 2048, "temperature": 0,
        "system": [{"type": "text", "text": RULES,
                    "cache_control": {"type": "ephemeral"}}],
        "tools": [TOOL],
        "tool_choice": {"type": "tool", "name": "report_findings"},
        "messages": [{"role": "user",
                      "content": "Review this diff:\n\n```diff\n"
                                 + diff_text[:MAX_CHARS] + "\n```"}],
    }
    for attempt in range(3):
        r = requests.post(f"{BASE}/v1/messages",
            headers={"x-api-key": KEY, "anthropic-version": "2023-06-01",
                     "content-type": "application/json"},
            json=body, timeout=120)
        if r.status_code in (429, 500, 502, 503, 504):
            time.sleep(2 ** attempt)
            continue
        r.raise_for_status()
        for block in r.json().get("content", []):
            if block.get("type") == "tool_use":
                return block["input"].get("findings", [])
        return []
    raise SystemExit("review failed: retries exhausted")

def to_markdown(findings):
    if not findings:
        return "### AI review\n\nNothing that needs action."
    out = [f"### AI review: {len(findings)} finding(s)", ""]
    for f in findings:
        out.append(f"- **{f['severity']}** `{f['file']}:{f['line']}` — {f['issue']}")
        out.append(f"  - Suggestion: {f['suggestion']}")
    return "\n".join(out)

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("--staged", action="store_true")
    ap.add_argument("--file")
    ap.add_argument("--fail-on", default="high", choices=["high", "medium", "none"])
    ap.add_argument("--markdown-out")
    a = ap.parse_args()

    findings = review(collect_diff(a.staged, a.file))
    for f in findings:
        print(f"{f['severity'].upper():6} {f['file']}:{f['line']}  {f['issue']}")
        print(f"       -> {f['suggestion']}")
    print(f"\n{len(findings)} finding(s)")
    if a.markdown_out:
        with open(a.markdown_out, "w", encoding="utf-8") as fh:
            fh.write(to_markdown(findings))
    if a.fail_on == "high" and any(f["severity"] == "high" for f in findings):
        sys.exit(1)

Three details matter. temperature: 0 makes the same diff roughly reproducible. max_tokens: 2048 allows a dozen findings — more than that means your rules are too loose. And system is an array of blocks with cache_control, which is Anthropic's caching syntax; repeat runs then skip the system-input charge. There's a catch, covered in the cost section.

Endpoint conventions and field details are in the Claude docs. The x-api-key and anthropic-version: 2023-06-01 headers are both required.

Step 3: pre-commit catches local issues

Speed is everything here. Past 20 seconds, developers start looking for --no-verify. Review staged changes only, block on high only:

Shell
#!/usr/bin/env bash
# .git/hooks/pre-commit
set -euo pipefail
[ "${SKIP_AI_REVIEW:-0}" = "1" ] && exit 0
command -v python3 >/dev/null || exit 0
python3 scripts/review.py --staged || {
  echo ""
  echo "AI review found high-severity issues; commit blocked."
  echo "Override with: SKIP_AI_REVIEW=1 git commit ..."
  exit 1
}

Keep review.py in scripts/ and point git config core.hooksPath scripts/hooks at a versioned hook directory, so new team members get it on clone instead of copying files by hand.

The layer where this breaks is "one line change, eight second wait". The fix is a cache keyed on the hash of git diff --cached: failed commits retry the same diff, and the wait drops to zero.

Step 4: review PRs in GitHub Actions

CI doesn't need to be fast, it needs to be stable and auditable. Trigger on PR events only, cancel superseded runs, post the result as a comment:

YAML
name: ai-review
on:
  pull_request:
    types: [opened, synchronize, reopened]
concurrency:
  group: ai-review-${{ github.event.pull_request.number }}
  cancel-in-progress: true

jobs:
  review:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    permissions:
      pull-requests: write
      contents: read
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install requests
      - name: Build diff
        run: |
          git diff "origin/${{ github.base_ref }}...HEAD" \
            --unified=3 --no-color -- . > /tmp/pr.diff
          wc -c /tmp/pr.diff
      - name: Review
        env:
          ZAIMOD_API_KEY: ${{ secrets.ZAIMOD_API_KEY }}
        run: python scripts/review.py --file /tmp/pr.diff --markdown-out review.md --fail-on none
      - name: Comment
        if: always()
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: gh pr comment "${{ github.event.pull_request.number }}" --body-file review.md

--fail-on none is deliberate: in CI the review never fails the build, it only comments. Letting a probabilistic component decide whether a PR can merge means the team disables it the first time it's wrong. The real gate lives in pre-commit, where a human is still at the keyboard and can see the context.

Keeping false positives down

Whether this survives two weeks depends entirely on its noise level. Four levers, in order of value:

One: write down what not to report. The "known false positives" block in RULES isn't filler — it does more work than the positive rules. Every false positive becomes a new line there. Two months in, the noise is down by half.

Two: be stingy with severity. high is reserved for data corruption, production outage, security holes. Marking everything high means there is no severity, and pre-commit becomes everyone's enemy.

Three: diff lines only. Models love to point at the legacy code next door, but that isn't this PR's responsibility. Rule 1 exists to stop it.

Four: baseline suppression. For the nightly sweep, record accepted-but-unfixed findings in .ai-review-baseline.json and filter them out of the report. Otherwise the same old issue shows up in the daily digest until nobody reads it.

Cost: one PR, and one month

Assuming 3,000 input + 800 output tokens per review, at live list prices (USD per 1M tokens, default group):

ModelInputOutputPer review30 PRs/day
gpt-5.3-codex$1.75$14.00$0.0165$0.50
claude-sonnet-5$4.00$20.00$0.0280$0.84
qwen3-coder$6.00$24.00$0.0372$1.12
claude-sonnet-4-6$6.00$30.00$0.0420$1.26
claude-opus-4-8$8.00$40.00$0.0560$1.68

For a team shipping 30 PRs a day, that's $15–50 a month — less than an hour of engineering time. The line item that actually needs planning is the nightly sweep: a mid-sized repo of 50k lines is about 1.5M input tokens per full pass, roughly $6 on sonnet-5. Weekly, that's $24/month, comparable to the PR reviews themselves. Rotate by directory instead of scanning everything nightly.

On cache_control: Anthropic enforces a minimum cacheable length (about 1,024 tokens on Sonnet). When RULES is too short, cache_creation_input_tokens stays 0 and nothing is cached. Padding the system block with conventions, known false positives and directory notes to around 1,500 tokens actually makes each call cheaper — more rules, lower cost. To verify it took effect, read cache_read_input_tokens; the method is in the base_url migration checklist.

Degrading gracefully

Three failure modes you must handle:

SymptomCauseHandling
429concurrency limitexponential backoff 1s / 2s / 4s, then skip this review
request timeoutoversized diff or a 60s gateway cutsplit per file, one request each
empty contenttool_choice left on autoforce the tool by name

The principle: a review may be missing, a merge must not be blocked. In the hook, a network failure passes and prints a warning. In CI, a failure skips the comment. The worst outcome for infrastructure like this is "the review service is down, so nobody can commit today".

Closing

What keeps an AI review pipeline alive isn't how much it reports, it's reporting little and being right. Only three things are technically load-bearing: forcing the output shape with tool_choice, sending diffs instead of whole files to control cost and noise, and using severity to decide what blocks versus what merely informs. The rest is rule tuning — and rule tuning means adding a line to RULES after each false positive, not upgrading the model.

Running the same diff on Opus doubles the per-review cost and won't necessarily halve the false-positive rate. Get the rules and the diff construction right first, then consider a bigger model. Prices move; the pricing page has live numbers. Field-level and streaming details for /v1/messages are in the Claude docs.

New accounts come with trial credit, so you can point review.py at your own repo today and decide on a daily model afterwards — pay as you go, balances don't expire. First-time setup is in the quickstart.

Further reading

#claude api code review#git hook#github actions#messages api#prompt caching
Get a free API key

Related reading