Aug 26, 2026 · 5 min read

Build an AI-Powered PR Reviewer Bot for GitHub

Code review is essential but time-consuming. What if you could get instant AI feedback on every pull request before human reviewers even look at it?

In this tutorial, you’ll build a GitHub bot that automatically reviews pull requests using AI. It checks for bugs, security issues, performance problems, and style violations. No API keys for external services, everything runs on your machine.

How It Works

A PR is opened or updated on GitHub
Your server receives a webhook
AI reviews the code changes
Review comments are posted on the PR

Prerequisites

Ollama installed
Python 3.10+
GitHub account
ngrok (for local development)

Step 1: Pull the model

ollama pull qwen2.5-coder:7b

Step 2: Create the review bot

#!/usr/bin/env python3
"""AI GitHub PR Reviewer Bot."""

from flask import Flask, request, jsonify
import subprocess
import json
import urllib.request
import os
import hashlib
import hmac

app = Flask(__name__)

OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL = "qwen2.5-coder:7b"
GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN")
WEBHOOK_SECRET = os.environ.get("WEBHOOK_SECRET")


def verify_signature(payload, signature):
    """Verify GitHub webhook signature."""
    if not WEBHOOK_SECRET:
        return True  # Skip verification in dev
    expected = "sha256=" + hmac.new(
        WEBHOOK_SECRET.encode(),
        payload,
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, signature)


def get_pr_diff(repo, pr_number):
    """Get the diff for a PR."""
    url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}"
    headers = {
        "Authorization": f"token {GITHUB_TOKEN}",
        "Accept": "application/vnd.github.v3.diff"
    }
    req = urllib.request.Request(url, headers=headers)
    with urllib.request.urlopen(req) as resp:
        return resp.read().decode()


def review_code(diff, filename):
    """Use AI to review code changes."""
    prompt = f"""Review this code diff and provide feedback.

File: {filename}

Diff:
{diff}

Provide a structured review:
1. Bugs: Any bugs or errors you see
2. Security: Any security vulnerabilities
3. Performance: Any performance issues
4. Style: Any style or readability improvements
5. Suggestions: General improvements

Format each issue as:
- [severity] description
  Line/area: specific location
  Fix: suggested fix

Be concise and specific. Focus on real issues, not style preferences."""

    payload = json.dumps({
        "model": MODEL,
        "prompt": prompt,
        "stream": False,
        "options": {"temperature": 0.3, "num_predict": 1000}
    }).encode()

    req = urllib.request.Request(
        OLLAMA_URL,
        data=payload,
        headers={"Content-Type": "application/json"}
    )

    with urllib.request.urlopen(req, timeout=120) as resp:
        return json.loads(resp.read())["response"].strip()


def post_review(repo, pr_number, review_body):
    """Post a review comment on the PR."""
    url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/reviews"
    data = json.dumps({
        "body": review_body,
        "event": "COMMENT"
    }).encode()
    headers = {
        "Authorization": f"token {GITHUB_TOKEN}",
        "Content-Type": "application/json"
    }
    req = urllib.request.Request(url, data=data, headers=headers)
    with urllib.request.urlopen(req) as resp:
        return resp.status


@app.route("/webhook", methods=["POST"])
def webhook():
    """Handle GitHub webhook events."""
    signature = request.headers.get("X-Hub-Signature-256", "")
    if not verify_signature(request.data, signature):
        return "Unauthorized", 401

    event = request.headers.get("X-GitHub-Event", "")
    payload = request.json

    if event == "pull_request":
        action = payload.get("action", "")
        if action in ["opened", "synchronize"]:
            repo = payload["repository"]["full_name"]
            pr_number = payload["pull_request"]["number"]

            print(f"Reviewing PR #{pr_number} in {repo}...")

            # Get the diff
            diff = get_pr_diff(repo, pr_number)
            if not diff:
                return "No changes", 200

            # Review each file
            reviews = []
            for file_diff in diff.split("diff --git"):
                if not file_diff.strip():
                    continue
                filename = file_diff.split("\n")[0].split(" b/")[-1]
                if filename.endswith(".py") or filename.endswith(".js") or filename.endswith(".ts"):
                    review = review_code(file_diff, filename)
                    reviews.append(f"## {filename}\n\n{review}")

            # Post combined review
            if reviews:
                combined_review = "## AI Code Review\n\n" + "\n\n".join(reviews)
                combined_review += "\n\n---\n*Reviewed by AI PR Bot using Ollama*"
                post_review(repo, pr_number, combined_review)

            print(f"Review posted for PR #{pr_number}")

    return "OK", 200


if __name__ == "__main__":
    app.run(port=5000, debug=True)

Step 3: Create GitHub webhook

Go to your repository settings
Click “Webhooks” then “Add webhook”
Set Payload URL to your ngrok URL + /webhook
Set Content type to application/json
Select “Pull requests” events
Add your webhook secret

Step 4: Run the bot

# Start ngrok
ngrok http 5000

# Set environment variables
export GITHUB_TOKEN="your-token"
export WEBHOOK_SECRET="your-secret"

# Run the bot
python pr-reviewer.py

Under the Hood

The bot listens for pull request events. When a PR is opened or updated:

It fetches the diff from GitHub
Sends each file’s changes to the AI
Posts a combined review as a PR comment

The AI focuses on:

Actual bugs and errors
Security vulnerabilities (SQL injection, XSS, etc.)
Performance issues
Code clarity and maintainability

Limitations

Reviews are based on diffs, not full file context
Complex architectural issues may be missed
False positives are possible (AI may flag correct code)
Large PRs may hit token limits

Real-World Use Cases

Small teams with limited review bandwidth. When your team has 3 developers and 10 PRs per day, AI review catches the obvious issues before human reviewers spend time on each one.

Open source projects. Automated first-pass review helps maintainers triage contributions. The bot can flag common issues before human review.

Security-sensitive code. The AI catches common security vulnerabilities (SQL injection, XSS, hardcoded secrets) that might slip through human review.

Code style enforcement. The bot can check for consistent coding style across the team, reducing nit-picks in human reviews.

Tips for Better Reviews

Customize the prompt. Focus on what matters most for your codebase: security, performance, or style.
Filter by file type. Only review code files, not configs or documentation.
Set severity thresholds. Only post comments for high-severity issues to reduce noise.
Review the bot’s suggestions. Use the bot as a starting point, not a final authority.

Variations

Line-level comments: Post comments on specific lines
Severity filtering: Only report critical issues
Auto-fix suggestions: Generate fix commits
Team learning: Track common issues and suggest training

My Take

This bot catches obvious issues before human reviewers spend time on the PR. It is not a replacement for human review, but it catches the low-hanging fruit: typos, security holes, performance antipatterns.

Rating: 7.5/10 — Good first pass before human review. Catches 60-70% of common issues.

FAQ

How accurate is the AI review?

Good at catching obvious issues (security vulnerabilities, common bugs). Weak at architectural decisions and business logic. Use as a first pass, not a final review.

Can I customize what it checks for?

Yes. Modify the review prompt to focus on specific areas: security, performance, style, or your team’s coding standards.

Does this work with private repositories?

Yes, if your GitHub token has the right permissions. The bot runs locally, so your code never leaves your machine.

Can I use this with GitLab or Bitbucket?

Yes. Modify the webhook handling and API calls to use GitLab or Bitbucket APIs instead of GitHub.

Related: Code Review Best Practices · GitHub Actions Tutorial · Ollama Complete Guide