←Back/Engineering/Claude/Claude
AdvancedEngineering

Build an AI Agentic Harness for Automated Security Bug Hunting

Create a custom AI agent that relentlessly hunts for security vulnerabilities in your codebase. This system uses a large language model with specific tools to find, test, and prove the existence of bugs.

Build an AI Agentic Harness for Automated Security Bug Hunting

Source episode · 05:32

How Mozilla Fixed 500 Security Bugs with Claude Mythos

with Claire Vo

05:32 to 23:00: Mozilla engineers wrap an LLM with repository tools, focused missions, and repeated test loops so it can search deeply for security bugs.

Read the full episodeWatch on YouTubeListen on SpotifyListen on Apple

Before you start

What you need

  • A repository checkout in an isolated test environment
  • Read, search, run, and test tools with explicit boundaries
  • A focused vulnerability class or suspicious subsystem

What you’ll make

A repeatable security-hunting harness that produces a reproducible vulnerability report with code location, exploit evidence, and impact.

Tools used

Step by step

The workflow

Follow the sequence once, then adapt the prompts, checks, and handoffs to your own setup.

7 steps

Step01

Select a Target

Select a specific source code file to analyze. The most effective targets are files identified as high priority by a separate scoring workflow.

Example prompt
Select a specific source code file to analyze. The most effective targets are files identified as high priority by a separate scoring workflow.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for select a target.
Step02

Initiate the Agent Loop

Initiate the main agent loop using a framework like the Claude Agent SDK. The agent needs two key inputs: access to a local checkout of the codebase and a clear mission statement.

Example prompt
You are an expert security researcher. Your mission is to find a memory safety vulnerability in the provided source code file: [path/to/target_file.cpp]. I have provided you with a set of tools, including a `browser_evaluator` that can run an HTML test case and detect crashes. Analyze the code, form a hypothesis, generate an HTML test case, and use the tool to verify it. We know there is a bug in this file; iterate until you find it and provide the exact test case that triggers the crash.
Step03

Focus the Agent with a 'Creative Lie'

Use a direct prompt that encourages the agent to search relentlessly. This 'creative lie' focuses the agent on the task by creating a strong assumption that a vulnerability exists and must be found.

Example prompt
We know there's a security bug in this file. You have to go find it.
Step04

Hypothesize and Generate Test Cases

The agent analyzes the target code to form hypotheses about potential exploits. Based on its reasoning, it generates concrete test cases, such as HTML files, designed to trigger a crash or expose a vulnerability.

Example prompt
The agent analyzes the target code to form hypotheses about potential exploits. Based on its reasoning, it generates concrete test cases, such as HTML files, designed to trigger a crash or expose a vulnerability.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for hypothesize and generate test cases.
Step05

Use Tools to Evaluate Tests

The agent uses the tools provided by the harness to evaluate its test cases. For example, a browser_evaluator tool can run a generated HTML file inside a special build of the software to detect memory safety errors and report back a pass or fail signal.

Example prompt
The agent uses the tools provided by the harness to evaluate its test cases. For example, a `browser_evaluator` tool can run a generated HTML file inside a special build of the software to detect memory safety errors and report back a pass or fail signal.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for use tools to evaluate tests.
Step06

Iterate Relentlessly

The agent operates in a continuous loop, receiving feedback from its tools on each attempt. If a test case fails to produce a crash, the agent analyzes the result and generates a new approach. This cycle repeats until a test case successfully triggers a vulnerability.

Example prompt
The agent operates in a continuous loop, receiving feedback from its tools on each attempt. If a test case fails to produce a crash, the agent analyzes the result and generates a new approach. This cycle repeats until a test case successfully triggers a vulnerability.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for iterate relentlessly.
Step07

Capture the Verified Output

When the agent successfully triggers a crash, the system captures the exact, reproducible test case that caused it. This verifiable proof of the vulnerability is the primary output of the workflow, eliminating the false positives common in unverified AI reports.

Example prompt
When the agent successfully triggers a crash, the system captures the exact, reproducible test case that caused it. This verifiable proof of the vulnerability is the primary output of the workflow, eliminating the false positives common in unverified AI reports.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for capture the verified output.

What good looks like

  • Every reported issue includes a reproducible test or failing case
  • The report identifies the exact code path and security impact
  • The agent records attempts and evidence instead of merely asserting a vulnerability

Build your next product with ChatPRD

Turn an idea into a PRD, user stories, and a plan.

Try ChatPRD free

After the steps

Runbook notes

How to recover when the loop fails and where human judgment helps.

Recover

If it goes sideways

The agent stops after shallow pattern matching
Give it a narrower subsystem, require call-path tracing, and run multiple investigation passes.
The agent claims a bug without proving reachability
Require a minimal reproduction that exercises the vulnerable path in the isolated environment.

Keep building

Related workflows

Automate UGC Video Editing From Raw Clips with ChatGPT
advancedContentChatGPT

Automate UGC Video Editing From Raw Clips with ChatGPT

Use ChatGPT's multi-modal capabilities to automatically edit user-generated content (UGC) style videos. The AI can transcribe clips, select the best takes, assemble a narrative, and redact sensitive on-screen information.

Open
Build and Deploy a Live Website with a Single ChatGPT Prompt
intermediateMarketingChatGPT

Build and Deploy a Live Website with a Single ChatGPT Prompt

use ChatGPT Sites to generate, build, and deploy a fully functional, data-backed website on a `chatgpt.site` domain using a single detailed prompt.

Open
Use ChatGPT Voice as a Personal Assistant for Logistics and Travel
advancedPersonalChatGPT

Use ChatGPT Voice as a Personal Assistant for Logistics and Travel

Use ChatGPT Voice with screen-aware context to delegate and parallel-process complex logistical tasks like booking travel and filing expense reports, all through a single conversation.

Open

Start shipping
better products.

Join 100,000+ product managers who use ChatPRD to write better docs, align teams faster, and build products users love.

Start building freeRequest a demo
Free to start
No credit card
SOC 2 certified
Enterprise ready