←Back/Engineering/Claude
AdvancedEngineering

Create an AI-Powered Patch and Verification Loop for Security Bugs

Automate the process of verifying and patching security bugs found by an AI agent. This workflow uses a verifier sub-agent and a patching agent to propose and test fixes, with a human expert in the final review loop.

Create an AI-Powered Patch and Verification Loop for Security Bugs

Source episode · 16:55

How Mozilla Fixed 500 Security Bugs with Claude Mythos

with Claire Vo

16:55 to 32:30: Mozilla sends each suspected bug through an independent verifier, then lets a patching agent propose a fix and rerun focused tests before expert review.

Read the full episodeWatch on YouTubeListen on SpotifyListen on Apple

Before you start

What you need

  • A reproducible candidate vulnerability from the hunting agent
  • An isolated repository checkout with the relevant test commands
  • A verifier and patching agent with separate instructions

What you’ll make

A verified security finding paired with a focused patch, regression test, and evidence that the exploit no longer succeeds.

Tools used

Step by step

The workflow

Follow the sequence once, then adapt the prompts, checks, and handoffs to your own setup.

4 steps

Step01

Deploy a Verifier Sub-Agent

After an initial agent finds a vulnerability, pass its findings to a separate 'verifier' sub-agent. The verifier's role is to confirm the exploit is legitimate and check for 'cheats' from the first agent, such as modifying the source code to introduce the bug or using a developer-only setting.

Example prompt
After an initial agent finds a vulnerability, pass its findings to a separate 'verifier' sub-agent. The verifier's role is to confirm the exploit is legitimate and check for 'cheats' from the first agent, such as modifying the source code to introduce the bug or using a developer-only setting.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for deploy a verifier sub-agent.
Step02

Engage a Patching Agent

Once a bug is verified, engage a 'patching agent'. Provide it with the vulnerability report and the relevant code. The agent's task is to analyze the issue and generate a plausible code patch that resolves it.

Example prompt
You are a security engineering expert specializing in C++. A vulnerability has been found and verified in the following code. Your task is to analyze the vulnerability and generate a code patch that resolves the security issue while preserving the intended functionality. Vulnerability Report:
[Paste vulnerability report, including the exploit test case] Code:
[Paste relevant code snippet(s)] Provide only the code patch in a standard diff format.
Step03

Automate Patch Verification

Use your agentic system's build environment to automatically test the proposed fix. The system should apply the patch, rebuild the software, and run the original test case that triggered the crash. If the crash no longer occurs, the patch is considered successful.

Example prompt
Use your agentic system's build environment to automatically test the proposed fix. The system should apply the patch, rebuild the software, and run the original test case that triggered the crash. If the crash no longer occurs, the patch is considered successful.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for automate patch verification.
Step04

Incorporate a Human Expert in the Loop

Pass the verified patch and a full report to a human engineer for final review. The engineer's role is to provide oversight, check for broader architectural implications, and make the final decision to merge the fix. This is a critical step, as an agent may produce a correct but localized solution that needs to be expanded upon.

Example prompt
Pass the verified patch and a full report to a human engineer for final review. The engineer's role is to provide oversight, check for broader architectural implications, and make the final decision to merge the fix. This is a critical step, as an agent may produce a correct but localized solution that needs to be expanded upon.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for incorporate a human expert in the loop.

What good looks like

  • The verifier reproduces the issue independently
  • The patch blocks the exploit without weakening unrelated behavior
  • A regression test fails before the fix and passes after it

Build your next product with ChatPRD

Turn an idea into a PRD, user stories, and a plan.

Try ChatPRD free

After the steps

Runbook notes

How to recover when the loop fails and where human judgment helps.

Recover

If it goes sideways

The verifier accepts the hunter conclusion without independent evidence
Start the verifier with only the report and repository, then require its own reproduction steps.
The patch removes the symptom but leaves another path open
Rerun the exploit, adjacent variants, and the affected subsystem tests before accepting the fix.

Keep building

Related workflows

Automate UGC Video Editing From Raw Clips with ChatGPT
advancedContentChatGPT

Automate UGC Video Editing From Raw Clips with ChatGPT

Use ChatGPT's multi-modal capabilities to automatically edit user-generated content (UGC) style videos. The AI can transcribe clips, select the best takes, assemble a narrative, and redact sensitive on-screen information.

Open
Build and Deploy a Live Website with a Single ChatGPT Prompt
intermediateMarketingChatGPT

Build and Deploy a Live Website with a Single ChatGPT Prompt

use ChatGPT Sites to generate, build, and deploy a fully functional, data-backed website on a `chatgpt.site` domain using a single detailed prompt.

Open
Use ChatGPT Voice as a Personal Assistant for Logistics and Travel
advancedPersonalChatGPT

Use ChatGPT Voice as a Personal Assistant for Logistics and Travel

Use ChatGPT Voice with screen-aware context to delegate and parallel-process complex logistical tasks like booking travel and filing expense reports, all through a single conversation.

Open

Start shipping
better products.

Join 100,000+ product managers who use ChatPRD to write better docs, align teams faster, and build products users love.

Start building freeRequest a demo
Free to start
No credit card
SOC 2 certified
Enterprise ready