Back/Data/Claude/Gemini
BeginnerData

How to Use the 'AI Duke It Out' Method to Improve Stubborn Model Outputs

When one model remains stuck, give a second model the original request, failed output, and desired example, use its critique to identify the mismatch, and test a targeted revision instead of repeating the same prompt.

How to Use the 'AI Duke It Out' Method to Improve Stubborn Model Outputs

When a model keeps producing the wrong result, Tomasz gives another model the original input, the unwanted output, and an example of the result he wants, then lets the models critique and refine the work through a small script.

Before you start

What you need

  • The original prompt and relevant context
  • The exact failed output
  • A concrete desired example or acceptance tests
  • A second model with a meaningfully different failure pattern
  • A safe way to remove secrets or personal data before sharing context

What you’ll make

A revised prompt or output that addresses a diagnosed mismatch and passes fixed acceptance tests, with the model contributions recorded so the result can be reproduced.

Tools used

Step by step

The workflow

Follow the sequence once, then adapt the prompts, checks, and handoffs to your own setup.

4 steps

Step01

Identify a Problematic Output

Save the original prompt, relevant context, model and settings, exact output, and the acceptance criterion it missed. Reduce the complaint to an observable mismatch rather than describing the output as merely bad.

Step02

Prepare Your Evidence

Create a small evidence packet with the original input, failed output, desired example, and several nearby test cases. Remove unrelated or sensitive context before sending it to another model.

Step03

Engage a Second AI Model

Ask a second model to diagnose the failure before it rewrites anything. Require a comparison of the instruction, actual behavior, desired behavior, and the smallest prompt or context change likely to help.

Example prompt
Diagnose this model failure. Original request: [input]. Actual output: [failed output]. Desired behavior: [example or criteria]. Test cases: [cases]. Identify the specific mismatches, likely instruction or context causes, and the smallest correction. Do not rewrite the output yet.
Step04

Frame the Competition

Apply the best diagnosis to a revised prompt, run it with the original and nearby test cases, and compare the results. Keep the competitive framing playful if it helps, but accept the revision only when it passes the fixed criteria.

Example prompt
Using this diagnosis: [diagnosis], produce a revised result for [input]. Meet these acceptance criteria: [criteria]. Do not copy the desired example verbatim. Then report which change addressed each mismatch and how the result performed on [test cases].

What good looks like

  • The second model identifies specific differences between the failed and desired outputs.
  • The revision changes the instruction or context responsible for the failure rather than adding empty pressure.
  • The result is judged against the same examples and tests used before the model switch.
  • Sensitive context remains within approved tools and is shared only when necessary.

Build your next product with ChatPRD

Turn an idea into a PRD, user stories, and a plan.

Try ChatPRD free

After the steps

Runbook notes

How to recover when the loop fails and where human judgment helps.

Recover

If it goes sideways

The second model insults the first but offers no useful diagnosis
Ask for a structured comparison of requirements, failed behavior, likely cause, and a minimal correction.
The revision copies the desired example and fails on nearby cases
Add several fixed examples and counterexamples, then test the prompt across all of them.
Models repeatedly rewrite each other without measurable improvement
Set acceptance tests and a round limit, and stop when no new diagnosis appears.
The full failed conversation exposes secrets or personal data to another provider
Minimize and redact the context, or use a second model within the same approved environment.

Start shipping
better products.

Join 100,000+ product managers who use ChatPRD to write better docs, align teams faster, and build products users love.

Free to start
No credit card
SOC 2 certified
Enterprise ready