Back/Engineering/Devin
BeginnerEngineering

How to Use an AI Agent for Quick Code Maintenance and Internal Tool Improvements

Hand a small internal-tool annoyance to an AI coding agent, then turn its focused fix into a pull request without pulling an engineer away from larger work.

How to Use an AI Agent for Quick Code Maintenance and Internal Tool Improvements

22:00 to 25:00: Sahil Lavingia asks Devin to clean up Gumroad’s weekly Slack recap, then inspects the code and routes the pull request to a teammate.

Before you start

What you need

  • A recurring internal-tool problem with a visible example
  • Access to the repository and the system the tool affects
  • A clear rule for what should change and what should stay the same

What you’ll make

A narrow pull request that removes the annoyance, updates any related prompt or logic, and includes enough evidence for a teammate to finish the merge.

Tools used

  • Devin

    AI software engineer by Cognition Labs

Step by step

The workflow

Follow the sequence once, then adapt the prompts, checks, and handoffs to your own setup.

5 steps

Step01

Identify an Internal Tool Annoyance

Pick a recurring annoyance with a crisp before-and-after. Sahil uses a weekly Slack recap that shows empty projects and treats backend-only work as a shipment.

Example prompt
In our weekly Slack recap, empty project sections appear even when nothing shipped, and backend-only work is described as a shipment. The output should show only projects with user-facing shipments. Keep the existing format for qualifying projects.
Step02

Formulate a Clear, Simple Prompt

State the behavior change in plain language and include one real example. Mention any likely prompt, filter, template, or scheduled job the agent should trace.

Example prompt
Find the code that generates the weekly Slack recap and the AI prompt it uses. Make two changes: hide project sections with no shipments, and tell the model to summarize only user-facing features, improvements, and bug fixes. Do not include backend-only work. Use [example project] as the before-and-after case.
Step03

Assign the Task to Devin

Assign the task in the correct repository with permission to inspect the generation path. Ask for a focused pull request rather than prescribing the exact file or implementation.

Example prompt
Work in [repository]. Trace the weekly Slack recap from data selection through the model prompt and final formatting. Implement the two behavior changes above, add or update focused tests, and open a pull request. Avoid unrelated refactors.
Step04

Let the AI Implement the Fix

Let the agent locate the recap code, its prompt, and the filtering logic. Require a sample output so the result can be judged as a report, not only as a diff.

Example prompt
Run the focused tests and generate a sample recap containing one shipped project, one empty project, and one backend-only change. Include the sample and a short explanation of the filter and prompt changes in the pull request.
Step05

Review and Merge the Pull Request

Read the sample output and the small part of the diff that determines inclusion. Route the pull request to the teammate who owns the tool when they can confirm repository-specific behavior or missing tests.

Example prompt
Address these pull request findings: [list findings]. Keep qualifying project sections unchanged, add any missing regression case, and update the sample output before requesting another review.

What good looks like

  • The changed output matches the stated inclusion and exclusion rules
  • The pull request is limited to the internal-tool fix and any necessary tests
  • A teammate can understand the behavior change from the example and pull request description

Build your next product with ChatPRD

Turn an idea into a PRD, user stories, and a plan.

Try ChatPRD free

After the steps

Runbook notes

How to recover when the loop fails and where human judgment helps.

Recover

If it goes sideways

The agent expands a small cleanup into a broad refactor.
Restate the expected output with one before-and-after example and prohibit unrelated restructuring.
The code change fixes filtering but misses a separate AI prompt that also controls the output.
Ask the agent to trace the full generation path and list every rule, template, and filter involved before editing.
The diff looks reasonable but there is no evidence that the recurring report still runs.
Have the agent run the focused test or generate a representative sample, then attach the result to the pull request.

Start shipping
better products.

Join 100,000+ product managers who use ChatPRD to write better docs, align teams faster, and build products users love.

Free to start
No credit card
SOC 2 certified
Enterprise ready