How Ryan Carson Manages 40 PRs a Day with Devin and Codex

It’s wild to think that just over a year ago, when I first had @ryancarson on the show, we were talking about hand-holding AI agents through three-step markdown files inside a local IDE. We were such babies! Today, the world of AI development has completely changed. When Ryan and I reconnected for this episode, the conversation was less about giving agents step-by-step instructions and more about managing a fleet of them working concurrently in the cloud. As Ryan puts it, "What the agents are doing is none of my business."

In this episode, I sat down with Ryan to get a look inside the new operating system that powers his fast-growing B2B startup, Untangle. He’s gone from a solo founder carefully guiding one agent to a manager of what he calls "throngs of agents," shipping around 40 PRs a day. It’s a masterclass in delegation, prioritization, and building systems that scale.

We explored his cloud-first development stack, his specific playbooks for quality control and operational oversight, and how he decides which tasks are for autonomous cloud agents like Devin versus local, hands-on tools like Codex. But maybe the most surprising part of our conversation was the anti-AI take: even with all this automation, the most important work—finding product-market fit and setting priorities—still happens away from the keyboard, by talking to humans and using a simple piece of paper.

Workflow 1: Managing a Fleet of Cloud Agents with Devin

The biggest shift in Ryan’s work over the past year has been moving from local development to what he believes is the future: "pretty much 100% cloud agents." He’s gone all-in on Devin, Cognition’s AI software engineer, and lives in their cloud environment. But with dozens of tasks running at once, the central challenge becomes management. How do you direct a team of 10, 20, or even 100 AI employees without losing your mind?

Ryan’s insight is that our job is to up-level our ability to manage agents, applying the same principles you’d use to manage a human team. You can't have 1,000 direct reports, and you can't have 1,000 active AI threads without a system.

`

`

The Management System

Here’s how Ryan structures his work in Devin to maintain focus and drive progress:

1. Prioritized Folders: He organizes all his Devin threads into folders that act as clear priority buckets. This is his version of a team structure:

• P0: The absolute highest-priority tasks, usually new features critical to the business. These are the things that must move forward no matter what.
• P1, P2: Important, but less urgent tasks.
• Bugs: A dedicated folder for bug fixes that need to be addressed quickly.
• Investors: A specialized area where he uses an agent with a custom playbook to help draft investor updates.

1. The Analog Anchor: To counteract the digital chaos of having eight screens and countless agent threads, Ryan uses a physical, handwritten to-do list. This simple piece of paper on his desk holds his weekly priorities, serving as a constant, tangible reminder of what truly matters.

`

`

"The skill is managing throngs of agents. Like that is what I do now. And to keep it straight in your head, you basically have to bucket these things."

This hybrid approach—digital organization for the agents and an analog anchor for himself—is how he stays grounded and ensures the army of AI assistants is always working on the right things.

Workflow 2: The 'Watchdog' Operational Playbook

As a solo founder juggling engineering, sales, and customer success, Ryan felt the stress of not knowing what was happening across all his customer accounts. To solve this, he built a reusable playbook in Devin called 'Watchdog'. This isn't about writing product code; it’s an operational workflow to maintain business health.

How Watchdog Works

When Ryan feels the need for a pulse-check on the business, he runs Watchdog. The process is fully automated:

1. Initiate the Playbook: He triggers the 'Watchdog' skill.
2. Iterate Through Accounts: The agent systematically goes into each customer account (in his case, family law firms using Untangle).
3. Gather Intelligence: For each account, it analyzes activity since the last check-in. It specifically looks for Sentry errors, UX bugs, and other anomalies.
4. Synthesize and Prioritize: Instead of just dumping raw data, the agent filters and sorts the information to surface the top three most critical problems across the entire customer base.
5. Check Status: Finally, it cross-references these problems with recent activity. It determines if a fix has already been shipped, if it's currently being worked on, or if there's an open PR that needs to be merged.

This gives Ryan a high-signal, low-noise report that tells him exactly where to focus his attention. It's a perfect example of using a coding agent for business operations, not just feature development.

Workflow 3: The 'Land PR' Quality Assurance Loop

Shipping 40 PRs a day is only impressive if the quality is high. To prevent chaos, Ryan created another playbook called 'Land PR' to standardize his code review and merge process. It’s his automated system for ensuring agents are producing solid, tested work before it goes into the main branch.

I have my own version of this called 'Merge Mommy', an Eve agent that assesses the risk of a PR and auto-approves low-risk changes. Ryan’s approach is a bit different and leverages Devin’s built-in capabilities.

The 'Land PR' Process

1. PR Ready: An agent finishes a task and reports that a PR is ready for review.
2. Trigger 'Land PR': Ryan kicks off the playbook.
3. Automated Review: The playbook first runs Devin Review, a built-in code review product, on the PR. It does this up to two times, allowing the agent to find and fix its own bugs in a loop.
4. Video Walkthrough: This is the key step. The agent records a video of itself testing the new feature in a browser. It narrates the test cases with captions and shows a checklist of tests passing or failing. It’s an incredibly powerful form of verification.
5. Human Approval: Ryan watches the short video to visually confirm the feature works as expected. He then gives the final command: Video approved. Land it.
6. Merge: The agent merges the PR into the main branch.

`

`

This workflow combines automated review with a quick, visual human checkpoint, creating a scalable yet reliable process for shipping code. It highlights how our CI/CD and DevOps practices need to evolve to incorporate agent-native verification steps.

Workflow 4: A Hybrid Approach to Design Systems

One area where many of us struggle with AI is design. I find that while models like Claude are fantastic conceptually, their execution can be lacking. Ryan and I are aligned on this. He still finds himself going back to Claude for design tasks, and I recently used a hybrid workflow to build the design system for my new business.

It’s a great example of using the right tool for the right job, combining the strengths of different models.

The Claude-to-Codex Design Workflow

1. Generate Concepts with Claude Design: Start with Claude Design. Give it a Figma file or an example site and ask it to create a design system. It excels at this, producing a clean design.md file with well-structured design tokens (colors, fonts, spacing, etc.).
2. Hand Off to Codex for Implementation: Download the design.md file. Then, feed it to a more implementation-focused agent like Codex.
3. The Prompt: My prompt was something like: "Take this design.md, plus what's in the Figma, and actually build me a technical design system."
4. Build Components: Codex takes the conceptual tokens and builds out the actual shared components in your codebase. It can even set up the file structure, such as creating an interconnected monorepo.
5. (Optional) Sync Back to Figma: Using the Figma plugin with Codex, you can even sync the final, coded design tokens back into your Figma file, closing the loop between design and development.

This process bridges the gap between a model that's great at high-level design thinking (Claude) and one that's a beast at implementation (Codex), resulting in a robust, production-ready design system.

The Real Work Still Happens Offline

After diving deep into these powerful AI workflows, we came back to a fundamental truth: more AI-generated code does not automatically create a successful product. The intelligence of these models has not magically created new markets or validated business ideas. Ryan discovered this firsthand when his initial consumer-facing version of Untangle failed to find traction. It was only when he got out of his chair and talked to a potential customer—a lawyer—that he found product-market fit.

Both of us, despite being all-in on AI, are now hiring humans. We both believe that the most important work a founder does is setting strategy and talking to users. My decision fatigue becomes a heuristic for what features to kill, and Ryan’s handwritten priority list is his anchor. The lesson is clear: use agents as a massive force multiplier to execute your vision, but never delegate the vision itself.

---

Thank you to our sponsors!

This episode is brought to you by:

• Work OS: Work OS gives you drop-in APIs for enterprise features like SSO, SCIM, and RBAC, so your app can become enterprise-ready and scale up market faster. Think of it like Stripe for enterprise features. Join leaders like OpenAI, Perplexity, and Cursor at workos.com.
• Jira by Atlassian: The teamwork graph in Jira delivers forty-four percent more accurate agent results with forty-eight percent less token usage. It pulls context from across your entire stack—Jira, Confluence, GitHub—and feeds it directly to your agents before they write a single line. Try them free at jira.dev.