←Back/Design/Cursor
IntermediateDesign

How to Build an Interactive Piano Prototype with Sound Using Cursor

Create a reusable Cursor playground for interactive prototypes, then build a small digital piano whose visual keys, keyboard input, audio engine, focus behavior, and mobile interaction are explicit and testable.

How to Build an Interactive Piano Prototype with Sound Using Cursor

Source episode · 12:35

How I AI: A Designer's Guide to Cursor with Elizabeth Lin – From Y2K Aesthetics to Interactive Pianos

with Claire Vo

Elizabeth uses a reusable Cursor rule and prototype playground to create an old Mac style digital piano in a fresh chat. The generated prototype responds to mouse clicks with sound, and she asks Cursor which audio library it used and what other sound behavior can be explored.

Read the full episodeWatch on YouTubeListen on SpotifyListen on Apple

Before you start

What you need

  • A clean prototype repository with documented run and build commands
  • A Cursor rule for creating isolated routes or folders without changing existing prototypes
  • A clear keyboard layout, note mapping, octave range, and visual direction
  • A permitted browser audio library or Web Audio API approach
  • Desktop, keyboard, touch, accessibility, latency, and browser support criteria

What you’ll make

A working digital piano prototype with visible and keyboard operated keys, reliable audio initialization, clear pressed and focus states, polyphony and cleanup, plus documented implementation choices another designer can modify.

Tools used

Step by step

The workflow

Follow the sequence once, then adapt the prompts, checks, and handoffs to your own setup.

3 steps

Step01

Automate Setup with a Cursor Rule

Create a short Cursor rule for adding each prototype in its own folder and route, preserving existing files and using the repository's run and build commands. Keep design specific instructions in the task, not the global rule.

Example prompt
When I ask for a new prototype, create it under [folder pattern], add one registry or route entry, preserve all existing prototypes, use current dependencies when possible, and run [commands]. Do not change global styles unless requested. Summarize files and dependencies changed.
Step02

Generate the Piano with a Single Prompt

In a fresh chat, specify the piano range, note labels, input mapping, audio behavior, visual direction, interaction states, accessibility, and device support. Ask Cursor to implement the smallest complete instrument first.

Example prompt
Create a digital piano prototype at [route] with [octave range]. Support mouse, touch, and these computer keys: [mapping]. Use [Web Audio API or permitted library], initialize after user gesture, support [polyphony], and release notes on keyup and blur. Show pressed and focus states, accessible labels and instructions, reduced motion, and the [visual direction] style. Run [tests and build].
Step03

Explore and Interact with the Prototype

Play every input path, inspect latency and stuck notes, resize to mobile, navigate by keyboard, and ask Cursor to explain the sound engine, mapping, envelope, polyphony, and cleanup. Make one experiment at a time from a checkpoint.

Example prompt
How can I change the sound of the piano keys to a [sine wave]? Explain the relevant code.

What good looks like

  • Mouse, touch, and assigned computer keys trigger the intended notes with visible feedback.
  • Audio starts only after a permitted user gesture, does not leave stuck notes, and handles multiple simultaneous notes within the chosen limit.
  • Controls have accessible names, focus states, instructions, and a nonaudio indication of interaction.
  • The new prototype is isolated, runs at its own route, and does not break existing playground entries or the build.

Build your next product with ChatPRD

Turn an idea into a PRD, user stories, and a plan.

Try ChatPRD free

After the steps

Runbook notes

How to recover when the loop fails and where human judgment helps.

Recover

If it goes sideways

The interface renders but no sound plays because browser audio is suspended or assets fail
Initialize audio after an explicit user gesture, expose status, handle load errors, and test the chosen browsers.
Computer keys trigger while typing elsewhere or produce stuck notes
Scope listeners to the active instrument, ignore editable targets and repeat events as appropriate, and release notes on keyup, blur, and visibility change.
Fast input creates clipping, runaway voices, or harsh volume
Set gain safely, limit polyphony, schedule envelopes, and dispose voices after release.
A repository rule overwrites routes or mixes prototype dependencies
Constrain the rule to a new folder and registry entry, inspect the diff, and run existing prototypes and the build.

Keep building

Related workflows

Create an Automated Workout Tracker with a Simple Claude Voice Note
beginnerPersonalClaude

Create an Automated Workout Tracker with a Simple Claude Voice Note

Log your workouts effortlessly without ever opening a spreadsheet. This workflow uses the Claude mobile app to parse a simple voice note and automatically update your workout tracking spreadsheet in the background.

Open
How to Rebuild Your Gmail Inbox Inside Claude to Manage Email
advancedOperationsClaude

How to Rebuild Your Gmail Inbox Inside Claude to Manage Email

Create a custom email management interface within Claude to read, triage, and draft replies without opening the Gmail tab. This workflow helps you reclaim focus and compound your learnings by training your AI on every interaction.

Open
Automate Client Proposals and Onboarding with a Claude 'Pipeline Operator'
intermediateSalesClaude

Automate Client Proposals and Onboarding with a Claude 'Pipeline Operator'

Build a fully automated system in Claude to generate personalized, interactive HTML proposals and client onboarding materials, saving hours on administrative work and creating a high-touch experience.

Open

Start shipping
better products.

Join 100,000+ product managers who use ChatPRD to write better docs, align teams faster, and build products users love.

Start building freeRequest a demo
Free to start
No credit card
SOC 2 certified
Enterprise ready