How to Conduct a Blind 'Vibe Check' to Evaluate AI Model Quality

Set up an interface, like a local HTML file, to display outputs from multiple AI models for the same prompt. Anonymize the model names (e.g., Model A, Model B) to prevent bias in your evaluation.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for set up a blind comparison interface.

Define a simple scoring system that reflects your personal or brand taste. For example, use a 1 to 5 scale based on a core question like: "Would I ship this? Does it sound like me?"

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for define your 'vibe check' criteria.

Role: Senior Product Manager Task: Transform the following messy meeting notes into a structured Product Requirements Document (PRD). The PRD should include:
- Background and Problem Statement
- Target Audience
- Key Goals and Success Metrics
- User Stories
- Functional and Non-functional Requirements
- Out of Scope items Notes to use:
[Paste your unstructured notes, bullet points, or meeting transcript here]

"Ugh, deploys are red again."
"Remind me why I even started this company, LOL."
"honestly, let's just yellow post straight to prod and skip the test. I'm so done today"

Record your manual scores and qualitative notes for each model and task in a structured format, such as a JSON file. This creates a consistent dataset for your final analysis and comparison.

My actual values:
[insert the files, settings, accounts, or constraints for this step]

Give me the exact commands, settings, or output to use. Finish with a pass or fail check for record your scores.