The Builder's Compendium

You are helping me set up a new Workbench prompt. I will describe the task; you produce a complete Workbench-ready prompt with all scaffolding.

TASK DESCRIPTION:
[Paste 2-3 paragraphs describing what the prompt should do, who it's for, what good and bad outputs look like.]

EXAMPLES I HAVE:
[Paste 2-3 input/output pairs, even rough ones. If you have none, say so — I'll generate synthetic ones.]

CONSTRAINTS:
• Target model: [Opus 4.7 / Sonnet 4.6 / Haiku 4.5]
• Target temperature: [0 / 0.3 / 0.7 / 1]
• Max tokens expected in output: [N]
• Must not: [things the prompt must refuse or avoid]
• Must always: [non-negotiable behaviours]

PRODUCE:

## PART 1: THE SYSTEM PROMPT
Structured with:
• Role framing (one sentence, concrete)
• Task statement (what, for whom, to what standard)
• Input format description (what the {{variables}} will contain)
• Output format specification (structured if possible — JSON schema, XML, or delimited)
• Failure modes and how to handle them
• Tone/voice instructions if relevant

Use XML tags for each section (<role>, <task>, <input_format>, etc.) — this is Anthropic's recommended structure.

## PART 2: THE USER MESSAGE TEMPLATE
With {{variable}} placeholders matching the input format. Wrap each variable in XML tags for clarity.

## PART 3: 3 EXAMPLES
In Workbench's structured Examples format (input → output pairs). If I provided fewer than 3, generate synthetic ones in the same style.

## PART 4: EVAL TEST CASES (seeds)
Produce 5 test cases spanning:
• Happy path (2 cases)
• Edge case (1 case — empty input, unusual format, etc.)
• Adversarial (1 case — attempted prompt injection, off-topic)
• Boundary (1 case — right at the limit of what the prompt should handle)

Format as a CSV I can paste into the Eval Tool.

## PART 5: TEMPERATURE & MODEL JUSTIFICATION
One paragraph explaining why the temperature and model I chose are right for this task. If you think a different choice would be better, say so and why.

## PART 6: "WHAT YOU'LL WANT TO ADD LATER"
Three things this prompt doesn't do now but you'd expect to need within 2-4 weeks of production use. Rate-limiting logic, fallback behaviour, observability hooks, etc.

I want to build a proper eval suite for an existing prompt before I trust it in production.

PROMPT TO EVALUATE:
[Paste the full system prompt and user message template.]

REAL INPUT SAMPLES:
[Paste 10-30 actual inputs the prompt handles in production. Anonymised.]

SUCCESS DEFINITION:
[One paragraph. Example: "The prompt should correctly extract the 5 structured fields 95% of the time, gracefully say 'insufficient information' 100% of the time when a field is genuinely absent, and never hallucinate a value that isn't in the source."]

TASK:

## PART 1: CASE CATEGORISATION
From my 10-30 samples, cluster them into categories. Report:
• How many cases per category
• What makes each category distinct
• Which categories are currently under-represented and where I should gather more samples

## PART 2: IDEAL OUTPUTS
For each of the samples, produce the ideal output (what a perfect response looks like). Use my prompt's expected format.

If any sample is genuinely ambiguous and has multiple defensible correct answers, flag it and list them.

## PART 3: THE TEST SUITE
A CSV formatted for direct import into the Eval Tool, with columns:
• case_id
• category (from Part 1)
• input (with variable names matching my prompt's {{placeholders}})
• ideal_output
• grading_criteria (what to check for — format, specific field values, tone, completeness)
• severity_if_wrong (blocker / high / medium / low)

Rows: all my samples + 5 synthetic edge cases you design (empty inputs, malformed inputs, adversarial inputs, inputs near the capability boundary, inputs in a different language if relevant).

## PART 4: RUBRIC FOR THE 5-POINT GRADE
Define what each of the 5 grades means for THIS prompt, specifically:
• 5 = [specific description]
• 4 = [specific description]
• 3 = [specific description]
• 2 = [specific description]
• 1 = [specific description]

Not generic ("excellent, good, fair..."). Tied to the actual task.

## PART 5: REGRESSION-TEST PROTOCOL
When should I re-run this suite?
• What changes should trigger a full re-run?
• What changes can use a subset?
• What's the target score that blocks a release?

## PART 6: THE CASES YOU EXPECT TO FAIL
Your best guess at which 3-5 test cases are most likely to fail on the first run, and why. This calibrates my expectations before I click Run.

I want to build my first Managed Agent. Walk me through it as a peer pair-programming session, not a tutorial. I have API access and the beta header; I've written prompts but never shipped a Managed Agent.

AGENT PURPOSE:
[One paragraph. Example: "A nightly agent that fetches yesterday's production errors from our logging API, clusters them by stack trace, and posts a prioritised summary to our #eng-oncall Slack channel."]

TOOLS/MCP NEEDED:
[List. Example: "Logging API (REST), Slack MCP, optional web_search for error-message lookups"]

TASK:

## PART 1: THE AGENT DEFINITION
Produce the exact API call to create the agent, in Python (using the official Anthropic SDK).

Include:
• Model choice (with reasoning — Sonnet 4.6 is usually right; justify if you pick different)
• System prompt (with a clear role, task statement, tool-use policy, failure handling)
• Tool definitions with JSON schemas
• MCP server references
• Any skills to attach

## PART 2: THE ENVIRONMENT CONFIG
The container setup:
• Pre-installed packages needed
• Network access rules (allow-list, not deny-list)
• Mounted files (if any)
• Vault references for credentials

Explain each choice. Tell me which defaults I should question.

## PART 3: THE SESSION LAUNCH
The code to start a session, send the initial user event, and stream SSE events back.

Include:
• How to parse the SSE stream
• Where to catch errors
• How to checkpoint and resume if the session fails mid-execution
• How to interrupt and redirect mid-run if the agent goes off-track

## PART 4: THE DEV LOOP
How I should iterate on this agent day-to-day:
• Local testing setup (what runs locally vs what's server-side)
• How to use Workbench to iterate on the system prompt without redeploying
• How to version the agent config
• How to run eval suites against the agent (not just the prompt)

## PART 5: SAFETY RAILS
Before I point this at production:
• What spend cap should I set and why that number
• What inputs should trigger human-in-the-loop escalation
• What outputs should trigger a halt
• What logs to capture for incident response

## PART 6: THE 90-MINUTE PLAN
Break the work into 6 × 15-minute increments. At each checkpoint, what should be working? What's the 'if this is broken, stop and fix' criterion?

Write this as I would actually execute it, not as a book chapter. Keep it in peer-voice.

I have a Workbench prompt that's being reused across multiple projects. Help me convert it to a proper Skill.

THE PROMPT:
[Paste the full system prompt, user message template, and any examples.]

WHERE IT'S BEING USED:
[List the 2-5 places this prompt currently lives. Example: "1. Our customer-onboarding agent. 2. A Zapier workflow. 3. An internal tool at the marketing team's request."]

TASK:

## PART 1: SHOULD THIS BE A SKILL?
Honest answer first. Not every reused prompt should be a Skill. Evaluate:
• Is the prompt stable, or still rapidly iterating? (Skills want stability.)
• Does it have a clear trigger — a description that tells Claude when to use it?
• Is it self-contained, or does it depend on context from the calling system?
• Is it genuinely reusable, or is each usage a slight variant that would diverge over time?

Recommend: convert to Skill / keep as Workbench prompt / split into multiple Skills / rewrite the calling systems instead.

If the recommendation is 'don't convert', stop here and explain why. Don't force it.

## PART 2: IF YES, THE SKILL SPEC
Produce the complete SKILL.md:

---
name: [skill-name-in-kebab-case]
description: [1-2 sentences that tell Claude when to use this Skill, not what it is]
---

# [Skill Title]

## Purpose
[One paragraph.]

## When to Use
[Bullet list of specific trigger conditions.]

## When NOT to Use
[Bullet list — critical for stopping Claude from over-applying.]

## Instructions
[The instructions, rewritten for Skill-first use. Should NOT assume the prompt structure of the original Workbench prompt. Should stand alone.]

## Examples
[3 concrete input → approach → output triads.]

## Output Format
[Spec of what a Skill-generated response should look like.]

## Failure Handling
[What to do when the input is outside scope.]

## Part 3: the migration plan
For each of the places the prompt is currently used:
• What changes
• What breaks if it changes wrong
• How to validate before rollout
• What to do if the Skill-generated output differs from the prompt-generated output

## PART 4: THE TESTING PROTOCOL
Eval suite for the Skill that covers:
• The original use cases (regression)
• New use cases the Skill now unlocks that the old prompt couldn't handle
• Edge cases where Claude might incorrectly trigger the Skill

## PART 5: DISTRIBUTION
Where should this Skill live?
• Personal (your claude.ai settings)
• Team (shared in your org's Claude workspace)
• Open-source (Anthropic skills repo or your GitHub)
• Bundled (inside a Managed Agent config)

Recommend one, with reasoning.

## PART 6: THE THING YOU'LL GET WRONG FIRST
Based on the prompt and its usage patterns, your best guess at the most likely failure mode once it's a Skill. This is where I'll spend my first week of debugging — warn me.

I want to adopt the Advisor Tool strategy on [AGENT/WORKFLOW]. Currently it's running fully on Opus 4.7; I suspect most of the work doesn't need Opus.

CURRENT SETUP:
• Workflow: [brief description]
• Current model: Opus 4.7 throughout
• Average cost per task: $[X]
• Average tokens per task: [N input / M output]
• Quality bar: [how you measure, and the current score]

TASK:

## PART 1: WORK-TYPE TAXONOMY
Break the workflow into distinct step-types:
• Retrieval steps (fetch, parse, extract) — executor territory
• Transformation steps (format, rewrite, summarise) — executor with occasional advisor
• Judgement steps (classify, prioritise, choose) — this is where advisor earns its keep
• Generation steps (write, design, synthesise) — advisor for the hard ones, executor for templated

For each step in my workflow, categorise it.

## PART 2: THE ROUTING STRATEGY
For each step, prescribe:
• Which model should run it (Executor: Sonnet 4.6 or Haiku 4.5; Advisor: Opus 4.7)
• What triggers advisor intervention (confidence threshold, specific conditions, always)
• What the advisor gets to see (full context, summarised context, just the question)
• What the advisor returns (guidance, full rewrite, thumbs up/down)

## PART 3: IMPLEMENTATION CODE
Produce Python (or TypeScript if preferred) showing:
• The advisor-tool beta header
• The executor-advisor message pattern
• How to pass state between them
• How to handle cases where the advisor disagrees with the executor

## PART 4: COST MODEL
A spreadsheet-ready breakdown:
• Current cost: 100% Opus. $ per task.
• Naïve Sonnet: $ per task (and expected quality drop).
• Advisor routing: $ per task (and expected quality preservation).
• Break-even: at what task volume does advisor routing pay back the engineering effort to ship it?

Don't just claim 60-80% savings — show me the math for my specific workflow.

## PART 5: THE QUALITY REGRESSION TEST
Before going live, I need to prove the advisor-routed version matches Opus quality on MY workflow.

Design:
• The test set (size, composition, sourcing)
• The grading protocol (LLM-as-judge, human eval, task-success metrics)
• The pass criterion (how much degradation is acceptable, per work-type)
• The fall-back protocol (when to revert to pure-Opus)

## PART 6: THE FIRST THING THAT WILL BREAK
Advisor routing fails in specific ways. Your best prediction of where this particular workflow will degrade, and what to monitor.

I want to build an MCP server that exposes [SYSTEM] to Claude. I've used MCP servers before but never built one.

SYSTEM TO EXPOSE:
[Description. Example: "Our internal customer database. Read-only for most operations; specific write endpoints for updating customer tags and notes."]

TARGET CONSUMERS:
[Where will this MCP server be used? Example: "Desktop app for our sales team, Managed Agents for automation, possibly Claude Code for internal dev tools."]

AUTH:
[API key / OAuth / mTLS / none. Example: "API keys scoped to a workspace, rotated quarterly."]

TASK:

## PART 1: THE TOOL SURFACE DESIGN
Before any code: what tools should this server expose?

For each tool:
• Name (verb_noun convention)
• One-sentence description (this is what Claude reads to decide when to call it)
• Parameters (name, type, required, description)
• Return type (and example response)
• Failure modes (what errors, when)

Rules:
• Prefer composable tools (list_customers, get_customer, update_customer_tag) over monolithic ones (manage_customers).
• Every parameter needs a description — Claude uses these to decide which to pass.
• Return types should be JSON, structured, with consistent error shapes.
• Don't expose destructive operations without an explicit confirmation step.

## PART 2: THE SERVER SKELETON
Produce the complete MCP server in [Python / TypeScript — pick Python unless I specified otherwise].

Include:
• SDK setup (mcp package)
• Tool registration with JSON schemas
• Auth middleware (API key validation or OAuth flow)
• Logging (request/response, latency, errors)
• Health-check endpoint
• Rate limiting or per-client throttling

Keep it production-grade but minimal. No premature optimisation.

## PART 3: THE HOSTING PLAN
For my target consumers [from brief], what's the right deployment model?
• stdio (local, per-user)
• HTTP/SSE (hosted, team-accessible)
• Cloudflare Worker / Vercel Edge (globally-distributed)

Recommend one and justify. Include the deploy command or config file.

## PART 4: CLIENT CONFIGURATION
Show the config changes needed in:
• Claude Desktop (.mcpb install or manual JSON)
• Claude Code (settings or config file)
• Managed Agents (agent config snippet)

Include the exact JSON/YAML each client expects.

## PART 5: SECURITY REVIEW
Before exposing this to a model, walk through:
• What data is at rest vs in transit
• What operations are read-only vs mutating
• What happens if a prompt injection tricks Claude into calling a destructive tool
• How to add a 'confirmation required' pattern for high-stakes operations
• What to log for forensics

## PART 6: THE TESTING STRATEGY
• Unit tests for each tool function
• Integration test that calls the running server via the MCP protocol
• End-to-end test in Claude Desktop or a Managed Agent session
• Red-team test: what happens with adversarial inputs

Produce the unit test file for Part 2's skeleton.

## PART 7: THE ONE TOOL YOU SHOULDN'T HAVE ADDED
Reviewing the tool surface from Part 1: is there one you'd advise against exposing on day one? What's the risk?

I want to add prompt caching to [AGENT/PROMPT] to cut costs on repetitive context.

CURRENT SETUP:
• System prompt length: [N tokens]
• Static context (docs, examples, reference material): [N tokens]
• Dynamic context per turn (user message, tool results): [N tokens]
• Average turns per session: [N]
• Average sessions per day: [N]
• Current monthly cost: $[X]

TASK:

## PART 1: CACHE-ELIGIBILITY ANALYSIS
Walk through my message structure. For each block:
• Block type (system prompt, doc context, examples, user message, tool output, assistant reply)
• Size in tokens (estimated)
• Volatility (never changes / changes per session / changes per turn / always changes)
• Caching recommendation (always cache / conditionally cache / never cache)

The rule: cache stable blocks that are at least 1024 tokens and used in multiple turns or sessions.

## PART 2: THE CACHED MESSAGE STRUCTURE
Rewrite my message construction to add cache_control markers in the right places.

Show:
• Where the cache_control: {"type": "ephemeral"} markers go
• The order matters: cached blocks must come before dynamic blocks
• The TTL default (5 min) and when to use 1-hour TTL instead (beta)
• What to do about the system prompt specifically

Include Python code for the correctly-structured message construction.

## PART 3: THE COST MODEL
Given my usage pattern, compute:
• Cache write cost per first-session turn
• Cache read cost per subsequent turn
• Break-even: how many cache hits do I need to come out ahead?
• Expected monthly cost post-caching (best/realistic/worst case)

Be honest about the worst case. If my traffic is too sparse for caching to help, say so.

## PART 4: THE CACHE-HIT MONITORING
Production caching requires observability. Design:
• What to log per request (cache_read_tokens, cache_write_tokens, cache_creation_tokens)
• Dashboards to build (cache hit rate, tokens-saved-per-day, cost-per-session)
• Alerts (cache hit rate drops below threshold, cache writes spike suggesting cache eviction)

## PART 5: CACHE INVALIDATION STRATEGY
The things that will break your cache:
• System prompt changes (any modification invalidates)
• Context doc updates (same)
• Different cache_control marker placement

How to deploy prompt changes without catastrophic cache-miss days:
• Staged rollout
• Warm-up period
• A/B variants during transition

## PART 6: WHEN CACHING DOESN'T HELP
Honest assessment. Caching is a bad fit when:
• Each session has unique reference material (per-customer docs, dynamic knowledge bases)
• Traffic is too sparse (fewer than ~10 requests per cache TTL)
• The static prefix is under ~1024 tokens
• The agent's context is being rewritten each turn anyway

Does any of this apply to me?

## PART 7: THE 1-HOUR TTL DECISION
Anthropic offers a 1-hour TTL (beta) at higher cache-write cost. When is it worth it?
• For my traffic pattern: recommend or not, with math.

I'm about to ship an autonomous agent to production. Before it runs, I need the full cost-guardrails setup.

AGENT CONTEXT:
• Purpose: [description]
• Expected runs per day: [N]
• Expected tokens per run: [best estimate, range]
• Monthly budget: $[X]
• Consequences of overrun: [what happens if it costs 10x your estimate]

TASK:

## PART 1: MULTI-LAYER BUDGET DESIGN
Design four layers of cost protection:

LAYER 1 — WORKSPACE SPEND CAP
Set the hard ceiling. At what $ should the workspace auto-suspend? This is the "I'm okay with losing this much" number. Usually 1.5-2x the monthly budget.

LAYER 2 — ALERT THRESHOLDS
Email alerts at 50%, 75%, 90% of budget. Route to whom, with what message.

LAYER 3 — AGENT-LEVEL BUDGET
Per-session token caps. Per-run max duration. When the agent approaches its limit, does it halt or escalate?

LAYER 4 — APPLICATION-LEVEL RATE LIMITING
If the agent is triggered by user action or webhook: rate-limit the triggers. Queue, don't DOS yourself.

For each layer, produce the exact config or code change.

## PART 2: RUNAWAY DETECTION
Most cost incidents aren't gradual — they're a bug or injection attack that causes a single run to loop or fan out.

Design:
• Per-run token-count hard limit (halt, not warn)
• Per-run wall-clock limit
• Tool-call count limit (e.g. max 50 tool calls per session)
• Recursion detection (same tool with same args 3+ times)

Produce the wrapper code that enforces these at the application layer.

## PART 3: THE INCIDENT PROTOCOL
When a cost spike happens — what's the response?
• Who gets paged
• First 5 minutes: what to check
• First 30 minutes: what to contain
• Post-incident: what to change

Write this as a runbook I can put in a wiki.

## PART 4: THE BUDGET AS A FEATURE, NOT A LIMIT
The advanced move: make the budget a first-class part of the agent's behaviour.

Show how to:
• Pass the remaining budget into the agent's system prompt ("You have $X left this month")
• Have the agent downgrade model choice as budget tightens (Opus early, Sonnet mid-month, Haiku late)
• Have the agent defer non-urgent work when budget is tight

This turns 'running out of budget' from an incident into a graceful degradation.

## PART 5: THE THINGS YOUR CURRENT BUDGET MISSES
Estimate the cost of:
• Retries (transient failures, rate limits, 5xx from tool servers)
• Prompt injection attacks causing extra token use
• Accidental prompt changes causing cache-miss storms
• Bugs that cause multiple sessions for one logical task

Add each to your budget or to the guardrails.

## PART 6: THE BUDGET POST-MORTEM QUESTION
Imagine you hit your budget cap unexpectedly in week 3. Write the 5 questions the post-mortem should answer — before the incident happens, so you know what to instrument.

I'm building a multi-tenant SaaS on Managed Agents. Each customer has their own data, their own tool credentials, and must never see another customer's data.

TENANT MODEL:
[Describe. Example: "100 SMB customers. Each has: their own Slack workspace (OAuth), their own GitHub repos (App install), and their uploaded knowledge base. A tenant is a single company; users within a tenant share data."]

ISOLATION REQUIREMENTS:
[What must never cross tenants? Example: "No cross-tenant data visibility in any retrieval. Logs scrubbed of tenant content. Failover between tenants never leaks state."]

TASK:

## PART 1: THE ISOLATION ARCHITECTURE
Design the layered isolation:

AGENT LEVEL
• One shared agent definition, parameterised by tenant, OR one agent per tenant?
• How tenant identity gets threaded through (request parameter, session metadata, custom header)

ENVIRONMENT LEVEL
• Per-tenant container config vs shared config
• Filesystem isolation (mount different files per tenant)
• Network allow-lists (some tenants may have IP allow-lists on their own services)

VAULT LEVEL
• Credential injection: per-tenant Vaults, one Vault with namespacing, or hybrid
• Key-naming convention (tenant_id is the natural namespace)
• Credential rotation story (must not cause cross-tenant outages)

SESSION LEVEL
• Session IDs and how tenant_id is tagged on them
• Event-history retention per tenant
• How to stream logs to the right tenant's observability system

For each layer, produce the config snippet and a one-paragraph justification.

## PART 2: THE TENANT-AWARE SYSTEM PROMPT
The agent's system prompt should know whose data it's operating on. Design:
• How tenant identity enters the system prompt (at session creation, via a Skill, via an init tool call)
• What the agent must never do across tenants (explicit prohibitions)
• How the agent handles ambiguous references ("the customer data" when there are multiple customers)

Produce the system prompt template with clear tenant-aware sections.

## PART 3: TOOLS AS ISOLATION BOUNDARIES
Every tool the agent calls must respect tenant scope.

For each tool in my setup:
• How tenant_id is passed in (required parameter, context variable, out-of-band)
• How the tool validates tenant context (authorisation check at the tool's backend)
• What happens on mismatched or missing tenant context (fail closed, not open)

Show the tool-call examples with tenant threading done right.

## PART 4: THE CROSS-TENANT LEAK TESTS
The tests you write knowing attackers will try to break isolation:
• Prompt injection attempting to query another tenant's data
• Tool-call argument manipulation (wrong tenant_id, no tenant_id)
• Session-confusion attacks (reusing session IDs across tenants)
• Cache contamination (prompt cache leaking context between tenants)

Write the test suite. Include the specific adversarial inputs.

## PART 5: LOGGING WITHOUT LEAKING
Logs must be debuggable per tenant AND scrubbed of cross-tenant contamination.

Design:
• Log structure with tenant_id as a top-level field
• What gets logged (metadata) vs what doesn't (content)
• How to handle logs during incidents (do you still see content? under what circumstances?)
• How to purge a tenant's logs if they offboard or request deletion

## PART 6: THE DISASTER RECOVERY QUESTIONS
Multi-tenancy makes DR harder. Walk through:
• What happens if a Managed Agent session for tenant A crashes mid-run — can it be retried without tenant-B side effects?
• What happens if a Vault rotation corrupts credentials for one tenant — blast radius?
• What happens if Anthropic has a regional outage affecting some tenants but not others?

## PART 7: THE AUDIT YOU'LL NEED LATER
If one of your tenants is SOC 2 / HIPAA / GDPR-regulated, they'll ask for evidence of isolation. Produce:
• The evidence artefacts (config diffs, log samples, test results)
• The narrative description for the auditor
• The known gaps (there will be some — better to name them than hide them)

I'm producing a platform-selection teardown for a client. They're deciding between Anthropic, OpenAI, Amazon Bedrock, and Google Vertex AI for [WORKLOAD].

CLIENT CONTEXT:
• Company type and size: [description]
• Existing cloud commitments: [AWS / GCP / Azure / multi / none]
• Existing data stack: [where data lives, what's sensitive]
• Compliance requirements: [SOC 2, HIPAA, GDPR, etc.]
• AI team maturity: [greenfield / some prototypes / ML-engineering team / full ML platform team]
• Budget sensitivity: [startup / mid-market / enterprise with no hard cap]

WORKLOAD:
[One paragraph. Example: "A customer-service copilot that reads tickets, fetches knowledge-base articles, drafts responses, escalates to humans when confidence is low. Expected 50,000 interactions/day at steady state."]

TASK:

## PART 1: EVALUATION FRAMEWORK
Produce the evaluation rubric, customised to the client. Dimensions should include (at least):
• Model quality on the specific task
• Total cost at projected scale (include surprising costs — inference, vector DB, data egress)
• Time to production (the "second job" of infrastructure)
• Ecosystem fit with existing stack
• Data-residency and compliance
• Vendor lock-in risk
• Roadmap alignment (what's each vendor likely to ship over the next 18 months?)
• Operational concerns (observability, debugging, incident support)

Weight each dimension based on the client's context. Justify the weights.

## PART 2: SIDE-BY-SIDE COMPARISON
For each of the four platforms, score against the rubric. Per dimension:
• Anthropic Platform
• OpenAI Platform
• Amazon Bedrock
• Google Vertex AI

Scoring: 5-point scale with concrete justification. No vendor gets a score without a specific reason.

Produce as a table, with a summary paragraph after highlighting the signal in the noise.

## PART 3: BENCHMARK ON THE CLIENT'S ACTUAL TASK
A design for running the client's workload on each platform for a bounded period:
• Benchmark task (narrowed version of their real task)
• Eval criteria (what does 'good' look like?)
• Test set size and composition
• Instrumentation (quality score, cost, latency per platform)
• Execution plan (who, when, how much)

## PART 4: THE REAL-MONEY COST MODEL
For each platform, produce the monthly cost at projected scale. Include:
• Per-token inference cost
• Batch API discounts (where available)
• Caching discounts (where available)
• Infrastructure cost (cloud resources beyond the model API)
• Support contract cost (if applicable)
• Hidden costs (egress, cross-region, premium features)

Plot best / likely / worst cases. Call out which costs are capped (budget caps) vs uncapped.

## PART 5: BUILD-VS-BUY ON THE ANTHROPIC SIDE
Given the client's profile, specifically address:
• Managed Agents vs building agent infra on any platform: worth it for this client?
• Anthropic-hosted MCP connectors vs building integrations: worth it?
• Workbench / Eval Tool vs rolling your own: worth it?

For clients with mature ML teams, the answer skews toward build. For greenfield clients, it skews toward Anthropic's managed surfaces. Where does THIS client sit?

## PART 6: THE 18-MONTH OUTLOOK
Based on each vendor's announced roadmap and current trajectory:
• What's Anthropic likely to ship that affects this decision?
• What's OpenAI?
• What's Bedrock/Vertex's story?

Recommend the decision that ages well over 18 months, not just the one that looks best today.

## PART 7: THE RECOMMENDATION
One page. Clear recommendation. Three scenarios under which the recommendation flips. A one-paragraph executive summary the CEO can read and act on.

## PART 8: WHAT WE'RE WRONG ABOUT
Where are you least confident in this analysis? What data would change the recommendation? This matters — clients value intellectual honesty more than confident wrongness.

I want to set up an eval-gated deployment pipeline for prompts. Today, prompt changes are deployed with the same ceremony as... no ceremony. I want every prompt change to run through evals before it reaches production.

STACK:
• Prompt storage: [location — git repo path, Workbench, config service]
• Eval Tool: [platform.claude.com Eval Tool]
• CI: [GitHub Actions / GitLab / CircleCI / other]
• Deployment target: [what consumes the prompt in production]
• Team size: [solo / small team / larger team with review requirements]

TASK:

## PART 1: THE DESIRED WORKFLOW
End-to-end, what should happen when someone changes a prompt:

1. Author changes the prompt in [where]
2. Opens a PR with the change
3. CI runs: [what evals? what thresholds?]
4. If pass, [what? auto-merge? still need human review?]
5. If fail, [what? the author sees what artefacts?]
6. On merge, deployment happens [how? immediately? staged rollout?]
7. In production, [what's monitored? rollback triggers?]

Produce this as a clear flowchart in text form.

## PART 2: THE GIT STRUCTURE
Prompts in a repo. Show:
• Directory layout (one file per prompt? directory per domain? how to organise)
• File format (YAML / JSON / markdown with frontmatter / raw)
• Accompanying files (eval suite, expected outputs, metadata)
• Versioning (semver? timestamped? git-sha-derived?)

Produce the example directory tree with actual file contents.

## PART 3: THE CI PIPELINE
The GitHub Actions workflow (or equivalent) that:
• Triggers on PR with changes to prompts/
• Extracts the changed prompts
• Runs each against its linked eval suite via the Eval Tool API
• Posts results as a PR comment (score, diffs vs main branch, regressions flagged)
• Fails the PR if scores drop or hard assertions fail

Produce the actual workflow YAML.

## PART 4: THE PROMPT-SCORE DIFF
The most useful PR comment isn't "score: 4.2". It's "new: 4.2, main: 4.0, delta: +0.2, regressions: 2 cases, improvements: 7 cases".

Produce the comment template and the script that generates it. Include:
• Per-case comparison (old score vs new score per test case)
• Regression highlights (cases where score dropped)
• New capabilities (cases that were failing and now pass)
• Cost impact (token count old vs new)

## PART 5: THE DEPLOYMENT STAGING
On merge:
• Stage 1: deploy to a shadow environment that mirrors production but doesn't affect users. Run the eval suite again against the live infra.
• Stage 2: 1% canary to production. Monitor for [what signals]?
• Stage 3: full rollout.

Produce the staging config and the rollback conditions for each stage.

## PART 6: THE DRIFT DETECTION
Once deployed, prompts don't stay "correct" — the world shifts under them.

Design:
• What to monitor in production (quality signals, not just cost)
• What triggers a "re-eval" (user-reported issue, model update, input distribution shift)
• How often to run the eval suite against live traffic samples

## PART 7: THE GUARDRAIL FOR THE HERO-DEVELOPER
One engineer bypasses CI in an incident ("just ship the fix"). Six months later, nobody remembers that commit. The eval never ran. The prompt is now broken in ways nobody sees.

Design the guardrail:
• How to allow emergency overrides without permanent damage
• Retroactive eval runs on bypassed merges
• Audit log of who bypassed what and why

## PART 8: WHAT THIS DOESN'T CATCH
Honest assessment: eval-gated deployment catches a lot, but not everything. List the failures this pipeline won't catch — so you know where to invest manual review.

I'm building a multi-agent workflow. A single agent is degrading — too many responsibilities, too much context, too many decision branches. I need orchestration.

THE WORKFLOW:
[Description. Example: "A loan-application evaluator that ingests an application packet, checks it against rules, requests missing documents from the applicant, runs background checks, synthesises a recommendation, and generates a compliance-ready report."]

CURRENT STATE:
[What's working, what isn't, on the single-agent version. Example: "Prompt is 4,500 tokens. Accuracy is 78%. Gets confused between fraud detection and completeness checking. Background-check tool calls sometimes fire in parallel incorrectly."]

TASK:

## PART 1: THE DECOMPOSITION
Break the workflow into discrete agents:
• Role name
• Sole responsibility (one sentence)
• Inputs it needs
• Outputs it produces
• Model choice (Haiku for simple, Sonnet for balanced, Opus for judgement)

Rules:
• Each agent does one thing.
• If an agent's description includes "and", split it.
• If an agent needs more than 1,500 tokens of system prompt, it's doing too much.

Produce the agent roster.

## PART 2: THE ORCHESTRATION PATTERN
Pick one. Justify.

OPTION A — ORCHESTRATOR + WORKERS
One agent is the orchestrator; others are specialists it dispatches to. Orchestrator holds the overall plan; workers return structured outputs.

OPTION B — PIPELINE
Agents chained end-to-end. Output of A feeds B feeds C. No orchestrator.

OPTION C — EVENT-DRIVEN
Agents react to shared state or messages. No central coordinator. Decoupled but harder to reason about.

OPTION D — HIERARCHICAL
Sub-orchestrators coordinate their own sub-agents. Good for very complex workflows.

For my workflow: which pattern? Why?

## PART 3: THE MESSAGE SCHEMA
Multi-agent systems live or die on the message contract between agents.

Produce:
• The structured format (JSON schema) each agent receives and returns
• Error types (including 'I can't complete this' — not just unhandled exceptions)
• Metadata (which agent, which step, which session, latency, cost)
• Versioning (how does Agent A know which version of Agent B it's talking to?)

## PART 4: THE SHARED STATE
Agents need to share something — task state, intermediate results, tool outputs.

Design:
• Storage (Managed Agents memory beta? External state store? Event history?)
• Consistency model (eventual? strong? per-agent local?)
• Who can write what
• Cleanup and TTL

If the research-preview multiagent feature fits, use it; explain how. If not, design the alternative.

## PART 5: THE FAILURE MODES
Multi-agent failures are weirder than single-agent. Walk through:
• One agent times out — does the whole workflow block, proceed degraded, or retry?
• Two agents disagree — who arbitrates?
• The orchestrator goes down mid-workflow — how is state recovered?
• A tool a sub-agent relies on becomes unavailable — local retry or escalation?
• The model returns invalid structured output — the retry/fix protocol

Produce the failure-handling playbook.

## PART 6: THE COST MULTIPLIER
N agents means N × base cost, plus coordination overhead. Model this:
• Expected cost per workflow run
• Where advisor routing can cut costs
• Whether any step could be non-LLM code instead (big wins here)
• When to fall back to single-agent for simple cases

## PART 7: THE OBSERVABILITY STACK
Debugging a single-agent run is annoying. Multi-agent is a distributed-systems problem.

Design:
• Per-agent trace with parent-child relationships
• End-to-end visualisation of a workflow (timeline, decision branches, tool calls)
• How to find the agent that caused a bad outcome when there are 5 in the chain
• What to log at the orchestration layer vs the agent layer

## PART 8: THE WORST MULTI-AGENT DESIGN DECISION
Reviewing the plan: where's the riskiest architectural choice you made? What would be the early warning sign that it was wrong? What's the fallback?

I want to replace fragile text-parsing with proper structured outputs on [PROMPT/AGENT].

CURRENT STATE:
[Description. Example: "Our agent returns a recommendation as a paragraph. Downstream, a regex extracts the decision, another regex the confidence, a third the reasons. About 3% of outputs break parsing; 1% are silent wrong parses."]

TARGET STATE:
[What downstream expects. Example: "A typed object: { decision: 'approve' | 'reject' | 'review', confidence: 0-1, primary_reasons: string[], red_flags: string[], recommended_next_actions: string[] }"]

TASK:

## PART 1: THE SCHEMA
Produce the JSON schema. Include:
• Every field, typed (no vague 'string'; use enums where possible)
• Required vs optional
• Constraints (min/max, patterns, lengths)
• Descriptions for each field (these become the LLM's spec — write them for an LLM reader)

## PART 2: THE PROMPT REWRITE
Rewrite the prompt to use structured outputs:
• The schema attached via the Messages API structured_output parameter
• The system prompt updated to reference the schema structurally
• Examples showing the exact output shape
• Failure modes and what to return in each

## PART 3: THE DOWNSTREAM CONTRACT
Now that the prompt returns typed data:
• The consumer code change (ditch the regex; type-check the response)
• The validation layer (schema-valid but semantically wrong outputs — how to catch)
• The backward-compatibility plan if this prompt has existing callers

## PART 4: THE ERROR HANDLING
Structured outputs can still fail:
• Model returns invalid JSON (rare but happens) — retry with error in context?
• Model returns valid JSON but wrong values (hallucinated enum, out-of-range number)
• Schema-valid but useless (all fields set to generic defaults)

For each, the detection and handling code.

## PART 5: THE TEST SUITE
Tests that exercise:
• Happy path (valid, useful output)
• Schema-valid but semantic failure (e.g., 'decision: approve' but reasons contradict)
• Ambiguous inputs (model should still produce structured output, maybe with lower confidence)
• Adversarial inputs (should still produce structured output, not free-form text)

## PART 6: THE LATENCY AND COST IMPACT
Structured outputs sometimes add tokens (the schema gets tokenised). Sometimes save tokens (no prose explanation). For MY prompt, estimate:
• Token count before / after
• Cost delta per call
• Latency impact (minimal, but measure)

## PART 7: WHEN NOT TO USE STRUCTURED OUTPUTS
Honest assessment. Structured outputs aren't always right:
• Open-ended generation (blog posts, long-form content)
• Outputs meant for human consumption with no machine parsing
• Cases where forcing structure degrades quality

For MY prompt, is structured output actually the right move? (If no, say so. Don't force it.)

I have a research pipeline that processes [DATASET] with a per-record LLM call. Currently it runs synchronously and costs too much. I want to move it to Batch API.

CURRENT STATE:
• Dataset: [N records of type X]
• Per-record prompt: [attach or describe]
• Current execution: [sync, via Messages API, takes [T] hours, costs $[X]]
• Use case: [one-off research / recurring job / on-demand but ad-hoc]

TASK:

## PART 1: BATCH ELIGIBILITY
Is this actually right for Batch API? Check:
• Can I wait 24 hours? (Some jobs can't.)
• Are records independent? (If record N depends on record N-1, batch doesn't help.)
• Is the dataset large enough? (Under ~1,000 records, the ops overhead often isn't worth it.)
• Do I need intermediate results? (Batch is all-or-nothing for the output.)

If any answer is 'no' — say so, and suggest alternatives (prompt caching, parallel async calls, Advisor routing).

## PART 2: THE BATCH FILE
The Batch API accepts JSONL with specific structure.

Produce:
• The exact line format for my prompt (custom_id, method, url, body with model/messages)
• The script that generates batch files from my dataset
• How to split into multiple batches if dataset exceeds single-batch limits
• Validation (malformed lines will fail the whole batch, sometimes silently)

## PART 3: THE SUBMISSION AND POLLING
Code to:
• Upload the batch file
• Submit the batch job
• Poll for completion (or use webhook if available)
• Download the output file
• Parse and match outputs back to inputs by custom_id

## PART 4: THE PARTIAL-FAILURE HANDLING
Batch jobs can have per-record failures. Design:
• How to detect failed records in the output
• Retry policy for failed records (re-batch? sync retry? abandon?)
• How to avoid losing the successful ~99% if the batch partially fails

## PART 5: THE COST MODEL
Compared to sync:
• Batch discount: 50% of standard token price
• But: batch jobs can fail partially, requiring reruns (which cost again)
• Storage costs for input/output files
• Engineering time to build batch pipeline vs one-time sync run

For MY dataset and job, the real savings: $[X] sync vs $[Y] batch = $[Z] savings. Break-even on engineering effort at [N] runs.

## PART 6: THE RUNBOOK
Operational doc for the humans who'll actually run this:
• How to submit a new batch
• How to monitor progress
• What 'stuck' looks like and how to unstick it
• How to reprocess a subset without resubmitting the whole dataset
• Cost projections for common dataset sizes

## PART 7: THE EDGE CASE THAT WILL BREAK YOUR FIRST BATCH
Based on typical first-batch mistakes: where will YOUR batch likely fail? (Encoding issues? Token-length overruns on outlier records? API response size limits? Something else?)

I want to build a library of 10 Skills for my team to standardise our common workflows. Today, everyone prompts Claude their own way; outputs are inconsistent; new hires re-invent the wheel.

TEAM CONTEXT:
• Size and roles: [description]
• Most-used Claude workflows: [list the 5-10 tasks people do weekly]
• Current pain points: [inconsistency? re-explaining context? output quality variation?]

TASK:

## PART 1: SKILL INVENTORY
From my workflow list, identify the 10 Skills worth building. For each:
• Skill name (kebab-case)
• One-sentence description (this is the trigger for Claude to load it)
• Who on the team uses it, how often
• What inconsistency it fixes

Rank by ROI (time saved × frequency × number of users).

## PART 2: THE 10 SKILL.MD FILES
For each of the 10, produce the full SKILL.md:

---
name: [skill-name]
description: [when to use]
---

# [Title]

## Purpose
## When to Use
## When NOT to Use
## Instructions
## Examples
## Output Format
## Failure Handling

Rules for each:
• Self-contained (don't assume context from other Skills)
• Composable (should work alongside other Skills, not fight with them)
• Opinionated (a vague Skill produces generic output)
• Tested (include 2-3 example I/O pairs per Skill)

## PART 3: THE DISTRIBUTION PLAN
Custom Skills are per-user on Pro/Max. For team adoption:
• Store canonical SKILL.md files in [a shared repo / Drive folder / Notion]
• Onboarding doc for new team members: how to install each Skill
• Update protocol: when a Skill changes, how does the team know?
• Version tracking (even informal — date + short changelog per Skill)

## PART 4: THE USAGE GUIDANCE
A one-page 'how to use our Skills' doc for the team:
• Which Skill for which task (decision tree)
• What to do when two Skills could apply
• What to do when no Skill fits (when to write a new one vs prompt freeform)
• When to bypass Skills (highly creative work, novel tasks)

## PART 5: THE QUALITY RUBRIC
Per-Skill quality metric: how do you know a Skill is working?
• Output consistency score (multiple runs, similar inputs, measure drift)
• Team adoption rate (are people actually using it, or reverting to raw prompts?)
• Task-outcome quality (did the output actually help the downstream work?)

## PART 6: THE SKILLS YOU SHOULDN'T HAVE BUILT
Reviewing your 10: which one is weakest? Which is most likely to be abandoned? Which would be better as a tool or a template instead of a Skill?

Be honest. Better to cut 2 and ship 8 strong ones.

## PART 7: THE EVOLUTION PLAN
In 3 months, your Skills library has either grown to 20 or shrunk to 5. Which is it for this team? What's the mechanism that makes the right one happen?

I have a client pitch coming up. I want to build a working demo that shows them an agent solving one of their real problems. I have less than a day.

CLIENT CONTEXT:
• Industry: [description]
• Specific pain point they mentioned: [quote them if possible]
• Current tooling: [what they use today that the agent would augment or replace]
• Demo audience: [technical / mixed / fully non-technical]

TASK:

## PART 1: THE DEMO NARROWING
What's the MINIMUM demo that shows the value? Resist scope creep.

Narrow to:
• A single workflow step that's currently painful for them
• A happy-path input (their real data if possible, sanitised if not)
• A visible output they'll immediately understand as better-than-today

Anti-pattern: showing end-to-end automation. The demo should show one decisive moment where Claude produces something that makes them say 'oh'.

## PART 2: THE AGENT DESIGN
For the narrowed demo, produce:
• The agent definition (model, system prompt, tools)
• The minimum tool surface (2-3 tools max — more is distraction)
• The environment config (keep it simple; pre-installed packages only)
• The session launch code

Write it to be readable in a live demo — skip abstractions, skip configs that don't matter.

## PART 3: THE DATA
The demo needs data to operate on. Options:
• Client's real data (ideal but requires legal/privacy clearance before the meeting)
• Sanitised sample of their data (good — strip identifying info, keep structure)
• Public data that looks like their domain (okay fallback)
• Synthetic data you generate (last resort — looks fake, undermines trust)

Recommend one; produce the data file if synthetic or sanitised.

## PART 4: THE SCRIPT
The actual flow of the live demo. 15 minutes max. Written as a script:
• Setup (30 seconds — what's on screen, what the client is looking at)
• Show the problem (2 minutes — what's currently painful)
• Run the agent (3-5 minutes — live, not recorded)
• Show the output (2-3 minutes — what Claude produced, why it's good)
• The ask (2-3 minutes — what you want from them next)
• Q&A (remaining time)

Include the exact words you'll say at each step. Not because you'll read them, but because writing them forces precision.

## PART 5: THE FALLBACK PLAN
Live demos fail. Prepare:
• If the agent errors mid-run: what do you say? What's the recovery move?
• If the output is weird: is there a 'here's what we've seen in testing' backup?
• If network is flaky: do you have a recorded version ready?
• If a demo-specific thing breaks (auth token, data file): what's the 30-second fix?

## PART 6: THE POST-DEMO ARTIFACTS
After the demo, what do you leave behind?
• A one-page summary (not the demo script — a take-away doc)
• A suggested next-step (pilot scope, not full implementation)
• The agent config itself (optional — depends on commercial stage)

Produce the one-pager.

## PART 7: WHAT YOU CAN'T LET THEM SEE
The gap between a demo and a production agent is real. Know what you're showing vs what's glossed over:
• Edge cases you're not handling
• Scale behaviour you haven't tested
• Auth patterns that work in demo but wouldn't in their enterprise
• Cost at real volume

If they ask, have answers. Don't pretend gaps aren't there.

I have a prompt that's working but not great. I want to run it through Prompt Improver, but I've learned that one-shot 'improve this' often over-engineers. Help me use it as a critique loop.

EXISTING PROMPT:
[Paste your current system prompt and user template.]

WHAT'S NOT GREAT:
[Be specific. Example: "Outputs are verbose when they should be terse. Occasionally misses the 'uncertain' flag. Sometimes hallucinates a field that isn't in the source."]

TASK:

## PART 1: THE DIAGNOSTIC BEFORE THE IMPROVEMENT
Before running Prompt Improver, diagnose: what's actually wrong?
• Is the task specification unclear?
• Is the output format under-specified?
• Are the examples mis-calibrated?
• Is the model choice wrong (too weak / too strong for the task)?
• Is the input itself ambiguous?

Prompt Improver fixes some of these; not others. Name the likely root cause. If Prompt Improver is the wrong tool, say so.

## PART 2: THE TARGETED PROMPT-IMPROVER RUN
If Prompt Improver IS the right move, don't feed it the whole prompt with vague feedback. Structure the input:
• The current prompt
• Specific feedback: "summaries are too basic for expert audiences" or "doesn't follow the specified JSON structure"
• Examples of failures (concrete inputs/outputs where the current prompt went wrong)

Produce the exact input you should feed Prompt Improver.

## PART 3: THE OUTPUT REVIEW
Prompt Improver's output is a first draft. Review it for:
• Length creep (did it add 30% more tokens for marginal gain?)
• Chain-of-thought bloat (useful for some tasks, useless for others)
• XML tag explosion (structure is good; over-structure is noise)
• Voice drift (did it lose your prompt's personality in favour of corporate-neutral?)

Produce a decision tree: what to keep from the improver's output, what to revert.

## PART 4: THE EVAL DIFF
Before committing the 'improved' prompt:
• Run both versions through your Eval Tool suite
• Compare scores per test case
• Flag regressions (where new is worse than old)
• Flag improvements (new wins)
• Flag non-changes (neither version handles X — unaddressed by Improver)

Which cases justify keeping the old version partially? Which justify the new one?

## PART 5: THE MERGE STRATEGY
Often the right move isn't 'keep old' or 'keep new' — it's merge. Take the Improver's best moves, discard the cruft, graft onto the existing structure.

Produce the merged prompt. Explain each change — why it's in, or why the Improver's version was rejected.

## PART 6: THE ITERATIVE LOOP
The loop:
1. Identify specific failure modes (not general 'improve this')
2. Run Prompt Improver with targeted feedback
3. Review for over-engineering
4. Eval-diff against the current version
5. Merge the keepers; discard the rest
6. Commit the incremental change
7. Next loop: what's the NEW weakest point?

When does the loop stop? When the weakest point is no longer about the prompt — it's about the model, the input, the task itself, or you're chasing a fractional gain that doesn't matter. Know when to stop.

## PART 7: WHAT PROMPT IMPROVER WON'T FIX
Be honest:
• Fundamentally wrong task framing
• Bad examples (garbage examples → garbage improvements)
• Wrong model for the task
• Ambiguity in the input that no prompt can resolve
• Mission creep (a prompt trying to do too much)

Does my prompt have any of these? If yes, Prompt Improver is treating symptoms. Point me at the root cause.

My agent calls external tools. Sometimes the tools fail. Today, the agent either gives up, lies about the outcome, or loops. I want a proper retry and fallback pattern.

AGENT CONTEXT:
• Agent purpose: [description]
• Tools it uses: [list with brief description]
• Most common failure modes: [what you've observed]
• Error budget: [how much retry-induced latency is acceptable]

TASK:

## PART 1: THE FAILURE TAXONOMY
Categorise tool failures:
• TRANSIENT — will succeed if retried (network blip, 5xx, timeout)
• RATE-LIMITED — will succeed if retried after a delay
• AUTH — will succeed only after re-authentication
• VALIDATION — arg is wrong; retry with same args will fail the same way
• PERMANENT — the resource doesn't exist, the operation isn't allowed
• DEGRADED — tool returned something, but the something is wrong/incomplete
• UPSTREAM-OUTAGE — the service is down entirely

For each tool my agent uses, map which of these can happen.

## PART 2: THE RETRY POLICY PER CATEGORY
For each failure type:

TRANSIENT: retry [N] times with exponential backoff. If still failing, escalate as UPSTREAM-OUTAGE.

RATE-LIMITED: respect the Retry-After header if present; otherwise, exponential backoff. Log for capacity planning.

AUTH: attempt one re-authentication; if that fails, halt and escalate to human.

VALIDATION: do NOT retry. Report the validation error back to Claude in the tool_result, let Claude re-plan the call.

PERMANENT: do NOT retry. Report back to Claude; Claude should accept this as information and continue without this data.

DEGRADED: depends on the tool. If validation catches it, report; if it doesn't, this is the silent killer — we'll handle separately.

UPSTREAM-OUTAGE: attempt [fallback strategy if one exists]; otherwise halt gracefully with a useful partial result.

Produce the retry-decision code for each category.

## PART 3: THE CLAUDE-FACING ERROR MESSAGE
When a tool fails, Claude reads the tool_result. How you phrase the error affects what Claude does next.

Good error messages to Claude:
• State what failed (which tool, which params)
• State why (in plain language, not stack traces)
• Suggest a next action when possible ("the customer_id format was wrong; try without the prefix")
• Don't encourage infinite retry ("the API is permanently deprecated — stop trying")

For each failure category, produce the error message template.

## PART 4: THE FALLBACK TOOL PATTERN
For each critical tool, design a fallback:
• Primary tool fails → what secondary tool does Claude try?
• Secondary fails → graceful degradation (partial result, caveat in output)
• All fallbacks fail → halt with structured error

Example: 'address validation API down → fallback to a regex sanity-check → if that fails, return the address as-provided with a flag'.

## PART 5: THE LOOP DETECTION
Agents that retry-loop are a classic cost and time sink. Detect and halt:
• Same tool + same args + same error 3 times in a row → halt and escalate
• Total tool calls per session > [N] → halt
• Total retry-induced latency per session > [T] → halt
• Claude re-planning the same approach twice → warn that planning isn't converging

Produce the wrapper code that enforces this.

## PART 6: THE DEGRADED-RESPONSE HANDLING
Tools that return 'success' but garbage are the worst:
• Empty response with 200 status
• Malformed data that passes schema validation
• Stale data (return from cache when fresh was needed)

For each tool, what's the 'sanity check' on the response before handing it to Claude?

## PART 7: THE HUMAN ESCALATION PATH
When retries exhaust, what happens? Options:
• Halt and log, let the user retry
• Email/page a human on-call
• Queue for later retry (for non-interactive workflows)
• Produce a best-effort partial result with a clear caveat

For MY agent's context, recommend one per failure type.

## PART 8: THE POST-MORTEM ARTIFACT
When a tool-use incident happens, you'll want the logs. Design what to capture:
• The tool calls attempted (name, args, timestamp)
• The responses received (success or failure, with details)
• Claude's reasoning between attempts (if extended thinking enabled)
• The total session cost and duration
• The eventual outcome (success, partial success, halt, escalation)

I need an observability stack for my agent infrastructure. Today, if something goes wrong, my debugging options are 'stare at my code' and 'hope'. I want to fix that.

STACK CONTEXT:
• Agent(s) running: [how many, doing what]
• Current observability: [what you have today; 'nothing' is a valid answer]
• Infrastructure: [Managed Agents / Messages API / hybrid]
• Platform: [Datadog / Honeycomb / cloud-native logging / custom]
• Volume: [requests/day, approximate]

TASK:

## PART 1: THE SIGNAL HIERARCHY
What to capture, at what level of importance:

CRITICAL (must have, from day one)
• Request ID / session ID (end-to-end trace handle)
• Model used
• Input token count
• Output token count
• Cache read tokens / write tokens
• Total cost per request
• Latency (total, time-to-first-token, time-between-tokens)
• Error code (if any)
• Tool calls attempted (name + success/failure)

IMPORTANT (should have by week 2)
• Structured input (selected fields, PII-scrubbed)
• Structured output (same)
• Claude's thinking tokens (if extended thinking on)
• Prompt version / hash
• Agent config version
• Per-tool-call latency and cost

NICE-TO-HAVE (add when you have the headroom)
• Full input/output (with aggressive PII scrubbing)
• Token distribution histogram (for cost spike diagnosis)
• User/tenant metadata
• A/B variant markers

Recommend what my stack should log at each level.

## PART 2: THE TRACE CORRELATION
One user action might cause:
• Your app to make an LLM call
• The LLM to make 3 tool calls
• Each tool call to hit a different backend
• The final response to generate a log line

All of this needs one trace_id. Design:
• Where the trace_id originates (HTTP header, session creation, generated per request)
• How it propagates (into the system prompt? tool-call metadata? OpenTelemetry-style spans?)
• How you query across spans to reconstruct a full session

## PART 3: THE DASHBOARDS
Three dashboards you need:

DAILY HEALTH
• Request volume, error rate, p50/p95/p99 latency, total cost
• Model breakdown (which models, what volume, what cost each)
• Tool success rate per tool
• Cache hit rate

INCIDENT RESPONSE
• For a given session_id: full timeline, every tool call, every error, cost, duration
• Ability to replay (or at least view) the prompt and response
• Correlation to upstream and downstream requests

COST INVESTIGATION
• Cost per agent, per session, per tenant
• Outlier detection (sessions > 3 standard deviations above average cost)
• Trends over time, vs budget

Produce the dashboard spec for each.

## PART 4: THE ALERTS
Alerts should be actionable. For each, define:
• The signal
• The threshold
• Who gets paged
• What the first action is

Proposed alerts:
• Error rate above [X]% over [T] minutes
• Cost burn-rate exceeding monthly-budget-divided-by-days projection
• p95 latency regression
• Cache hit rate dropping below [X]% (suggests cache eviction or prompt change)
• Tool success rate below [X]% for any specific tool
• Loop detection (same session hitting [N] tool calls)

## PART 5: THE PII & COMPLIANCE LAYER
Observability meets data protection:
• What fields MUST be redacted before logging (PII, credentials, sensitive content)
• How to log 'something happened' without logging the content
• Retention policy (7 days for full logs? 90 days for metadata? forever for audit trails?)
• Access controls (who can see full logs vs aggregated metrics)
• Subject-access-request support (if a user asks for their data, can you find and delete it?)

## PART 6: THE INCIDENT-RESPONSE PLAYBOOK
When an alert fires:
• Triage: how do you know it's real vs noise?
• Containment: what stops the bleeding?
• Investigation: how do you find root cause?
• Communication: who needs to know, in what order?
• Recovery: how do you verify the fix?
• Post-mortem: what goes in the doc?

Produce the playbook.

## PART 7: THE 'DEBUG THIS ONE REQUEST' WORKFLOW
A user reports 'this response was wrong'. They give you a session_id (or equivalent).

In under 10 minutes, you should be able to:
• Find the full request
• See the exact prompt Claude received
• See the exact response
• See every tool call and result
• See the Claude reasoning (thinking tokens if enabled)
• See the cost and latency
• See what version of the prompt/agent was live

If your current stack can't do this, Part 7 is your highest priority.

## PART 8: THE COST OF OBSERVABILITY
Observability costs money. Be honest:
• Logs at full verbosity: $[X]/month for your volume
• Traces with full token content: $[Y]/month
• Retention at [N] days: $[Z]/month

Where's the cost/value curve? Where do you cut?

I'm writing a build-vs-buy memo for our agent infrastructure decision. The question: do we build the agent harness ourselves (LangGraph, custom sandboxing, our own state management) or adopt Managed Agents (or similar)?

CONTEXT:
• Team: [size, ML/AI maturity, availability]
• Workload: [one paragraph — the actual agent workload at stake]
• Current state: [what you have today — prototypes, partial production, nothing]
• Timeline pressure: [when does this need to be in production, and what forces that date]
• Control requirements: [data residency, custom model integrations, vendor-agnostic, etc.]

TASK:

## PART 1: THE HONEST SCOPE OF "BUILD"
Building agent infrastructure is not a weekend. Enumerate:
• Sandboxing (secure container execution for untrusted agent code)
• State management (persistence across session interruptions)
• Tool orchestration (dispatch, retry, timeout, logging)
• Credential vaulting (never show keys to the model)
• Event streaming (SSE or equivalent for real-time UX)
• Checkpointing (resume a long-running session after a crash)
• Observability (tracing, logging, cost attribution)
• Multi-tenancy (if applicable)
• Incident response and on-call rotation
• Ongoing maintenance (updates, security patches, new model migrations)

For each: rough engineer-months to build to production quality. Be honest. Add 50% for the 'works in prod but weird edge case every Tuesday' tax.

## PART 2: THE HONEST SCOPE OF "BUY"
Managed Agents (or similar) handles most of Part 1. But not all.

What you still build on top of a managed platform:
• Your domain-specific tools and MCP servers
• Your prompts, skills, eval suite
• Your application integration (UI, webhooks, business logic)
• Your data pipeline feeding the agent
• Your monitoring on the buy-platform's monitoring
• Your fallback for when the buy-platform has an outage

Rough engineer-months for the "still build on top" portion.

## PART 3: THE COST MODEL
Build:
• One-time (engineering to production)
• Ongoing (maintenance, on-call, updates)
• Infrastructure (cloud compute, storage, observability)
• Opportunity cost (what else could the team be building?)

Buy:
• Per-runtime-hour ($0.08 for Managed Agents)
• Token costs (separate)
• Lock-in risk (what if the vendor raises prices or deprecates the product?)
• Migration cost (out, if ever needed)

Compute for MY workload at MY expected volume. Be specific. 'It depends' is not an answer.

## PART 4: THE RISK COMPARISON
Build risks:
• Timeline slip (likely)
• Security bug in sandbox (catastrophic)
• Team attrition (knowledge loss)
• Becoming an infra vendor to yourself (years of opportunity cost)

Buy risks:
• Vendor lock-in (moderate — MCP and Skills are somewhat portable)
• Pricing changes (watch the roadmap)
• Feature gaps (if something the vendor doesn't support is core to your use case)
• Compliance / data-residency gaps

Quantify where possible. Use likelihood × impact.

## PART 5: THE DECISION MATRIX
For each of the following team profiles, who should build vs buy?
• Greenfield startup, small team, fast timeline, no strong platform commitments
• Mid-market with existing ML infra, moderate timeline, multi-vendor strategy
• Enterprise with compliance requirements, long timeline, platform-lock-in tolerance varies
• Hyperscaler with ML platform team, no timeline pressure, build-everything-ourselves culture

For MY team profile, which bucket is closest, and what's the recommendation?

## PART 6: THE HYBRID OPTION
Rarely is it 100% build or 100% buy. Most real answers are:
• Start on managed. Migrate to custom if/when specific needs justify.
• Use managed for production; build a wrapper so you can switch vendors if needed.
• Build the domain-specific parts (tools, prompts); buy the generic parts (runtime, state).

Which hybrid makes sense for my context? What's the switch-out cost if I need to leave?

## PART 7: THE 2-YEAR LOOK
Technology moves. Think out 2 years:
• Will Managed Agents still be the best managed option, or will competitors (OpenAI, AWS) ship equivalents?
• Will agent infra become so commoditised that building is silly? Or so specialised that buying creates ceiling?
• How does your own team's maturity shift the answer over time?

Recommend the decision that's robust to uncertainty — not just the one that looks best today.

## PART 8: THE EXECUTIVE READ
One page. For the CEO/CTO/board. Clear recommendation. The three scenarios under which the recommendation flips. The one question I haven't answered that would change the decision.

## PART 9: THE DECISION DOCUMENT
Not the memo — the actual decision. In writing. With an owner. With a review date.
• Decision (build / buy / hybrid, specifically)
• Why (the 2-3 deciding factors, not the full memo)
• Review trigger (when does this decision get revisited?)
• Owner (whose neck is on the line)

This is what you file in the team's decision log.