agent-architecture-evaluator

# Agent Architecture Evaluator Version: `1.0.0` ## Overview This skill reviews the architecture of an agent system, not just its prompts or its attached skills. Use it for architectures involving components such as: - planner / executor splits - routers and specialists - tool-use layers - memory systems - human approval gates - multi-agent coordination ## Use this skill when - A user wants to assess an existing agent architecture. - Reliability, latency, cost, or coordination problems appear to be architectural. - A team needs a structured architecture review and optimization roadmap. - You need system-level test scenarios rather than single-skill evals. ## Do not use this skill when - The problem is one isolated skill. - The task is to create a new skill from scratch. - The main need is portfolio review across many related skills. Use `agent-test-measure-refine` or `agent-skill-portfolio-evaluator` in those cases. ## Output contract Always produce these named outputs: - `architecture_inventory` - `failure_mode_map` - `architecture_test_plan` - `optimization_roadmap` - `measurement_plan` - `architecture_recommendation` ## Review dimensions Evaluate at least these dimensions: 1. `component clarity` 2. `routing correctness` 3. `memory usefulness` 4. `coordination reliability` 5. `cost and latency efficiency` 6. `observability and debuggability` ## Quick start 1. Map the current architecture. 2. Identify critical paths and failure-prone handoffs. 3. Define architecture-level test scenarios. 4. Identify bottlenecks in routing, memory, tools, or coordination. 5. Recommend the smallest structural changes with the highest leverage. ## Workflow ### 1. Build the architecture inventory Capture: - components - responsibilities - inputs and outputs - state or memory boundaries - human approval points - observability signals ### 2. Map failure modes Look for: - planner produces unusable tasks - router sends work to the wrong specialist - memory pollutes current decisions - tool calls are slow, redundant, or poorly validated - multi-agent handoffs lose context - approval gates appear too late ### 3. Design system tests Cover: - happy path - degraded upstream input - partial component failure - tool unavailability - stale or noisy memory - high-latency coordination - rollback or recovery behavior See `references/architecture-review-framework-v1.0.0.md`. ### 4. Prioritize architectural changes Prefer: - clarifying responsibilities before adding components - removing weak indirection - tightening interface contracts - adding observability before adding complexity - isolating state when cross-contamination is likely ### 5. Define measurement Recommend concrete metrics where available: - task success rate - retry rate - fallback rate - cost per successful task - latency by stage - human intervention rate ## Anti-patterns - adding new components to hide unclear ownership - keeping weak memory because it sounds sophisticated - optimizing one stage without measuring system impact - blaming prompts for structural routing failures ## Resources - `references/architecture-review-framework-v1.0.0.md` for system review steps. - `references/optimization-patterns-v1.0.0.md` for architecture optimization guidance. - `assets/architecture-review-template.md` for the final report structure. - `assets/example-architecture-review.md` for a realistic filled review. - `assets/architecture-input-example.json` for structured input. - `scripts/render_architecture_review.py` to normalize a structured architecture review into Markdown.