Team Harness Design Checklist

Key takeaways

• This is a scored checklist across eight areas: repo/docs, roles, evaluation, tools, HITL, sandbox boundaries, workflow distribution, and operations.
• Score each area 0 (in people's heads), 1 (partial/inconsistent), or 2 (shared, reusable routine) for a total out of 16.
• Totals map to stages: 0-5 personal habit, 6-11 early practical, 12-16 mature, each with a concrete next action.
• A minimum harness package ships AGENTS.md, docs/, a task contract, hooks, and a provider-setup plugin around a read to implement to verify to update loop.
• Suspect overdesign when rule files grow unread, reviewers rarely catch new failures, or tiny tasks always require the full loop.

Checks added in this revision

• Record harness operator and compute provider separately.
• Distinguish RunState, session state, and snapshots; test loss of each.
• Check whether explicit capabilities replace defaults.
• Separate restricted environment-key exposure from application-key authority.
• Include revoked approvals and interruption after external writes in regression cases.
• Check proxy bypass and secrets in read-only mounts.

Retain evidence from the evaluation loop.

Harness design is less about philosophy and more about removing the bottlenecks agents actually hit.

Quick Scoring

Score each area from 0 to 2.

1. Repository and Docs

• What is the first document the agent should read?
• Is AGENTS.md or an equivalent entry doc short and current?
• Are architecture, domain rules, and release criteria in the repo?
• Are important rules only in chat, Notion, or meetings?
• Are owners and links clear?

2. Roles and Boundaries

• Which roles are actually needed: planner, builder, reviewer, QA?
• Which failure requires role separation?
• Are role inputs and outputs clear?
• Does the reviewer find concrete failure modes instead of giving generic approval?

3. Evaluation and Done Criteria

• What means "done"?
• Are lint, typecheck, build, tests, and browser QA required where relevant?
• Are domain gates defined, such as payment correctness or migration safety?
• Are criteria automated or only written in a prompt?

4. Tool Access

• Can the agent access needed logs, metrics, traces, and browsers?
• Is repository context enough, or does the task require external systems?
• Are UI products being modified without browser verification?

5. HITL and Approval

• Which tasks can proceed automatically?
• Which destructive, deployment, or security-sensitive changes require approval?
• Are approval criteria consistent across people?
• What must be re-verified after approval?
• If auto approval is used, are trust boundaries, block rules, and allow exceptions configured?
• Does denial lead to a safer route instead of a workaround?
• Are high-risk infra, payment, and security changes hard human gates?

6. Sandbox and Integration Boundaries

• Which tools are open by default: shell, filesystem, apply_patch, browser, MCP?
• Are sensitive data and production credentials separated from generated code?
• Is private or on-prem MCP connected without public exposure?
• Do hooks automate validation, logging, and memory creation?
• Are humans reachable for long-running remote approval?
• Are session log, harness, and sandbox separated?
• Do subagent handoffs need separate checks?

7. Workflow Distribution

• Are good team practices distributed as commands, skills, templates, hooks, or plugins?
• Are expert routines still passed by word of mouth?
• Can new team members reach similar baseline quality?
• Can provider setup, API keys, and troubleshooting become plugins?
• Do domain templates include skills, connectors, subagents, and approval flows?

8. Operations and Cleanup

• Is there a stale-doc routine?
• Are unused commands and skills removed regularly?
• Are scaffolding steps reduced when models improve?
• Does an updates log explain why the harness changed?

Score Sheet

Total Score

Minimum Harness Package

Required loop:

1. read AGENTS/docs -> implement -> verify -> update log
2. Add browser QA for UI work.
3. Separate human gates for risky changes.
4. Fix MCP allowlist and sandbox permissions before using external tools.
5. Store trust boundaries and deny rules when auto approval is enabled.

Overdesign Check

Suspect overdesign if two or more are true:

• rule files keep growing and nobody reads them;
• reviewer steps rarely catch new failures;
• tiny tasks always require the full loop;
• there are many commands but no one knows which to use;
• approval exists but criteria are empty.