Harness Engineering for Beginners

Harness engineering is the next phase after "prompt engineering". It consists of developing the supporting systems around LLM models to help develop and push things into production at a faster rate of speed and quality. Below is a beginner's primer into harness engineering; we'll go over:

1. Mental shifts needed to begin understanding how to become a great harness engineer
2. Mental frameworks
3. Drills to help practice becoming better
4. How to know if you're improving

The Mental Shift First (Most Important)

Before drills, you need to rewire how you think about problems.

Old mental model: "How do I solve this?" New mental model: "How do I build a system where this class of problem gets solved automatically?"

The difference is the same as being a player vs. being a coach who designs practice. You're always one level of abstraction higher than the work itself.

Three specific reframes you need to internalize:

1. Mistakes are system design failures, not one-off errors. When an agent (or Claude Code, or a pipeline) produces bad output, your instinct will be to fix the output. Resist this. Ask: what in the environment allowed this mistake to be possible? Then fix that. Every mistake is a signal about a gap in your harness.

2. The agent only knows what's in context. Context is your product. Stop thinking of prompts as instructions. Start thinking of context as infrastructure — something you architect, version, and maintain. A well-designed context is like a well-designed database schema: deliberate, minimal, and structured so queries (agent tasks) return reliable results.

3. Constraints are features, not limitations. In a human-first workflow, strict rules can feel pedantic or constraining. With agents, they become multipliers: once encoded, they apply everywhere at once. The more you constrain the solution space, the faster and more reliably agents work. Counterintuitive but true. OpenAI

The Mental Frameworks

Framework 1: The Four Layers of a Harness

Think of any agent system you build as having four layers, from bottom to top:

• Knowledge — what the agent can access (docs, memory files, structured context)
• Constraints — what the agent is not allowed to do (linters, structural rules, schema enforcement)
• Feedback — how the agent knows if it succeeded (tests, evals, CI, observability)
• Governance — how humans stay in control (approval gates, audit trails, escalation paths)

Every time you debug an agent failure, diagnose which layer broke. This stops you from treating every problem as a one-off prompt fix.

Framework 2: Feed-Forward vs. Feedback

• Feed-forward = giving the agent what it needs before it starts (good docs, clear schemas, constraints that prevent mistakes)
• Feedback = catching and correcting mistakes after they happen (tests, evals, monitoring)

Most beginners only do feedback (re-prompting after bad output). Expert environment designers invest heavily in feed-forward. The goal is catching problems before the agent even has the chance to make them.

Framework 3: "New Employee" Test

Before deploying any agent task, ask: if I hired a brilliant new contractor who had never seen this codebase or project, and I handed them exactly what I'm handing this agent — would they succeed? If no, your harness isn't ready. This mental test reveals documentation gaps, ambiguous instructions, and missing context faster than any other method.

Framework 4: Drift as a First-Class Problem

Systems degrade. Docs go stale. Constraints get bypassed. Your harness will rot if you don't treat drift as something to engineer against proactively — not just fix reactively. OpenAI literally ran background agents to scan for drift and auto-submit cleanup PRs. Build drift detection into your systems from day one, not as an afterthought.

The Drills and Practices

These are concrete, repeatable things you can do starting now.

Daily (15 min): The Mistake Autopsy. Any time an AI tool produces a bad output — wrong format, wrong tone, wrong logic — spend 5 minutes asking "what would have prevented this environmentally?" Write one sentence. Over time, these become harness improvements. Build a running doc called harness-log.md.

Weekly (1 hour): The CLAUDE.md Audit. Review your memory and context files for any project you're actively using Claude Code on. Ask: is anything stale? Is anything missing that would have prevented a mistake this week? Is the file growing too large and unfocused? Refactor it. Treat it like production code, not a note to yourself.

Per project: The "what can go wrong" harness pre-mortem. Before you kick off a new agent task, spend 10 minutes listing the 5 most likely failure modes. For each one, ask: can I prevent this environmentally (feed-forward) or do I need to catch it (feedback)? Then implement at least one prevention before starting.

Monthly: The System Archaeology Drill. Pick one pipeline or agent workflow you built 3–4 weeks ago. Come back to it cold and ask: could a new Claude session pick this up from the context alone? Where does it break down? Fixing those gaps is exactly what OpenAI meant by building for agent legibility.

Ongoing practice: Schema-first thinking. For any data or output your agent produces, define the schema before you write the prompt. Not just "return a JSON object" — specify every field, type, and constraint. Then prompt toward the schema.

How to Know If You're Improving

Leading indicators (you'll see these first):

• Your first-pass agent outputs require fewer rounds of correction
• You can describe why a harness failed in structural terms, not just "the AI gave a bad answer"
• You're writing constraints and context files proactively, not reactively
• Your CLAUDE.md files are getting shorter and more precise over time, not longer and fuzzier

Lagging indicators (you'll see these in 4–8 weeks):

• You can hand a project context to a fresh Claude session and it picks up correctly without you re-explaining anything
• Mistakes stop repeating — the same class of error doesn't happen twice because you encoded a fix
• You spend more time designing than correcting
• Other people can use your systems without asking you how

The clearest single signal: Keep a log of how many correction rounds any agent task requires. If week 1 is 5 corrections per task and week 6 is 1.5 — your harness is getting better. If the number isn't moving, your harness isn't improving, just your prompting is — which is the trap.

Applied to Your Stack

Ask your AI agent (Claude Code, Codex, etc.) the following:

Look at my current project structure, files, and any existing CLAUDE.md. Based on what you can see: (1) where is my harness weakest right now — what context or constraints are missing that would cause you to make mistakes or ask me clarifying questions? (2) what is the single highest-leverage feed-forward improvement I could make today? (3) what one constraint could we encode right now that would prevent my most common class of mistake?