Prompt vs Context vs Harness Engineering: Building Better AI Agents

For the last couple of years, a lot of the conversation around building with large language models (LLMs) has centered on prompt engineering. We spent time figuring out how to give the model a role, how to structure instructions, how to provide examples, and how to get a predictable output format.

All of that still matters.

But once you start building something beyond a simple chatbot or a single application programming interface (API) call, you quickly find that the prompt is only one piece of the system.

A diagram I recently came across broke the problem into four areas: prompt engineering, context engineering, harness engineering, and loop engineering. I think it is a useful way to explain how AI applications are evolving, but I would change the relationship between those four concepts slightly.

The important idea is not that prompt engineering is going away. It is that the engineering work around the model is becoming much more important.

That becomes especially obvious when you start building AI systems for network operations (NetOps), security operations, infrastructure, or any environment where the answer has to be based on real operational data and where the system may eventually be allowed to take an action.

What Is Prompt Engineering Actually Telling the Model?

Prompt engineering is the part most people already understand.

We give the model a role, instructions, examples, constraints, and an expected output format.

For example, a network troubleshooting prompt about border gateway protocol (BGP) might look conceptually like this:

You are a senior network engineer.

Review the provided BGP information.

Identify:
- the likely cause
- supporting evidence
- potential impact
- recommended next troubleshooting step

Do not recommend a configuration change unless the evidence supports it.

Return the response as structured JSON.

There is nothing wrong with this approach. In fact, good prompting is still an important part of building reliable AI systems.

The problem comes when we expect the prompt to do work that belongs somewhere else in the architecture.

Suppose I ask the model:

Why is atl-core-r1 experiencing BGP flaps?

A better prompt may improve how the model reasons about the question, but the prompt cannot magically give the model access to the router configuration, interface state, BGP neighbor history, recent change records, topology information, or the troubleshooting runbook my team uses.

At some point, improving the wording of the prompt has diminishing returns.

The next problem is not prompt engineering.

It is context engineering.

What Does Context Engineering Add That Prompt Engineering Cannot?

Context engineering changes the question from:

How do I ask the model this better?

to:

What information does the model need to make this decision?

That distinction matters.

If I am troubleshooting a network problem, I may want to give the model several different types of information:

• the user’s question
• current device configuration
• operational state from pyATS or another collection tool
• topology information
• intended state from NetBox
• relevant runbooks
• previous troubleshooting steps
• recent change records
• incident notes
• results from earlier tool calls

Now we have a very different problem.

We cannot necessarily dump all of that into the model every time. Some of it may be irrelevant. Some of it may be outdated. Some of it may contradict another source. Some of it may consume thousands of tokens without helping answer the question.

The engineering work becomes deciding what to retrieve, what to keep, what to drop, what to summarize, and what should be presented to the model for this particular decision.

This is one reason I think RAG (Retrieval-Augmented Generation) is better understood as a context engineering technique than as a complete AI architecture.

Retrieval-Augmented Generation helps us find relevant information and place it into the model’s working context. That could be a troubleshooting runbook, a configuration standard, a vendor document, or a previous incident.

But retrieval is only one source of context.

A production NetOps assistant may need something closer to this:

User Question
      +
Runbook Retrieval
      +
NetBox Intended State
      +
Current Device State
      +
Recent Changes
      +
Previous Tool Results
      ↓
Context Selection / Curation
      ↓
LLM

That is much more useful than thinking of RAG as simply “put documents in a vector database and ask questions.”

The job is not to give the model more information.

The job is to give the model the right information at the right point in the workflow.

Why Is the Context Window a Constraint, Not the Architecture?

There is another distinction here that I think is important.

We often talk about the model’s context window as though a larger context window automatically solves the problem.

It does not.

A larger context window gives us more capacity, but capacity and relevance are different things.

If I can fit 100,000 tokens into a model call, that does not mean putting 100,000 tokens of configurations, logs, tickets, and documentation into every request is a good architecture.

You still need to decide what matters.

You also have to decide what information from one model interaction should carry forward into the next interaction.

That should not happen automatically.

The application around the model should control what becomes future context. Otherwise, old assumptions, incorrect observations, or unnecessary information can accumulate as the workflow continues.

This brings us to the next layer.

What Is Harness Engineering, and Why Build a System Around the Model?

Harness engineering is where AI development starts looking much more like traditional systems engineering.

The model sits inside a larger application that controls what it can see, what it can do, and how its results are validated.

A basic agent harness might include:

User
  ↓
Agent Harness
  ├── Gather context
  ├── Build prompt
  ├── Call LLM
  ├── Execute approved tools
  ├── Validate results
  ├── Record state
  ├── Apply guardrails
  └── Return response

For a network operations system, that harness might also need to handle:

• NetBox queries
• pyATS execution
• representational state transfer (REST) APIs
• secure shell (SSH) access
• Model Context Protocol (MCP) tools
• structured output validation
• authentication
• permissions
• logging
• tracing
• retries
• timeouts
• model selection
• tool selection
• error handling
• human approval

This is where the difference between a demo and an operational system starts to become clear.

Imagine I give an agent a tool called:

get_bgp_neighbors()

That seems simple enough.

But now there are more questions.

Which routers can the agent query?

What credentials does it use?

Is the tool read-only?

What happens if the device does not respond?

What happens if the command returns incomplete data?

Can the model call the tool ten times?

Do we log the request?

Do we validate the returned structure?

Can the model move from observing BGP state to changing BGP policy?

Those questions are not prompt engineering problems.

They are harness engineering problems.

How Does the Harness Make AI Guardrails Real?

This is also why I am cautious when people describe AI guardrails entirely as prompt instructions.

You can tell a model:

Never make a production network change without approval.

That is useful.

But the stronger control is that the model simply does not have permission to execute a production change without going through an approval mechanism.

Those are two very different levels of protection.

The prompt expresses policy.

The harness enforces policy.

For example, imagine an agent determines that an interface should be shut down.

A safer workflow might look like this:

Agent Recommendation
        ↓
Evidence Validation
        ↓
Change Risk Check
        ↓
Generate Proposed Change
        ↓
Human Approval
        ↓
Execution Tool
        ↓
Post-Change Validation

The model can participate throughout that workflow, but it should not necessarily control every step.

This is an important design principle as we move from AI assistants that answer questions to agents that can take actions.

Where Does Loop Engineering Fit?

The original diagram treats loop engineering as another separate category alongside prompt, context, and harness engineering.

I think the concept is useful, but I would place it slightly differently.

I see loop engineering as part of the harness.

Once an agent can use tools, gather additional information, evaluate results, and try again, we need to control that loop.

A basic agent loop might look like this:

Goal
 ↓
Gather
 ↓
Reason
 ↓
Act
 ↓
Verify
 ↓
Finished?
 ├── Yes → Return result
 └── No  → Gather more evidence and repeat

That sounds straightforward until you ask what happens when the agent never reaches the goal.

This is why loops need engineering around them.

We need things such as:

• maximum iterations
• time limits
• token limits
• cost limits
• duplicate-call detection
• progress detection
• validation tests
• success criteria
• failure criteria
• escalation paths

Without those controls, an agent can continue gathering information or calling the same tool without actually making progress.

Why Isn’t “Try Again” an Agent Strategy?

One of the more interesting controls in the diagram is progress detection.

I think this deserves more attention.

Imagine an agent calls:

get_bgp_neighbor("atl-core-r1")

The command fails.

The agent reasons that it still needs BGP information, so it calls:

get_bgp_neighbor("atl-core-r1")

again.

Then again.

The model is technically reasoning each time, but the system is not making progress.

A properly engineered loop should recognize that the same operation with the same arguments has already failed and change strategy.

Maybe it should:

1. check reachability,
2. query another telemetry source,
3. inspect the monitoring platform,
4. retrieve a recent configuration backup,
5. or escalate the missing data to the operator.

That is much closer to how an experienced engineer troubleshoots.

The important part is not merely allowing the agent to loop.

It is controlling how the agent decides whether another iteration is useful.

How Does a NetOps Troubleshooting Question Move from Prompt to Agentic Workflow?

We can see the progression clearly with a simple troubleshooting question.

Suppose an engineer asks:

Why did BGP flap on atl-core-r1?

What Does the Prompt Engineering Step Look Like?

The first implementation might send the question to an LLM with instructions:

You are a senior network engineer.

Explain likely causes of BGP flapping.

The answer may be technically correct, but it is generic.

The model knows what can cause BGP flaps.

It does not know what caused this BGP flap.

What Does the Context Engineering Step Add?

Now we retrieve:

atl-core-r1 configuration
BGP troubleshooting runbook
recent incident notes
interface logs
recent change records

The answer becomes much more specific because the model has relevant evidence.

What Does the Harness Engineering Step Add?

Next, we let the system gather current information itself.

The agent can:

query NetBox
run pyATS
retrieve monitoring data
search troubleshooting documentation
inspect recent changes

Now we are no longer just asking a model a question.

We have built a system around the model.

What Does the Loop Engineering Step Add?

Finally, suppose the first evidence is inconclusive.

The agent might:

Check BGP state
      ↓
No clear cause
      ↓
Check interface errors
      ↓
Still unclear
      ↓
Check recent changes
      ↓
Policy change discovered
      ↓
Compare intended vs operational state
      ↓
Validate hypothesis
      ↓
Return cited root-cause analysis

That is where the loop becomes valuable.

But it also needs limits.

For example:

Maximum tool iterations: 5
Maximum execution time: 90 seconds
Production changes: prohibited
Required confidence: evidence from at least two sources
If unresolved: escalate to engineer

Now we are starting to build something that can operate responsibly.

How Do These Four Ideas Fit Into One Architecture?

Instead of treating prompt engineering, context engineering, harness engineering, and loop engineering as four competing ideas, I think it is more useful to see how they fit together.

I would visualize the architecture like this:

┌────────────────────────────────────┐
│           AGENT HARNESS            │
│                                    │
│   Goal / Policy / Permissions      │
│                ↓                   │
│        Context Engineering         │
│                ↓                   │
│         Prompt Engineering         │
│                ↓                   │
│               LLM                  │
│            ↙       ↘               │
│         Tools     Sub-agents       │
│            ↘       ↙               │
│           Verification             │
│                ↓                   │
│             Success?               │
│          Yes ↙      ↘ No           │
│            Done     Loop           │
│                       ↓            │
│                Gather More         │
└────────────────────────────────────┘

Prompt engineering lives inside context engineering.

Context engineering lives inside the larger application.

The harness controls the application.

The loop controls how the application continues working toward a goal.

The boundaries are not always perfectly clean, but this model is much closer to what we actually build.

Why Does This Matter for Network Engineers?

I think this shift is particularly important for network engineers getting into AI.

It is easy to look at AI development and assume the primary new skill is learning how to write better prompts.

Prompting matters, but network engineers already have many of the skills needed for the harder parts.

We already think about:

• state
• dependencies
• permissions
• APIs
• failure conditions
• timeouts
• retries
• validation
• observability
• change control
• rollback
• source of truth
• operational state

Those concepts map directly into agentic system design.

A network engineer understands that seeing an interface configured as up in a source of truth does not guarantee that the interface is operationally up.

That same mindset applies to AI.

A model saying an action succeeded is not the same thing as verifying that the action succeeded.

A model saying a configuration looks correct is not the same thing as comparing it against intended state.

A model producing an answer is not the same thing as having enough evidence to trust that answer.

The first word in network automation is still network.

AI does not remove that requirement.

If anything, giving AI systems more autonomy makes domain knowledge, validation, and operational controls even more important.

What Is the Next Phase of AI Engineering?

The model will continue to matter. Better models will give us better reasoning, larger context windows, stronger tool use, and more capable agents.

But I think a lot of the engineering advantage will come from what we build around those models.

How well do we assemble context?

How do we expose tools?

How do we control permissions?

How do we verify what happened?

How do we detect when the system is no longer making progress?

How do we decide when an agent should stop and ask an engineer for help?

Those are systems questions.

And for people coming from networking, infrastructure, security, or automation, that should feel familiar.

The future of practical AI is not just about writing a better prompt.

It is about building a better system around the model.

Recommended Reading

• P.E.N.E.: The Prompt Framework I’m Using to Build AI Agents for Network Operations — a deeper, field-tested framework for the prompt and context layers described here, applied to real NetOps agent workflows.
• AI Network Automation: Where It Helps, Where It Breaks, and Where Humans Still Matter — a practical look at where the harness and human-approval layer actually earns its keep in production network automation.
• “Source of Truth” Is Confusing Network Automation Teams — the intended-vs-operational-state distinction this post leans on when explaining why a model’s answer still needs verification.
• Introducing MCP Servers in Tines — My Hands-On Guide — a hands-on look at MCP (Model Context Protocol) tools, one of the harness building blocks referenced above.

Frequently Asked Questions

What is the difference between prompt engineering and context engineering? Prompt engineering is about how you ask the model something — role, instructions, examples, and output format. Context engineering is about what information the model has available when it answers — retrieved documents, current device state, intended state, and prior tool results. A well-written prompt cannot compensate for missing or irrelevant context.

What is harness engineering in AI agents? Harness engineering is the application layer built around the model: gathering context, calling tools, enforcing permissions, validating results, logging, and applying guardrails. It is what turns a single model call into an operational system, and it is where policies like “no production changes without approval” get enforced rather than just requested.

Is loop engineering the same thing as an agent loop? Not quite. An agent loop is the basic gather-reason-act-verify cycle. Loop engineering is the set of controls placed around that cycle — maximum iterations, time and cost limits, duplicate-call detection, and progress detection — so the agent does not repeat a failed action indefinitely instead of changing strategy.

Why doesn’t a larger context window solve the context engineering problem? A larger context window increases capacity, not relevance. Filling it with every available configuration, log, and document does not help the model reason better, and it does not solve the separate problem of deciding what information should carry forward from one step of a workflow into the next. That decision still has to be engineered deliberately.

Why should network engineers care about AI agent architecture specifically? Network engineers already work with state, permissions, failure conditions, validation, rollback, and the gap between intended and operational state. Those are the same concerns that show up in harness and loop engineering for AI agents, which makes the transition to building or evaluating AI systems more natural than it might first appear.