What is an ops agent, and how is it different from Zapier, n8n, or a runbook?

A ticket lands. Someone has to read it, look up the customer, check the last deploy, write a note, and close it or escalate it. The finish line is a closed ticket with a true note, not a chat that describes what a person should do next.

Most desks already have two answers. One is a Zapier or n8n path: a trigger, a graph, a last step that fires. The other is a runbook: a page a person walks when the pager goes off, and both are honest tools, neither an agent.

I call the third shape an ops agent. I give it the job, a short list of tools, and a way to know it is done. It thinks, calls a tool, reads what came back, and continues until the artifact exists or it hits a limit I set.

If I still click the next step, I built a slow assistant. That is the same loop I use for any AI agent, pointed at tickets, deploys, inboxes, and vendor threads.

An ops agent is a worker on operations jobs. You name the finish line and the tools, and the runtime owns the loop and the stop. Keep a Zap or a runbook when you can draw the path, and use an agent when you cannot and you can still test done.

What an ops agent is

An ops agent is software given an operations job and a short tool list. It plans, calls those tools, reads the result, and stops when an artifact a script can see exists. If the only output is more chat, it is still a chatbot, and if you drew every step in advance, it is a workflow.

I mean a job with a test. “Look at the inbox” is not a job. “PDF in /out/brief.pdf, junk archived, thread left in the right folder” is a job.

“Help with this ticket” is not a job. “Ticket closed with a note that names the customer, the last deploy, and the action taken, or escalated with the same evidence attached” is a job.

The tools are named functions with limits. Search mail is not send mail. Read a ticket is not close a ticket, and open a pull request is not merge to production.

The description of the tool is the prompt. I wrote the contract in AI agent tool calling. The ops cut is stricter because the side effects are tickets, money, and production.

The runtime owns the loop. It retries a dropped connection, remembers what already ran, and caps spend and steps. It stops on success, error, or budget, and people stay at the edge: the goal, the grants, and a look at what shipped.

• A goal you can test. If a script cannot tell the job finished, a person will stay in the loop.
• Tools with names and grants. Search is not send. Read is not deploy. Draft is not the list.
• A loop the runtime owns. Retries, memory, and the stop rule do not live in the prompt.
• A budget for steps and spend. Unbounded loops become invoices.
• A log of every tool call. If you only keep the final message, you will not see the double close.

If the run ends in a paragraph that tells a person what to do, I treated it as a chatbot. That split is the one in AI agent vs chatbot. OpenAI’s agents documentation puts the other half simply: an agent independently accomplishes tasks on behalf of a user.

How Zapier and n8n decide the next step

Zapier and n8n run a path you configured: a trigger fires, then the steps you placed run in order, with branches you drew. An LLM step on that path is still a workflow. The model writes a field, and it does not own the next tool, the retry budget, or the stop.

A Zap starts with an event you named: a form, a new row, a label on a ticket, a file in a folder. Then it walks the nodes you placed: filter, map, create a record, send a message.

The graph is the product. You can read it and change it. A miss is usually a silent skip, a dead branch, or a double send if the trigger is not idempotent.

n8n is the same shape: nodes, credentials, a canvas, a last node that fires. You still drew the path. The engine does not choose a new API because the ticket was weird.

When the cases are few and named, a path is cheaper than a model. Do not spend a model on a path you already know.

Teams then drop an LLM node in the middle and call the whole thing an agent. The model rewrites a subject line, classifies a sentence into a label you already listed, or drafts a reply the next node sends.

The control flow is still yours. Anthropic’s Building effective agents (19 December 2024) calls that a workflow: LLMs and tools orchestrated through predefined code paths.

I use those LLM steps. They are not an ops agent. An ops agent is allowed to choose the next tool based on what the last tool returned, inside a list you still control.

If the ticket is a refund, it may look up the order, then the last charge, then draft a note. If it is a deploy regression, it may read the canary, then the last commit, then open a hold.

You did not draw both graphs in advance. You wrote the finish line and the grants. You fix a path by editing the graph, which is the right repair when the graph is the decision, and the wrong repair when the next action has to wait on the world.

What a runbook still requires

A runbook is a procedure a person follows. It names checks, commands, and who to call, and the human is the loop: they read, decide, type, and sign off. That is useful when judgment is the product, and slow when the same ticket shape repeats and the person is still clicking every tool.

I have walked those pages: check the dashboard, check the last deploy, run the known commands, or page the named person. The document is the memory of the last bad night, written so the next person does not start from zero.

The runbook is not software. It does not call the tools or stop on a budget, because there is no budget, only a tired person. Done is a human signature, which is often the right last step and a bad inner loop.

Put a chatbot on the runbook and you still have a runbook. The model recites the page, and reciting is not acting.

I treat that as a chatbot until it can use tools and stop on a done condition. That is the line in AI agent vs chatbot.

A good runbook is still the source of the finish line. I steal the checks and turn them into tools and tests. Keep the person at the edge when the act is expensive: merge, wire money, send the list, kill a cluster.

Zapier path vs runbook vs ops agent

Three honest shapes sit on an ops desk. The path fires the graph, the runbook asks a person to walk a page, and the ops agent is given a job and tools until the artifact exists or a limit you set. Choose by whether you can draw the path.

Read the table as a choice, not a ladder. A path does not graduate into an agent by adding a model node. A runbook does not become an agent because you pasted it into a prompt.

If you still own every branch, you have a workflow. If a person still owns every click, you have a runbook. If the model chooses tools toward a testable done, you have an ops agent.

I keep all three on a real desk. The Zap writes the row I already understand. The runbook still sits on the expensive acts, and the agent takes the jobs where the first tool’s result decides the second.

Workflows and agents, for operations

Anthropic’s 2024 split is the one I use on ops work. A workflow is a predetermined path for LLMs and tools, and an agent is a model that directs its own tool use. Most ops jobs should stay workflows, and you use an agent when the steps will not sit still and done is still testable.

Anthropic published Building effective agents on 19 December 2024. The authors group workflows and agents under agentic systems. The architecture is not the same.

They tell you to find the simplest solution, and only increase complexity when needed. Workflows offer predictability for well-defined tasks. Agents fit when you need model-driven decision-making and cannot hardcode a fixed path.

YouTube

Open original

LangChain walked the same graph-as-ops-loop idea on 17 Jan 2024.

Their agent is an LLM using tools based on environmental feedback in a loop, with stopping conditions. Ground truth comes from the tools, not from the model’s recap. On an ops desk the environment is the ticket system, the repo, the metrics, the inbox.

They warn about compounding errors and ask for sandboxed testing and guardrails. I take that as eval plus grants. How to put an ops agent in production is the companion to this piece.

When the ticket does not fit the graph

A path is the right tool when the cases are few and named. An ops agent fits when the next action depends on the last tool result, and you cannot list every branch. You still write a finish line, and wandering is a chat with APIs.

Vendor mail is the example I use because it is boring and real. I do not want a summary in a chat. I want a one-page brief in a folder, and the junk archived, which I can test: the file exists, the threads are filed, spend stayed under the cap.

A support ticket that might be a refund, a bug, or a feature ask is the same shape. The first read does not tell you the second tool, so you look up the customer, then the last order or the last deploy, then you write the note and close or escalate.

A path that tries to encode all of that becomes a maze. A runbook that tries to encode all of that becomes a novel the night shift will not finish.

The test I use is crude. Can I list the next ten tickets and, for each, write the sequence of tools before I see the payload? If yes, I draw a path.

If no, and I can still write what done looks like, I write an ops agent. If I cannot write done, I do not have an agent. I have a chat with extra steps.

Anthropic says agents fit problems where it is difficult or impossible to predict the number of steps, and where you cannot hardcode a fixed path. The path can be unknown, but the destination cannot.

“See what you can do with the inbox” is not a destination. The PDF and the archive rule are. If a script with no model would do, write the script.

How the job survives a crash

An ops job can run longer than a request, because it waits on a human, a deploy, or a vendor. If the process dies, a Zap retries the whole path or drops it. Durable execution keeps a history of each step and resumes from the last success, and the runtime owns that, not the model.

A request-response loop is the wrong container for a ticket that needs a second look tomorrow. The process dies, the memory dies with it, and a retry from zero risks a second refund, a second close, or a second page.

Temporal’s durable execution documentation describes the other container. Durable Execution is the guarantee that the application runs to completion despite crashes. If a worker dies, replay rebuilds state from the event history and continues from the last success.

That is the ops case: a refund waiting on a spend grant, a deploy waiting on a canary window, a brief waiting on an attachment overnight. A Zap that retries the whole graph from the trigger is how you double-send.

I need the property, not the brand: completed tool calls stay completed, the wait can last, and a crash is not a new personality. The model should not store that history in the prompt. If a write cannot be idempotent, the runtime needs a dry-run flag or a record of the attempt.

When to use an SDK for the loop

OpenAI Agents SDK is a loop you can borrow: turns, tool execution, guardrails, handoffs, and sessions. Use it when you want that runtime to run, and own the loop when the stop rule, grants, and eval have to live in your system. The SDK brand matters less than whether you can kill a runaway trace.

OpenAI Agents SDK documentation draws a clean line in 2026. Use the Responses API when you want to own the loop, the tool dispatch, and the state.

Use the Agents SDK when you want the SDK to manage turns, tool execution, guardrails, handoffs, or sessions, and when the agent should produce artifacts or operate across multiple coordinated steps. You do not have to choose globally. Many applications use both.

The runner performs the tool loop, switches agents after handoffs, and stops when the run finishes or pauses for approval. That is the ops shape: investigate, hand off, wait on a grant, record the result.

If the job is one model call that classifies a ticket into a label, own the call. If it must investigate, hand to a specialist, pause for a refund approval, and record the result, that is the SDK’s example of a managed run, and the shape of an ops agent.

The question is who owns the stop, the grants, and the log you will read on Thursday. Anthropic’s 2024 note still applies: start close to the API, and do not let a framework hide the prompts and the tool results.

If you cannot see the tool call, you cannot score the agent. If I use an SDK, it has to expose named tools, a pause for approval, a stop, and a trace I can freeze in eval. The agent can draft, and the grant decides whether it may act.

Tools, grants, and the stop

Search is not send, a draft is not the list, and a branch is not production. Each tool has a name, a schema, a timeout, and a grant. The runtime stops on done, on budget, or on a forbidden call, and extra tools are how an ops agent wanders through the company.

I start under ten tools: ticket read, ticket write, customer lookup, deploy read, canary read, file write, mail search, mail archive, and maybe a draft. Each one has a JSON schema, a timeout, and a permission.

Anthropic’s 2024 appendix on tool design is the part teams skip: spend as much care on the tool interface as you would on a human interface. A refund tool that accepts a free-text amount is a hole. A deploy tool that shares a credential with the researcher is a hole.

Grants are how I split a shop without splitting a prompt too early. Read the inbox without send, draft the refund without execute, open the pull request without merge.

One agent with every permission is how a bad step becomes a bad week. I split the body before I split the fleet.

The stop rule is not a suggestion in the system prompt. The runtime kills the trace at the cap, kills a forbidden call, and kills a loop that already wrote the artifact.

Return a short true result from every tool: not found, denied, timeout, or the record. If evidence is missing, the agent calls a tool. It does not invent a recap.

Retrieval is one tool among others. An agent that only retrieves and replies is still close to a chatbot. RAG in front of a runbook is a nicer page, not an ops agent, until something in the world changes and a script can see it.

How I score an ops agent

I do not grade an ops agent on how the thread reads. I freeze jobs I already know, run them more than once, log every tool call, and score the artifact. A clean note on a still-open ticket is a miss, and pretty traces that skip the file are zeros.

The method is the same one I use in how to evaluate AI agents, pointed at operations. Freeze a set of jobs that look like production, write the finish line for each, and run competing traces. The winner produced the artifact.

Log every tool call: input, output, latency, cost, error. Failures hide in the tools. Score pass or fail against the finish line, with cost and steps as second scores.

Keep a baseline: last week’s agent, a Zap, a script, or a careful human. If the new loop does not beat the baseline on the frozen set, it does not sit on the desk. Anthropic is blunt in Building effective agents (2024): add complexity only when it demonstrably improves outcomes.

LLM-as-judge is fine for tone and partial credit. It is a bad only-score. A judge that grades writing will pass an agent that never closed the ticket.

Promote one job at a time. Inbox triage can go unsupervised once it beats the baseline. Deploy might still need a person on the merge.

Autonomy is a grant, like a permission. How to put an ops agent in production is the five-step version of that promotion.

When the Zap or the runbook should stay

Keep the Zap when the path is stable and a miss is cheap to reverse. Keep the runbook when the cost of a wrong act is a bad week, and a person should still hold the baton. Write an ops agent when the cases will not sit still, done is testable, and grants can lock.

A new-order path that writes a row and sends a receipt should stay a path. A nightly export should stay a script. Capture and completion are different jobs.

I will not put an unsupervised agent on first contact with a list. A runbook for killing a cluster, wiring a large refund, or merging a schema change should stay a runbook until the eval is boring and the grant is split.

Drafting the note can move first. The button can wait. The owner designed the runtime, the eval, and the grants, and is not the next token.

If you cannot say what done looks like, do not build an ops agent. Write the finish line on paper, decide whether a path can hit it, and only then give a model tools.

If the only reason you want an agent is the label, keep the Zap. I would rather ship a boring graph that files the row than a clever thread that misses the file.

One operator before a fleet

One ops agent with a short tool list is a worker, and a fleet is a shop. Split only when research should not deploy, or when retrieval, writing, and review fight in one prompt. Extra agents add extra failure at the handoff, so the default on my desk is still one operator.

Teams reach for a fleet because a slide said multi-agent. Then they spend a month debugging which specialist forgot the goal. I wrote that warning in multi-agent orchestration.

Split when one body would hold keys that should never sit together. The researcher who reads the corpus should not hold the deploy token. The agent that drafts the refund should not hold the ledger write.

Those splits are grants first. They become specialists when the tools fight in one prompt or the clocks differ. The outgoing agent is not done until it writes a packet: finish line, evidence as file ids, next action, limits left.

That is the same packet I use in how to hand off an agent to a human. Anthropic’s 2024 guidance is the same rule: add orchestration when the task is genuinely a graph of specialists, not because the slide said multi-agent.

If you are putting an ops agent on a desk, start with one job, one runtime, one score, one week with a person reading the traces. Then take the person out of the inner loop where the score holds.

Expand only after that week is boring. That is production. The rest is a demo with a pager.