How do I build an AI agent that closes email and Slack tickets instead of chatting?

I treat an inbox agent as a worker with a finish line: a ticket goes from open to closed in email or Slack. Search, send, archive, and escalate are four grants, not one mega-tool. The runtime stops when the ticket is gone from the queue or the budget is spent. Fluency in the thread is not a close.

Most inbox software still summarizes the queue. The ticket is still sitting there when the thread ends.

I do not want a comment about the inbox. I want the item gone: answered, archived, or handed to a person with a packet they can act on.

If you cannot point at a field that flipped, you built a chatbot with mail credentials.

I start from the definition in what is an inbox agent, then from the filter in how to auto-triage an inbox. Close is the third job.

Triage labels. Close changes state. Mixing them in one prompt is how a model archives a refund request because the subject looked like noise.

The rest of this piece is the build: a close you can test, four tools with four permissions, guardrails on the writes, one worker across email and Slack, then an eval on frozen threads. The production layer around it is the same discipline as how to put an AI agent in production. Finish line, tools, runtime, score, grants, one desk.

Closed is a state in the queue

A close is a write to the system of record. The customer got an answer, or a person got a packet, and the original item left the queue. A draft in a chat window is not a close: I score the state change, not the essay the model wrote about the ticket.

Write the test before you write the prompt. A script should be able to ask: was a customer-visible reply sent, or was an escalate packet written to a human queue?

Is the original Gmail thread archived, the Slack item off the support channel, the ticket status set to done? If any of those is missing, the run is a miss.

• Done: a reply went out on the same thread, or a person received a packet with thread ids and a next action, and the item is gone from the open queue.
• Not done: a summary in Slack, a draft in a side panel, a label with no send, a reaction emoji, a chat that says it would reply.
• Forbidden close: delete, silent archive with no reply and no escalate, send to a list, refund or legal language the policy did not allow.

Gmail, Slack, Intercom, Zendesk, Linear. The brand of the queue does not change the job.

Somewhere there is a thread id, a status, and a place a human still sits. I bind the agent to those three. I do not bind it to a chat UI.

Silent archive is the miss I care about: the customer is still waiting and the dashboard looks clean. The model reads a messy thread, decides it is done, and files it.

I require a reason code on archive: replied, escalated, or matched a written policy such as bounce or duplicate. No reason code, no archive.

Double-send is the other one. The model retries.

The runtime retries a dropped connection. The customer gets two answers.

Send must be idempotent on thread id plus a hash of the body. If that pair already logged as sent, the tool returns already_sent and does not post again.

The model can continue after already_sent. It cannot post twice.

If only a person can tell the ticket is done, a person will stay in every loop. People stay at the edge: the policy, the grants, a look at the weekly score. They do not click send on every draft.

Draw the path before you open a loop

Most inbox work is a path you can draw in code: classify, fetch, apply a policy, then act. I only open a model-directed loop on the residue I cannot draw. Flexibility costs latency and extra tool calls, so I pay that cost when the next step is genuinely unknown.

Anthropic’s Building effective agents (19 Dec 2024) draws the line I use on every inbox.

That is not marketing. It is an architecture choice. Anthropic’s advice in that post is to find the simplest solution and only increase complexity when needed.

Workflows give you predictability on well-defined tasks. Agents are for cases where you cannot hardcode the path and you still trust the model to pick tools. I apply that to mail and Slack without romance.

A password reset is a workflow. Match the intent, pull the reset link from the identity tool, send the template, archive.

A “where is my order” with a tracking id is a workflow. Fetch the shipment, send the status, archive.

I do not need a loop that shops among thirty tools for those. I need a router and a template.

The residue is the agent. A thread that spans billing and a broken integration.

A Slack huddle that turned into a written complaint. A vendor who changed the SKU names.

You cannot draw the next lookup in advance. The model has to search, read, decide between send and escalate, then stop. That is the loop I actually ship.

Anthropic’s appendix on agents in practice names customer support as a fit because success can be measured through resolutions, and because tools can pull customer data and take actions. I take the measurement part as the constraint.

If I cannot name the resolution, I do not get an agent. I get a chat widget.

Routing is the workflow I put in front of the loop. Classify: known macro, spam or bounce, needs a person, needs the agent.

The known macro never reaches the agent. Spam never reaches send.

Legal and refunds go to escalate without a model vote. How to auto-triage an inbox is that router. Close sits behind it.

Prompt chaining is the other workflow I keep. Draft, then a policy check, then send.

The policy check is code or a cheap model with a tripwire, not a paragraph in the system prompt. If the check fails, the chain stops.

The send tool is never called. That is Anthropic’s gate on an intermediate step, applied to mail.

I open the agent loop only after those gates. The runtime owns retries, memory of what already ran, spend, and stop.

The model picks among the four tools. If the next action is not search, send, archive, or escalate, the run ends and the ticket stays open for a person.

Search, send, archive, escalate

Search, send, archive, and escalate look like one inbox job, and they are four permissions. Search can always be on. Send writes to a customer, archive writes to your queue, and escalate writes to a person, so if those four share one function a bad argument is a public mistake.

The OpenAI Agents SDK Tools guide (2026) is blunt: tools let an agent take actions. Function tools wrap a local function with a JSON schema so the model can call it. The description is what the model reads.

YouTube

Open original

LangGraph’s 7 May 2024 support tutorial names tools one responsibility at a time on a travel assistant, not on Gmail. I still split search, send, archive, and escalate because a shared function turns a bad argument into a public send.

Be exact. One responsibility per tool. I follow that as written.

I declare four function tools, not a generic use_inbox. Hosted web search is the wrong search.

File search against a vector store is the wrong search. The inbox search hits Gmail or Slack or the ticket API by thread id and query. The model should not browse the public web to answer a customer about their own order.

Search is get_thread and search_threads. Arguments are thread id, query, channel, and a time window.

The result is short and true: not found, denied, timeout, or a record. I do not wrap tool output in helpful prose.

The next thought needs facts. OpenAI’s own tool guidance says the same: validate inputs, keep descriptions short, avoid side effects in error handlers.

Send is send_reply. Arguments are thread_id, body, and in_reply_to.

There is no to: field. The tool already knows the thread.

If the model wants a new recipient, that is a different job and it does not have that tool. I have seen “help the customer” turn into a CC on a distribution list. The schema is the fix, not a scolding prompt.

Archive is archive_thread. Arguments are thread_id and reason.

Reason is an enum: replied, escalated, bounce, duplicate, policy_match. Free text reasons are how silent archive sneaks in.

The tool checks that a send or escalate already logged for that thread, unless the reason is bounce or duplicate. Code enforces that. The model does not get a vote.

Escalate is escalate_ticket. Arguments are thread_id and a packet.

The packet is the same shape I use for agent handoff: the original finish line, evidence (thread ids, order ids, not a vibe), the next action for the person, and the limits left. The incoming human should not have to mine a transcript.

Shared memory is a pile. A packet is a decision.

Gmail and Slack map to the same four calls: threads.get and conversations.history are search, messages.send and chat.postMessage on the thread are send. Removing INBOX, or moving the item off the support channel, is archive.

A reaction emoji is not archive. Escalate writes the packet to one human queue. Do not hand the model admin scopes, sendAs, or filter-create.

Timeouts belong on every tool. OpenAI’s function tools accept a per-call timeout.

I want search to fail closed so the model can escalate instead of waiting forever. I want send to fail without retrying the business call.

A dropped connection can retry. A 409 from already_sent cannot.

The runtime knows the difference. The prompt does not.

Build it in four steps

I build the agent in four moves: write the close, split the four tools, put writes behind guardrails, then score frozen tickets. Skip a move and you get a chatbot with API keys. The traces are the work.

I do not start with a fleet. One worker, four tools, one runtime that retries a dropped search, remembers that send already fired, kills the trace at the turn cap, and keeps the tool log.

Split only when search credentials and send credentials should never sit in the same body. Extra agents fail at handoff. A tool that exists only in a notebook is a demo.

Put guardrails on the writes

A prompt that says be careful is not a control. Input checks run before the expensive model, and tool checks run before and after each write. Approval pauses send until a person or a policy says yes, and I put those checks in the runtime, next to the function.

OpenAI Agents SDK Guardrails guide (openai.github.io/openai-agents-js/guides/guardrails) and the Guardrails and human review page on the OpenAI API docs describe three families. Input guardrails run on the initial user input.

Output guardrails run on the final agent output. Tool guardrails wrap each custom function tool, before and after execution. The boundaries matter.

Input guardrails run only for the first agent in a chain. Output guardrails run only for the agent that produces the final output.

If you put a manager in front of a closer, the manager’s input guardrail will not save you when the closer calls send. Tool guardrails run on every function-tool invocation. That is why send and archive wear their own checks.

Execution mode is a real choice. Parallel input guardrails start with the model, which is faster, and the model may already have called a tool if the tripwire later fires.

Blocking mode (runInParallel: false) finishes the check first. If it trips, the expensive model never starts.

For an inbox I block. A jailbreak that reaches send is not a latency problem.

When a guardrail fails it raises a tripwire and the runner stops. Tool guardrails can also rejectContent: skip the tool and hand the model a message, or throwException and halt.

I rejectContent on a send that fails policy so the model can escalate instead. I throw on silent archive and on a send that tries to leave the thread. Those are not recoverable guesses.

Human review is the other control in that same OpenAI guide. Guardrails are automatic.

Approvals pause the run so a person or a policy can allow a side effect. The run records an interruption instead of executing the tool.

You approve or reject, then resume the same run from state. I set needsApproval on send until the class of ticket holds on eval.

The approval is not “read the essay.” The reviewer sees the thread id, the proposed body, the policy result, and the customer of record. Approve posts.

Reject returns the model to escalate or to search again. If review might take hours, serialize the run state and resume later. That is still the same ticket, not a new chat.

• Input, blocking: prompt injection, requests to ignore policy, attempts to extract other customers’ mail.
• Tool input on send: missing thread_id, new recipients, refund or legal language, send-to-list, empty body, secrets in the draft.
• Tool input on archive: no reason code, no prior send or escalate unless bounce or duplicate.
• Tool output on search: redact API keys, tokens, and fields your policy treats as secret.
• Output on the final message: do not let a “done” claim through if the close test failed.

OpenAI is explicit that isEnabled hides a tool from the model and does not replace authorization on the arguments. A send tool can be visible and still be illegal for this thread.

Enforce that inside execute, or in the tool input guardrail, or with needsApproval. I do all three for send. Visibility is not permission.

I do not put hosted computer-use or a local shell on an inbox agent. Those are different jobs.

An inbox closer should not have a browser, a terminal, or a patch tool. Extra surface is how it wanders.

The Tools guide lists those categories so you can pick. I pick function tools with a tight schema.

Treat email and Slack as one inbox

Gmail and Slack are adapters. The job is the same: find the thread, decide, write or hand off, then take it off the queue. Two agents that do not share a close condition will fight, so keep one worker, two transports, and one finish line.

Customers do not stay on one transport. They mail, then ping, then mail again.

If email-bot and slack-bot both try to close the same complaint, you get two answers or two silent archives. I key the ticket on a person plus a topic, and I store the email thread id and the Slack thread ts on that record.

Search can follow either id. Send replies on the channel the customer last used, unless policy says otherwise.

The four tools stay four tools. The execute function has a transport switch: send_reply on Gmail is users.messages.send in-reply-to, and send_reply on Slack is chat.postMessage with thread_ts.

The model does not see the switch. It sees send_reply({ ticket_id, body }).

If you let the model pick raw Slack arguments, it will invent channels. The schema is the ACI Anthropic talks about in that December 2024 post: make the tool hard to misuse.

Anthropic’s appendix on tool design is the part teams skip. They spent more time on tools than on the prompt when they built their SWE-bench agent.

They changed a file tool to require absolute paths because relative paths failed after a cd. Inbox has the same class of bug.

Require a ticket_id the runtime issued. Do not let the model pass a raw email address it scraped from CC.

Escalate has to name the transport the person should use. The packet says “reply on Gmail thread X” or “reply in Slack thread Y,” plus the evidence.

A human who gets a paste of both transcripts will re-ask the customer. That is how close gets undone.

The packet is the baton. Memory can hold the files. The packet holds the next action.

Slack can be faster than mail. I still use one closer.

I change the deadline in the limits, not the tool list. A second “fast Slack agent” with a looser send grant is how you skip eval.

Escalate writes to one human queue with one packet schema. Staffing is yours. The model does not guess who is on today.

Score closed tickets, not replies

I freeze a set of real tickets and run the same jobs more than once. Pass means the ticket left the queue for the right reason. Fail means it chatted, double-sent, or archived silence, and pretty traces that leave the ticket open are zeros.

Anthropic’s Building effective agents post repeats it: the key to success is measuring performance and iterating, and you add complexity only when it demonstrably improves outcomes. I do not ship an inbox closer because the draft sounded like us. I ship it when the close test beats a baseline on a frozen set: last week’s agent, a script, or a careful human.

The set is tickets you already know. Ten is a start and fifty is better, and you write the close for each one before you run.

Include the ugly ones: two transports, a CC’d lawyer, a duplicate, a partial refund you must not promise, a thread that looks like spam and is not. If you change the set every week, the score is noise.

Log every tool call: name, arguments, result, latency, cost, error. Failures hide in the tools.

A model that “closed” by sending a beautiful reply to the wrong thread is a fail you will miss if you only keep the final message. Search-not-found followed by send is a fail.

Send followed by no archive is a fail. Archive with reason replied and no send in the log is a fail.

I score three things in order: did the close condition hold, did any tool trip a policy, then cost and steps as a second score. A cheap miss is still a miss.

A long trace that closed the ticket correctly is a pass you can later shorten. Do not cut the search that was the actual work.

LLM-as-judge is fine for tone on the body that did send. It is a bad only-score.

A judge that grades writing will pass an agent that never archived, and will pass a silent archive that “sounded resolved.” The primary score is the state change plus the reason code. I will not quote a closed-ticket count here.

If you cannot run the set, you do not have a number. You have a story.

Promote one class at a time. Bounces and duplicates can go unsupervised once the archive reason holds.

“Where is my order” with a tracking id can follow. Messy billing stays on approval.

Autonomy is a grant, like send. I do not flip unsupervised on for the whole mailbox because one macro worked on Tuesday.

When the score drops, I read traces, not chats. Tighten the tool, add a routing rule, or put that class back behind approval.

I do not add a second agent to supervise the first. That is how you get two essays and one still-open queue.

Keep the known path as a script

If a rule can close the ticket, write the rule: password resets, known macros, out-of-office, spam. The agent owns the residue: messy threads, missing context, a judgment between send and escalate. Complexity is a cost, and I add it when the script loses.

This is Anthropic’s simplest-solution rule applied to a mailbox. For many applications, optimizing a single LLM call with retrieval is enough.

For many tickets, no model is enough. A Zap that archives bounces is not an insult to the agent. It is how you keep the agent on work that needs a loop.

I keep a written list of tickets the agent is not allowed to close: legal holds, chargebacks, executive mail, and anything that changes money, access, or production. Those route to escalate in code before the model sees them. The model should not have to be “responsible” about a grant it was never given.

What is an inbox agent is the definition. How to auto-triage an inbox is the router in front.

This piece is the closer. How to put an AI agent in production is the layer around it: runtime, eval, grants, one desk.

If you skip the layer, the closer will send. It will just send the wrong thing, twice, on a Friday.

Build the close, split the four tools, guard the writes, and score the frozen set. Then point the worker at one real queue and watch traces until a week is boring. That is an inbox agent, and everything else is a chat window on top of mail.