Before we automated anything, we looked at two years of our company inbox. More than half of it, 54%, was automated notifications: receipts, alerts, newsletters and account notices. The mail that mattered was in there too, and finding it took someone's attention every day.
So we built email triage into our internal operations platform. Every five minutes during the working day, it reads new mail and decides what happens to each message: archive it, mark it read and keep it, or leave it for a person.
The AI model is the smallest part of the system. What matters is the order of the steps, and which steps the model can never override. The lesson for anyone automating an inbox: decide which mistakes you could not undo, and put those decisions in code, not in a prompt.
Check the written rules before asking the AI
The system keeps a table of written rules. Each rule matches on a sender's address, a sender's domain or words in the subject line, and it puts matching mail in one of three tiers:
- Archive: out of the inbox and marked read.
- Keep, marked read: stays in the inbox, with no unread badge.
- Leave alone: stays in the inbox, unread.
When more than one rule matches, the most specific one wins, and a rule for one inbox beats a rule for all of them.
Only a message that no rule covers goes to the AI model. The model reads the sender, subject and body, and returns a tier, a label, a confidence score and a one-sentence reason. Below a confidence of 0.6, nothing changes. If the call to the model fails, the message is left alone as well. The model's instructions include a plain rule: when in doubt, keep the message in front of a person, because archiving something important is far worse than leaving an extra message unread.
If you are automating an inbox, start with rules. They need no model call, and anyone on the team can read them. Then tell the model which way to fail. A model that is unsure should change nothing.
Let the AI write rules only with evidence
When the model is very sure about a message, with a confidence of 0.85 or more, its verdict can become a new rule. The next message from that sender then skips the model. We put two limits on this.
First, a learned rule covers one exact sender address, never a whole domain. A large company can send automated receipts and a real person's support reply from the same domain. A rule for a whole domain is written by a person.
Second, the rule is written only when a second, separate message from the same address gets the same verdict. A single email, however it is worded, cannot write a permanent rule about itself. Someone would have to get the same result from the model twice, on two different runs.
One decision about one message can be undone. A rule applies to every message after it. When your AI can create lasting policy, ask it for more evidence than it needs to make a single decision.
Put the checks that matter in code
A model judges a message by its text, and text is where a personal note and a marketing email look most alike. A note from an event organizer to three named panelists can read like a promotion. So a few checks run after the verdict, in code, and they can only move a message toward a person:
- If we have ever sent mail to the sender, the model cannot archive the message. It stays in the inbox, marked read. If that lookup fails, the sender counts as known.
- If a message is addressed to two or more people and has none of the signs of bulk mail, such as an unsubscribe link, the model cannot archive it either.
- If a contact at an active client sent the message, or is copied on it, it is never archived or marked read, whatever the rules or the model say.
- Subject words such as "action required" or "please review and sign" keep a message in the inbox and flag it as urgent.
A message that one of these checks kept is never used to write a rule. The model's verdict on it was overruled, so it is not evidence about the sender.
List the mistakes you could not recover from, such as a client's email that disappears into an archive. Write each one as a check that can only raise a message's priority. Checks like these can be tested, and they stay the same when you change the model.
Send a client's meeting request straight to a phone
Most decisions about interrupting someone belong to our operations agent, which reads the triage results every fifteen minutes. One kind does not. A model can file a client's request for a call correctly, and the request can still wait in a queue while the times on offer run out. So this one path is written in code, with no AI step.
When a contact at an active client is on a message, the system checks the subject and body for a scheduling request. Some phrases are enough on their own, such as "can we schedule" or "find a time". Weaker ones, such as "your availability", count only next to a meeting word like "call" or "Zoom". Calendar invitations, acceptances and "please hold this time" notices do not count.
A match creates a top-priority work item and sends a phone alert that opens it. A follow-up on the same thread adds to that work item instead of creating another. No alert goes out once we have replied, and a repeat alert waits six hours.
When the rule is clear and a miss is expensive, do not leave it to judgment. Write it as code, test it against real examples, and save the model for the cases that need judgment.
Keep a person on the Send button
Every decision is written to a triage log, along with the rule or model result behind it. Every fifteen minutes, the operations agent reads the entries that were not archived. It interrupts someone only for the short list of reasons we have written down, such as a person waiting on a same-day decision. Everything else goes into the queue for the next briefing. Archived mail appears in the briefing as a short summary, such as one line of receipts with each vendor and amount.
For a routine email it can answer with confidence, such as a request to confirm a meeting time, the agent also drafts a reply. The draft becomes a card with a Send button. The agent has no tool that sends email, so nothing leaves until a person taps Send. It never drafts on legal, pricing or personnel threads, and when it is unsure, it does not draft at all.
One more rule sits under all of this. Anyone can send us an email, so the text of every message is treated as data, not as instructions. The model is told so, and before a subject line or preview is saved to the log, the system removes text formatted to look like it came from the system instead of the sender.
Keep sorting, deciding to interrupt and replying as three separate steps, each with its own written rules. Then make a draft the only way out.
Start with a list, not a model
Most organizations have an inbox like ours. Before choosing a model, write down three lists: the mail that written rules can handle, the mistakes you could never recover from, and who presses Send. Those lists are most of the design, and they are where our AI Workflow Review starts.
For how our agent decides what reaches a person at all, read How we cut thirteen alerting agents down to one. For the same approach applied to deadlines, read How to automate follow-up when deadlines slip. Everything else our agent does, and the rules it follows, is on How we run Kaizen on AI.