Automation
The Draft That Broke Trust
By John J. BakerJuly 20, 20265 min read
A sales rep at one of our clients sent me screenshots one afternoon with a short note: here is the proposal that keeps generating drafts. He was reporting a broken system, and he was reporting it the right way, with evidence attached.
So I went and looked. The draft had been generated on June 16, when the quote was two weeks old and nobody had replied. The client wrote back on June 19. The sequence stopped that day and never produced another thing for that lead. By the time the rep found the draft sitting in his queue, the system had been quiet on that client for two weeks.
Nothing was broken. Every rule fired correctly, in the right order, on the right day. And the rep was still right to flag it.
A system can be correct and still lose the room
Generates a follow-up draft
The quote is fourteen days old with no reply on record, so sequence step three comes due and the system drafts the nudge. Correct.
Nothing yet
The draft sits in a queue. Nobody has opened it.
Generates a follow-up draft
The quote is fourteen days old with no reply on record, so sequence step three comes due and the system drafts the nudge. Correct.
Nothing yet
The draft sits in a queue. Nobody has opened it.
Stops the sequence
The client replies. The sequence halts and no further follow-ups are generated for this lead. Also correct.
Nothing yet
The conversation moves on. The June 16 draft is still sitting there.
Stops the sequence
The client replies. The sequence halts and no further follow-ups are generated for this lead. Also correct.
Nothing yet
The conversation moves on. The June 16 draft is still sitting there.
Generates nothing
Two weeks of silence on this lead. The system is doing precisely what it was built to do, and has been all along.
A draft with no date on it
The rep finds a follow-up addressed to a client who already replied. He sends screenshots: the proposal that keeps generating drafts.
Generates nothing
Two weeks of silence on this lead. The system is doing precisely what it was built to do, and has been all along.
A draft with no date on it
The rep finds a follow-up addressed to a client who already replied. He sends screenshots: the proposal that keeps generating drafts.
The system did exactly what it was told. The rep was still right to flag it.
The draft carried no timestamp and no reason, so the only explanation available to the person holding it was that the machine had lost the thread. That is a visibility failure, not a logic failure, and no amount of better logic would have prevented it.
Put yourself in his position. You open your drafts and find a follow-up addressed to someone you already spoke to. The draft does not say when it was written or why. You have no window into sequence steps or reply timestamps. You have one artifact in front of you and it looks wrong, so you reach the only conclusion available: this thing has lost the thread.
That reaction is not a failure of understanding. It is a completely rational read of the evidence he was given.
This is worth separating from the more familiar problem, where an automation breaks silently and produces plausible garbage. That one costs you a bad decision. This one is close to the opposite: the output is right and the trust is gone anyway. The cost is not a wrong number in a forecast, it is a rep who starts checking everything by hand, then stops opening the tool, and never files a ticket because as far as he knows he already told you.
A team's confidence is an output of your system. It is one you do not get to see on a dashboard, and it is the one that decides whether anything you built is still in use in six months.
Decide what should never be a judgment call
The instinct when an AI system behaves confusingly is to improve the AI. Better prompt, better model, more context. That is usually the wrong lever, because in most of these systems the AI was never supposed to be making the decision in question.
Walk through what a follow-up automation actually does. Has this client replied? Is this lead paused, closed, or won? Which step of the sequence is due, and has enough time passed? Every one of those has exactly one right answer, knowable from the record. None of them is a judgment call. They are rules, and rules belong in code, where they run the same way every time and can be read out loud in a meeting.
One thing in that list is genuine judgment: how the message should read. That is the part worth handing to a model.
State
What is true right now?
Has the client replied? Is the quote won, lost, or open? Read from the record, never inferred.
Eligibility
Should this person be contacted at all?
Paused, closed, opted out, already in conversation. Any one of these ends the run here.
Timing
Is a touch actually due?
Which step in the sequence, how many days out, and has that window arrived.
Wording
How should this read?
The one genuine judgment call: tone, the specific ask, the detail worth referencing. This is the box.
The gate
Does a person approve it?
Nothing reaches a customer unseen. The rep sends, edits, or kills it, with the reason in view.
The AI never decides whether a customer hears from you. It only decides how the sentence reads.
Every question with a right answer is a rule. Judgment is the only thing worth spending a model on, and even that gets a person in front of it before it leaves the building.
This is the discipline that matters most, and it is the one most often skipped: make deterministic everything that should be deterministic. When a model is asked to decide whether a customer gets contacted, you have converted a question with a right answer into a question with a probable answer, and you have made the system impossible to reason about when it surprises someone. When the model only writes the sentence, a surprising output has a small number of possible explanations and every one of them is checkable.
The phrase we use internally is building a box around the AI. Not a cage, and not a smaller model. A defined space with rules on every side: the model gets one job, it gets the context it needs to do that job, and it hands its work to a person before anything leaves the building. What comes out is bounded by design rather than by hope.
The wrong way and the right way
The same automation can be built either way, and on an ordinary Tuesday both versions look fine.
Hand the whole job to the model
- What the AI decides
Whether to reach out, who to reach out to, when, and what to say. One prompt holds the whole job.
- What the person sees
A finished message. No indication of what triggered it or when it was written.
- On an edge case
The model improvises. It is fluent, so the improvisation is confident and hard to spot as improvisation.
- When output looks odd
Nobody can tell whether it misfired or worked as designed, so the safe assumption is that it is broken.
- What the team concludes
Stop trusting it. Check everything by hand. The automation quietly stops being used.
Rules everywhere, AI in one place
- What the AI decides
The wording, and nothing else. Who, whether, and when are rules the model never touches.
- What the person sees
The message plus its reason and its timestamp. Drafted July 2, step three, quote sent June 18, no reply on record.
- On an edge case
The run stops and flags. An unhandled case is a question for a person, not a guess.
- When output looks odd
The reason is attached, so the team either understands it in five seconds or reports a real bug.
- What the team concludes
Trust it and keep using it, including on the days it surprises them.
Look at the last row, because that is where the whole thing is decided. Both builds produce good follow-ups on a good day. They separate completely on the day something looks strange, and that day always comes. In the loose build, an odd output is unexplainable, so it reads as a defect and the tool loses a user. In the boxed build, the reason travels with the artifact, so the person either understands it in five seconds or files a report that is actually about a bug.
Notice how small the fix in our story was. Not a rebuild, not a better model. Put the date and the trigger on the draft, and give the rep a Pause and a Reset button inside the record he is already looking at rather than somewhere else in the tool. That is a visibility change and a control change. It costs almost nothing and it is the difference between a system people trust and one they route around.
This applies to internal systems too, and people forget
Client-facing automations get scrutiny because the downside is obvious. Nobody wants a strange email going to a customer. So the guardrails go on, the human gate goes in, and the team reviews before sending.
Internal tools tend to get built with a looser hand. It is just for the team, the thinking goes, so a rough edge is survivable. The consequence is real, it is just slower and harder to attribute. Your team is the first customer of every system you build, and their trust is the thing that determines whether your next automation gets adopted or quietly ignored. A finance lead who gets one inexplicable output from an internal forecast will check that forecast by hand for months afterward, and will bring exactly that expectation to the next tool you hand her.
Worse, distrust does not stay local. It generalizes into a belief about the whole category: this AI stuff is unpredictable. That belief is expensive and slow to reverse, and it is usually earned by one unexplained artifact rather than by any actual error.
So the standard is the same on both sides of the wall. Rules where there are right answers. AI in a defined box, doing the one thing that needs judgment. A person in front of anything that carries weight. And every output arriving with enough context that a reasonable person can tell, quickly, whether it makes sense.
Where ClearOak comes in
We build automations for companies where the work is real and the margin for weirdness is thin. That shapes how we build. We spend most of the design conversation deciding what has to be deterministic, then we draw the box for the AI, then we decide what has to be visible on every single output so the team can audit it at a glance.
The goal is not an automation nobody ever questions. The goal is one that answers the question. When a rep looks at something odd and can see why it happened, you have a team that keeps using the system and reports real problems when they find them. That is the compounding asset, and it is built out of small, unglamorous choices about visibility and control.
If you have a tool your team has quietly stopped trusting, that is usually fixable, and the fix is usually smaller than a rebuild.
Schedule a call and tell me where your team has started double-checking the machine. We will look at what they can see, what they can control, and what the system is deciding that it should not be.
Prefer email? Reach me directly at john@clearoakconsulting.com and describe the output that made someone suspicious. That is usually enough to find the gap.