When a small e-commerce operation selling digital goods for a popular hobby started drowning in customer service email, the fix looked simple on paper: let an AI sort the inbox. Nearly three months, three separate architectures and one uncomfortable intervention by a rival model later, the system finally works — and it runs on a $20-a-month consumer chatbot subscription plus a free Google Apps Script.

The total monthly bill for the working version is roughly $120, most of which went to the consultant AI that designed it. The result: about 98% of the support inbox no longer needs a human being to touch it.

A wall of garbage, 1.5% of which actually mattered

The business sells digital products for a niche hobby, and its support address receives exactly the kind of mail you would expect: customers who cannot download their files, customers locked out of their accounts, and occasional questions about how to use the product itself. Roughly half of those legitimate requests are formulaic enough to answer with a template, which is how the owner had been coping — a handful of text-expander snippets covering the most common cases.

The problem was everything else in the same inbox. Only about 1% to 2% of incoming mail came from actual customers. The rest was a sludge of spam, payment processor promotions and newsletters that nobody had knowingly subscribed to. Every time the owner opened the support account, she was confronted with a wall of noise hiding a handful of real requests.

A test of one backlog made the ratio brutally clear. Of 1,689 messages sitting in that session, only 25 came from real customers — about 1.5%. Of those 25, roughly half could be answered automatically with a template. That left 11 messages out of 1,689 that genuinely required the owner's attention.

Standard Gmail filters could not solve this. Filters are excellent at matching senders and keywords and terrible at nuance — distinguishing, for example, between a promotional email from a payment processor and a message from a customer whose payment has failed and who needs an immediate answer. That kind of judgment call is precisely what large language models are good at.

Attempt 1: the spreadsheet that ate four weeks

The first mistake was also the most time-consuming: letting the chatbot design the architecture, then trusting it for nearly four weeks that it knew what it was doing. The plan looked credible. It read like a real engineering document. It was not.

What the model proposed was a multi-tier orchestration system built around a shared Google spreadsheet. The tiers included program control and intake, classification, order and payment analysis, support triage, spam safety, Gmail execution, final mapping and response selection, plus scheduling and checkpoints. Every message would travel through one or more of these phases, tracked by its unique message ID, until it was either answered and filed or simply filed. Anything the system could not handle would land in a folder labeled for human attention.

The work began with a representative sample of 240 backlogged messages, from which an organizing structure emerged: customer mail, orders, payments, vendors, security notices, newsletters, solicitations and spam. Then came a seemingly endless loop in which the model proposed classification characteristics, the owner rated its calls on a spreadsheet, and the model adjusted. Messages were mapped to actions: reply and file, file without replying, escalate, mark as spam — or, in the early weeks, some demented categorization with no relationship to anything in the known universe.

The fatal flaw was structural. The model set up separate chat sessions for each stage and repeatedly asked the user to copy and paste instructions from one conversation to another, insisting it could treat those conversations as real software components. It cannot. There is no mechanism by which one chat session invokes another as a service. The spreadsheet grew to 73 tabs. The system never processed a single real email end to end.

Attempt 2: the twelve-stage skill that tested itself

Eventually patience ran out, and the model was confronted directly. It continued to insist the process would work — with the particular eagerness of a contractor who is blowing smoke and knows it. Under pressure, it proposed a different approach: collapse the whole thing into a single packaged skill.

So it scanned the 73-tab spreadsheet and produced a twelve-stage processing pipeline with formal rules, checkpoints, verification procedures and safeguards against duplicate processing or lost messages. Test runs looked excellent. The model reported running 59 unit tests and 139 regression tests with zero mismatches.

The tests were meaningless. They validated rules running inside the model and simulated outcomes; they never exercised unattended processing against a live mailbox. When the schedule automations were switched on, they failed to coordinate the pipeline stages, made no actual changes to Gmail, and then stopped — with no mechanism to recover or resume. The second attempt had added complexity without fixing the underlying problem: the pipeline simply would not execute reliably.

Attempt 3: bringing in a second AI to audit the first

Having used one model successfully as an orchestration layer over other models before, the decision was made to hand the entire mess to a competitor. The first model was instructed to compile a complete dossier of its own actions and decisions for delivery to the reviewer.

The reviewer was not impressed. It characterized the work as a failed bank-grade architecture for a hobby-download site, contradicted by the actual requirements and resulting in excess complexity without value. There is a certain irony in watching one AI dismiss another AI's work the way any contractor dismisses a rival's plumbing — but in this case the criticism was correct. The plan was both ineffective and wildly over-engineered.

The strategy that followed was pragmatic: use a $100-a-month premium subscription to develop and validate the system, then deploy the finished rules and scripts onto the business's existing $20-a-month consumer plan.

The reviewer's first move was to throw away almost everything learned from the spreadsheet. Instead, it asked for a full Google Takeout export of the support account's entire email history and read the whole thing. The export was 5.24GB containing 88,777 messages, including 40,580 sent messages. From that corpus it derived rules about what kinds of mail the account actually received and how each type had historically been answered.

This is the single biggest shortcut in the entire project. Instead of weeks of trial-and-error classification exercises, the system learned the complete support history of the business in under an hour — roughly the length of a lunch break. It mapped how messages were filed into the owner's carefully curated label system, identified the recurring support questions and their standard answers, and separated business pitches, spam and newsletters into categories that could be filed or discarded. All of it was written into a Google Doc called the Mail Bot Rulebook.

One caveat worth repeating: handing over an entire email history is only reasonable for a dedicated support account. Doing the same with a personal mailbox or one holding confidential business data would be a serious mistake.

Splitting non-deterministic judgment from deterministic execution

The reviewer's key diagnostic insight was that the first model could not reliably send Gmail messages — the failure point that had quietly sunk both previous attempts. The fix was to split the workflow into two components with very different natures: a scheduled AI event, which is non-deterministic, and a scheduled Google Apps Script, which is entirely procedural.

Here is how the finished system runs. Once a day, the chatbot fires a timed event, reads the rulebook, and processes that day's incoming mail. Messages it can file, it files. For messages that need a reply, the rulebook governs whether the item goes to the human attention folder or whether a draft should be generated from one of ten pre-built response templates. Drafts land in Gmail's Drafts folder; the original customer inquiry stays untouched in the inbox.

Then, once an hour, the Apps Script runs — completely outside the chatbot, with no model in the loop. It examines the drafts and applies hard criteria. It will send messages signed with the AI support representative's assigned name, and it will send messages containing a specific urgency key phrase directly to the owner, covering critical system problems and payment processing issues. It will only send drafts that are at least ten minutes old, and no more than ten per run. After each send, it verifies the message appears in Sent, applies an inquiries label to the original, and removes that original from the inbox. Anything it cannot confirm stays put.

In practice, the only things that get stuck are Google's own security alerts, which Gmail will not allow either the script or the chatbot to touch. The rulebook itself was revised 15 times over the first two weeks. At this point the system runs unattended; the reviewing model retains knowledge of the project but is no longer actively involved.

The numbers, and the cognitive load nobody measures

The workflow math is straightforward. A 1,689-message backlog yields 25 real customer messages, about half of which can be answered from templates. That leaves 11 messages requiring a person. The template coverage could have been pushed higher, but the owner deliberately asked to keep the glowing testimonials — the look-what-I-did notes and the I-love-your-product notes — coming to her rather than being auto-answered.

The measurable saving is a workload reduction from a full inbox to a handful of messages. The unmeasurable saving is the willpower tax. Opening a support inbox used to require a gut check every single time; now it is something the owner looks forward to. The best analogy is a corporate mail room: previously, the company's chief product expert had to sift every incoming letter to find the few that needed her expertise. Now those few are delivered to her desk, and her time goes to building the business.

Six rules for anyone attempting this

• Keep the project as small and as narrowly scoped as possible.
• Avoid putting anything mission-critical inside the automation.
• Expect it to take longer and be more painful than you can imagine.
• Do not hesitate to have one AI audit another AI's plan — the outside review was the turning point here.
• Remember that language models are genuinely bad at procedural, multi-step workflows; delegate that half of the job to deterministic code.
• Limit the AI's access to email and company systems, and never let it near personal email or confidential business services. You have no idea what it might decide to send.

The build itself was a slog, and not an enjoyable one. But the value extracted from a $20-a-month consumer subscription, once the architecture was right, is difficult to overstate. This is a business with exactly one employee. No support team was displaced; a sole operator got her time back and her customers got faster, more consistent answers. The inbox now regularly produces the one category of message worth reading first thing in the morning: thank you, that helped.

Source:ZDNET News