AI bots, helpers, and automation tools are flooding my social media feed—and I’m sure I’m not alone. But seriously, is it really as easy as those slick social media ads make it seem?
Well, yes and no.
If all you want is to get a large language model (LLM) to draft a response based purely on the content of the email—and maybe the preceding email chain—then yes, AI can do that. But that’s about the limit of what these heavily advertised email drafting bots can actually do.
And there are some serious risks to be aware of!
One of our major clients at Excite Labs relies on us to manage their customer service function. We recently developed an AI system for handling their inbound customer service emails, which range from 200 to 500 messages per day. We found that around 90% of these emails fall into roughly 15 common categories.
For those categories, we created 15 powerful response templates—crafted in the client’s unique voice and tone, and aligned with our shared philosophy of “freaky good service.” These templates are so effective they regularly turn customer frustration into satisfaction. They also form part of the valuable joint intellectual property (IP) between our client and us—so protecting them is non-negotiable.
We designed our AI email drafting module to work like this: it first attempts to match each incoming email to one of the 15 templates. If it determines that the match is too weak, it instead uses an LLM to generate a new response—on the assumption that the query falls outside the normal range.
We learned the hard way that if you try to guide an LLM using old email responses, the output can vary wildly in tone, quality, and appropriateness. That’s why we’ve made sure the templates and all source materials used to train the AI are kept “dark”—hidden from future LLM training—to protect our and our client’s IP. Those off-the-shelf bots advertised on social media? They don’t offer that kind of protection.
We’re now developing five more response templates, which we estimate will allow the AI system to handle an additional 5% of inquiries effectively. And while a human still reviews every response before it goes out, the time and cost of drafting emails have dropped dramatically. We’ve passed those savings on to the client.
So, what’s the upshot?
Despite the charming AI mascots in those ads, you definitely get what you pay for—and in many cases, what you get will either be underwhelming or a potential IP disaster.
