Nelson Joyce

Tettra

Doubled AI Slack Bot Answer Rate from 30% to 60% to Recover Customer Conversions

Our AI Slack bot, Kai, had become a critical bottleneck in our sales process. We weren't closing any new customers because prospects testing the bot during trials were getting too many unanswered questions, prompting them to choose competitor tools instead. With customer conversions at a standstill, improving Kai's answer rate became an urgent priority. I worked closely with our sales team and prospects to identify exactly which questions weren't being answered. Through this investigation, I discovered the root cause: we had recently migrated our search infrastructure from Algolia to Typesense, and the new platform wasn't properly tuned for our AI bot's needs. I led the effort to heavily tweak the search configuration, refine the RAG (Retrieval-Augmented Generation) prompt, and optimize the prompt context to better serve our users' queries. The results were immediate and dramatic. Kai's answer rate more than doubled from approximately 30% to over 60%, and new customer conversions resumed. This work directly unblocked our sales pipeline and restored confidence in our product during the critical trial phase when prospects evaluate whether our solution meets their needs.