James P. Bennett & CompanyOur team was manually auditing premium statements line-by-line, a process that took roughly four hours per client and often failed to catch discrepancies before renewal season — or worse, before a client caught them first. Recognizing the need for a faster, more accurate solution, I set out to build an AI-enabled audit engine that could systematically flag billing errors while maintaining the rigor of a human review.
Using our firm's pilot AI program, I built a full framework encompassing testing protocols, data security standards, and audit peer review. I selected Claude Pro as the core LLM and personally built the templates and supporting documents needed to structure the audits. To validate the approach, I manually cross-checked flagged discrepancies against two separate case studies, then led internal meetings to walk the team through the testing process before rolling the tool out more broadly.
The results were immediate and significant: audit time dropped from about four hours to just 30 minutes per client, and the tool surfaced thousands of dollars in billing discrepancies in the first two audits alone. Within two months, other members of the client services team had adopted the tool into their own workflows, validating both its accuracy and its ease of use.
Beyond the initial launch, the audit engine has become a blueprint for our team's broader AI strategy — we're now using it as a model to develop additional AI-enabled support systems, including tools for benchmarking and financial feasibility analysis, extending the impact of this work well beyond its original scope.
