Total Economic Impact of Astute Bot: Forrester Study
Astute commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study that examines the potential return on investment (ROI) businesses may realize with Astute Bot
Astute provides a suite of consumer engagement software that helps organizations improve their omnichannel experience for consumers whether with a live or virtual agent. Specifically, Astute Bot allows customers to easily self-serve to find answers to common questions with the ability to seamlessly escalate to a live agent without having to start over.
Astute commissioned Forrester Consulting to conduct a Total Economic Impact™ (TEI) study that examines the potential return on investment (ROI) businesses may realize by adding Astute Bot to their contact center. The purpose of this study is to provide readers with a framework to evaluate the potential financial impact of Astute Bot on their organizations.
To better understand the benefits, costs, and risks associated with this investment, Forrester interviewed a women’s accessories retailer with eCommerce sales volume of $80 million. The retailer has been an Astute customer for nearly a decade with Astute Agent and Agent Assist. Its customer service agents can already manage customer contacts more efficiently, find information, and quickly find standard responses and suggested articles to speed up contact resolution. Recently, the retailer added Astute Bot as a new way for customers to find assistance and deflect some contacts, with quick answers to questions about topics such as exchanges, shipment status, or address queries.
With Astute Bot’s implementation, the retailer improved its customer satisfaction rates and deflected email and phone contacts. This allowed some agent-led contacts to be completed more quickly, as a contact started from an automated chat conversation may still require a phone call with an agent, but some of those questions have already been answered.
The retailer found it had enabled an additional, unexpected benefit. Astute Bot self-service chat tools were primarily located on post-sales customer support pages; for the purpose of answering questions about returns, exchanges, product issues, etc. But the retailer saw many consumers were using Astute Bot to answer presales purchase and product questions — such as whether an item comes in a particular color or size. Furthermore, the retailer identified many of these customers as first-time buyers that would have not completed their purchase without first finding answers with Astute Bot.
Quantified benefits. The retailer experienced the following three-year, risk-adjusted present value (PV) quantified benefits:
- New sales revenue and margin. Astute Bot provides another way for consumers to find answers to both support and product questions; especially relevant for those that preferred not to email or speak to an agent. By giving consumers an easy way to find answers, Astute Bot reduced friction in the buying cycle, leading to more purchases. The retailer estimates year-over-year sales growth of 4% in Year 1, growing to 13% by Year 3. It also estimates that 12% of this growth is due to Astute Bot, adding up to a three-year PV of nearly $219,000 in operating
- Support cost savings with a 16% deflection rate. With Astute Bot, the retailer estimates its contact center avoids having to handle 16% more contacts. At a cost of $5.50 per email contact, the retailer estimates a three-year PV of more than $546,000.
- Contacts started with Astute Bot are 40% shorter. Some questions may require an agent. But even though a question started with Astute Bot may lead to a phone call, email, or person-to- person chat, often some of the customer’s questions are already answered, meaning the agent can resolve the request more quickly. The retailer estimates the cost of these contacts are reduced by 40%. At a cost of $5.50 per contact, the three-year PV adds up to nearly $32,000.
Unquantified benefits. The interviewed organization experienced the additional benefits, which are not quantified for this study:
- Eighty-nine percent customer satisfaction. Customer satisfaction scores prior to implementing the Astute Bot solution averaged around 83%. Since Astute Bot, quarterly customer satisfaction ratings have been in the range of 87% to more than 89%.
- Reduced shopping cart abandonment. Customers discovered that even though Astute Bot was located on customer support pages, they could use it to get product-related information. They were able learn what they needed to be ready to complete a purchase, such as details about color, size, or fabric type. The retailer has also prioritized reviewing and scripting answers to common questions to provide answers within Astute Bot more often and with more accuracy. Abandonment rates for most shopping cart stages were improved, as customers were able to resolve questions without stopping to complete their purchases.
- Reduced support contact abandonment. With Astute Bot, the retailer can provide self-service options for those customers who either find it more convenient or prefer not to talk to a person. This helps to shorten phone hold times and speed up response times for questions which are sent over email or agent chat. With the variety of options — and faster response times — fewer customers lose patience and abandon the support request. This is expected to further contribute to transaction completion and improvement in customer satisfaction.
- Costs. The retailer experienced the following risk-adjusted PV costs:
- Astute Bot licensing. For the retailer’s volume, annual license costs are estimated to be less than $149,000.
- Deployment and management resource costs. Astute Bot ingests product and support information to answer many questions automatically. But the organization added resource time to identify common and important questions and prepare more specific answers to provide more specific details and highlight brand and cross-sale opportunities. For the retailer, the team used 1.5 FTEs working for two months. Additionally, the retailer estimates an additional 4 hours each week are spent managing Astute Bot, reviewing questions, and revising or adding new scripted answers. Upfront costs add up to nearly $25,000, and the three-year PV total for all upfront and ongoing costs adds up to a little more than $49,000.
Forrester conducted a financial analysis based on the interview with a retailer that has added Astute Bot to their existing Astute implementation. The interview and financial analysis found that this customer experiences benefits of $797,000 over three years versus costs of $199,000, adding up to a net present value (NPV) of $599,000 and an ROI of 301%.
“We want to be known for our great care of customers searching for items and ensuring that they are getting them to items that they need.”
– Customer service manager, retailer
“We operated through holiday this year with fewer employees than the previous year, but our abandon rate was much improved. That’s in part attributed to contact diversion with the chatbot. Within a month’s time of Astute Bot being launched, customer satisfaction was above 87%.”
– Operations manager, retailer
TEI Framework And Methodology
From the information provided in the interviews, Forrester constructed a Total Economic Impact™ (TEI) framework for those organizations considering an investment in the Astute Bot.
The objective of the framework is to identify the cost, benefit, flexibility, and risk factors that affect the investment decision. Forrester took a multistep approach to evaluate the impact that the Astute Bot can have on an organization:
- Due diligence
- Customer interviews
- Financial model framework
- Case study
The Astute Bot Customer Journey
Drivers leading to the Astute Bot investment
For this study, Forrester interviewed two representatives at a women’s accessories retailer: a customer services manager and an operations manager. The retailer is a longtime Astute customer that has recently added Astute Bot to its contact center. Before Astute Bot, the retailer had the following characteristics:
- Eighty million dollars in annual eCommerce revenue.
- About 260,000 to 270,000 contacts received each
- Low single-digit year-over-year (YOY) eCommerce
As a longtime Astute customer, the retailer had continued to improve its customer support processes. A natural next step in the direction of progress would be in providing customers (and prospective customers) with more self-service options. The retailer started testing Astute Bot to help:
- Reduce the number of simple and frequently asked questions. Many questions were heard repeatedly by support agents, but they had to take the time to answer and resolve each request as if asked for the first time. Directly exposing some customers to the knowledge base and FAQ content was helpful, but it wasn’t ever enough. The customer service manager stated, “We started with the knowledge tool for the internal CSRs and had looked at possibly exposing that in some different ways.”
- Improve customer satisfaction. Leaders set a corporatewide goal to improve this score by 5 percentage points. The customer service team recognized that they could significantly contribute to this goal with more convenient customer service channels, reduced customer support wait times, and more visibility into the customer service flow. The customer service manager stated, “Before we started working with Astute, the team was similar in size and we had similar roles, but we were not capturing any of the detail around our cases.
- Consolidate and simplify knowledge resources. The retailer had previously given knowledge and learning resources to agents that had Astute Knowledge software; integrated with the Astute Agent contact system. But for customer-facing channels, a chatbot solution could allow a user to simply type in their own question to find an answer, rather than go through the complicated process of looking for the right FAQ and then search for their answer
With Astute Bot, the retailer enabled:
- Growth in annual sales revenue as customers used Astute Bot to get quick answers to product questions, which led to more completed sales.
- Reduced contacts and contact center costs as many contacts were deflected to Astute Bot, avoiding the need for an agent-led support contact.
- More efficient agent-led phone, email, and chat contacts that do still occur.
Analysis Of Benefits
Quantified benefit data
New Sales Revenue And Margin
Evidence and data. Without an automated chat channel, customers had to call, email, or open a chat window and wait to talk to a real person. Some drawbacks included: being put on hold, waiting for an email response, and being required to engage with a person to find a solution.
With Astute Bot, customers were able to quickly find:
- Answers to simple support questions — such as “Where is my order?” and “Can you help me reset my password?” — with links to the right resources and
- Answers to product questions — such as “Does this come in blue?” or “Is this my size?” — with appropriate, and up-to-date
Product questions were a welcome surprise. Astute Bot was initially integrated with support pages, but the retailer quickly identified that customers had more than support questions. It made sure to keep monitoring and updating the Astute Bot message flows to be able to instantly answer as many of these presales questions as possible.
Those product answers also led to new sales transactions. Many customers with a product question were not taking the time to make a phone call or write an email, and as a result, they would change their mind about purchasing. But with Astute Bot, they found a place to quickly answer their product questions, getting the quick information they needed — color, fabric type, etc. — to complete their purchase.
The retailer has prioritized reviewing and scripting answers for common questions to be even more ready to handle customer questions about product details and provide more accurate and standardized information. The customer services manager at the retailer said, “Astute Bot allows us to speak with an extra level of brand consistency.”
Percentage of annual growth attributed to Astute Bot: 12%
Modeling and assumptions. The retailer expected $80 million in eCommerce sales at the start of the first year, based on relatively flat growth. With Astute Bot, it estimates:
- Higher year-over-year growth rates of 4% through the first year, and it now expects 8% in Year 2 and 13% in Year 3
- Astute Bot is attributable to 12% of that growth. The retailer has other customer-focused strategies underway, which also impact growth
- The retailer has an 11% operating margin
Risks. As it can be difficult to measure the impact of one solution across all sales, Forrester risk-adjusted this benefit downward by 10%.
Results. Annual benefits add up to a three-year, risk-adjusted total PV (discounted at 10%) of nearly $219,000.
Deflected Support Contact Cost Savings
Evidence and data. Some customers leveraged Astute Bot to answer questions before they were ready to purchase, but many still had traditional support questions, typically after they had made a purchase. Questions and issues — such as “Where is my order?” or “I’d like to make a return?” or “I forgot my password?” — are still very common.
Similar to product-related questions, the retailer was able to configure Astute Bot with specific and detailed answers to expected common questions — but also knew Astute Bot would be able to handle many other questions automatically. The retailer can provide customers with another channel to find information and links to find account details, address information, return instructions, or instructions to reset a password. The customer service manager at the retailer said, “We want to be known for our great care of customers searching for items and ensuring that they’re getting them to items that they need.”
Modeling and assumptions. With Astute Bot, the retailer estimates:
- It receives between 260,000 to 270,000 contacts each year.
- It has deflected 16% of contacts that would otherwise involve an agent, which adds up to about 43,000 to 51,000 contacts avoided each year.
- The retailer estimates that Astute Bot is directly attributable to 90% of this benefit (in addition to other knowledge base and FAQ improvements).
- Each agent-led contact averages $5.50 (including phone, email, and person-to-person chat).
“Before we started working with Astute, the team was similar in size and we had similar roles, but we were not capturing any of the detail around our cases.” –Customer service manager, retailer
Risks. As the mix of phone and other contacts (and thus the average cost per contact) may vary, Forrester has applied a 5% risk adjustment.
Results. The three-year, risk-adjusted benefit total adds up to $546,000.
Call deflection rate with Astute Bot: 16%
Contact Time Savings Due To Astute Bot
Evidence and data. In addition to contact deflection, Astute Bot helps complete calls, emails, and in-person chats more efficiently. If a customer uses Astute Bot to answer a question, in many cases some information is shared before the Astute Bot decision tree ends at a point where the customer should be escalated to a support agent.
Modeling and assumptions. The retailer estimates contacts are more efficient:
- The number of Astute Bot interactions that lead to an escalation adds up to between 6,000 and 6,300 each year.
- For those contacts (at the same base price of $5.50 per contact), it is estimated they are completed about 40% more quickly.
Risks. Forrester applied a 5% risk adjustment to model the cost-per-contact and the number of contacts more conservatively.
Results. The three-year, risk-adjusted total PV is nearly $32,000.
Additional benefits were reported by the retailer, though the financial impact of those benefits was not considered — either because the benefit was too recent or too broad. Astute Bot is one of several drivers that have led to an overall improvement in these areas. These benefits include:
- Improved customer satisfaction to more than 89%. Before Astute Bot, customer satisfaction scores hovered around 83%. Since Astute Bot, quarterly customer satisfaction ratings have risen to the range of 87% to +89%.
- Reduced support contact abandonment. With Astute Bot, the retailer can provide self-service options for those that find it more convenient or prefer not to talk to a person. This helps to both shorten phone hold times and speed up response times for questions sent over email or agent chat. With the variety of options — and faster response times — fewer customers lose patience and give up. This is expected to help further contribute to the completion of transactions and to the improvement in customer satisfaction. The operations manager at the retailer said, “We were able to manage this year with much lower abandon rate than we have in the past.”
The value of flexibility is unique to each customer. There are multiple scenarios in which a customer might implement Astute Bot and later realize additional uses and business opportunities. The retailer has identified several options that it is planning to add, or is considering adding, to its Astute Bot solution, including:
- Adding Astute Bot on more pages. The retailer is already working to surface Astute Bot on more web pages, including product pages, which would make it easier for customers to ask any question without having to leave the product or catalog page they are viewing. This is expected to help drive even more call deflections as more customers know about the automated chat service and can use it
- Additional Astute investment. The retailer has also discussed adding more Astute chatbot and agent chat options to help more customers who are asking questions, and to allow for more convenient agent escalations when necessary, reducing call times and improving customer experience.
Flexibility would also be quantified when evaluated as part of a specific project (described in more detail in Appendix A, see PDF document).
Analysis Of Costs
Quantified cost data
Astute Bot Subscription Costs
Modeling and assumptions. Astute Bot requires a subscription fee to license. For the retailer’s volume, this is estimated to be about $60,000 per year.
Results. Forrester did not adjust this value for risk. The three-year total PV (discounted at 10%) is $149,000.
Evidence and data. To implement Astute Bot, some preparation is necessary. While the AI and natural language processing inherent to Astute Bot can automatically find an answer to many questions, that answer does not always: 1) have the right information; 2) highlight new campaigns or branding; or 3) involve clean formatting. To prepare for the Astute Bot launch, employees were involved in planning and preparing answers to expected questions; i.e., The operations manager was involved for the two-month implementation period, as well as a variety of employees being involved occasionally, adding up to another 0.5 FTEs.
Modeling and assumptions. Once launched, the retailer spends about 4 hours each week reviewing each Astute Bot session, updating answers for new or preexisting questions and preparing reports
Resource costs include:
- Implementation period lasting over two months for 1.5 FTEs with an average fully burdened salary of $90,000 — this adds up to $22,500 during the initial period.
- Ongoing management involves the same type of resources with the same average annual salary for an average of four weeks total per week, which adds up to $9,900 per year.
Risks. As implementation and maintenance needs may vary, Forrester added a 10% risk adjustment to this cost.
Results. The three-year, risk-adjusted total PV (discounted at 10%) is less than $50,000.
Forrester Consulting provides independent and objective research-based consulting to help leaders succeed in their organizations. For more information, visit forrester.com/consulting.
Readers should be aware of the following:
This study is commissioned by Astute and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.
Forrester makes no assumptions as to the potential ROI that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the report to determine the appropriateness of an investment in the Astute Bot.
Astute reviewed and provided feedback to Forrester, but Forrester maintains editorial control over the study and its findings and does not accept changes to the study that contradict Forrester’s findings or obscure the meaning of the study.
Astute provided the customer names for the interviews but did not participate in the interviews.