Tethr Evaluations
Giving users the power to add a human perspective to Tethr’s powerful AI generated conclusions
Overview
Project timeline: 2 months
Tasks: UX/UI Design, user research, user testing
Project Type: Full time
What is Tethr?
Tethr is an AI powered speech analytic software that makes post call surveys obsolete. It listens into conversations between customers and service agents and is able to come to intelligent conclusions: metrics like customer effort, positive agent behaviours, level of customer frustration, and much, much more.
Defining the problem
Customers love Tethr’s core experience as it not only makes analyzing cause and effect extremely efficient, but it offers a quantifiable way to understand how a call went. However, the request from customers was that they wanted the ability to augment Tethr’s insights with their own perspective – to give the insights more dimension with qualitative data.
In the tech industry, there is currently an obsession with imbuing AI into every possible corner of a product. The issue is, after doing some industry research and attending conferences with expert speakers on this topic, customers still don’t fully trust “AI” and want to feel like they possess a level of control over its output. The pain point we identified within Tethr’s product is in lockstep with this wider ethos.
Our first pass
Getting started
Preliminary interviews
The first step was to discover the needs of our customers, so, before taking pen to paper, Carl (the VP of Product and Design) and I conducted interviews to discover what those were. These were generative interviews, where we came in with a standardized set of questions, including prompts like “are there any standout pain points you’ve encountered with Tether?”, to understand their experiences and the general types of solutions they were looking for. Then, we went a level deeper by asking questions like "When using Tethr, have you ever encountered situations where you felt the need for more capacity to express your own opinions about how a call went? If so, can you provide specific examples or details about those moments?"
Iterate
Using these findings, we took a double-pronged approach to solve our problem. We called the first part Tethr Hub and the second part Evaluations (a V1 version we tested). We brought both of these concepts to Apple and Peloton, walked them through the product experience, and sought their feedback.
Screenshots and descriptions are attached in the following pages of both concepts. These are a few of the actual screens we used in our presentation to Apple and Peloton.
Our presentation
These are the presentation slides we used to present both experiences to Apple and Pelaton during a 2-hour call. On the left is Evaluations and on the right is Tethr Hub.
Takeaways after testing
Tethr Hub’s Redundancy
While the concept of Tethr Hub was intriguing, they approached it with pragmatism, noting that their tech stack already allowed for similar forms of communication. They used platforms like Slack or Microsoft Teams, which were already leveraged to discuss reports, dashboards, specific calls, and more. For this reason, they considered it a "nice-to-have" rather than an immediate necessity.
Limited question types
V1 of Evaluations only allowed for multiple choice, but the first thing our customers noted was the need for more question types. They felt that adding a matrix table and open-ended text fields would meet their needs.
Too many forms
V1 of Evaluations permitted multiple forms within a single Evaluation. However, as we conducted continuous tests with customers, it became evident that this approach was unnecessary and caused confusion. Our customers found it more logical for each Evaluation to have a single dedicated form. They used words like “over-engineered” when describing our original version.
Conditional triggers out of scope
The engineers clarified that implementing the conditional triggers (the blue containers between each form) was beyond their scope. Initially perceived as a straightforward addition, it became apparent that the necessary logic for implementation would demand a fundamental alteration to our codebase.
Timeline
There was skepticism regarding our ability to meet a reasonable deadline. We openly communicated that achieving the originally proposed three-month timeline would likely be challenging due to the substantial engineering effort required for both Tethr Hub and Evaluations.
Incorporating the feedback
We synthesized the feedback during these interviews into 2 categories. First, understanding the usefulness of Tethr Hub as a whole and where to go with it next, and second, how to improve Evaluations as that was the feature our customer truly wanted.
1. What to do with Tethr Hub
Ultimately, we realized that we had to put Tethr Hub to the side for this V1 push. The feedback around it was consistent: it was a “nice to have” over a “need to have” and the build time + the redundancy with their current tech stack didn’t justify putting so many resources into building it.
2. Refining Evaluations
As a reminder, here is the specific feedback we received about the limitations of Evaluations:
Limited question types: V1 of Evaluations only allowed for multiple choice, but the first thing our customers noted was the need for more question types. They felt that adding a matrix table and open-ended text fields would meet their needs.
Too many forms: V1 of Evaluations permitted multiple forms within a single Evaluation. However, as we conducted continuous tests with customers, it became evident that this approach was unnecessary and caused confusion. Our customers found it more logical for each Evaluation to have a single dedicated form. They used words like “over-engineered” when describing our original version.
We took this feedback and decided that although we didn’t want to throw the baby out with the bathwater, there were a few key areas that we needed to redesign to satisfy our customer’s needs for the final version of Evaluations.
Evaluations: Take 2
Evaluations V2 wireframes
Below are some rudimentary wireframes of our final Evaluation designs. The 4 screens show the following:
1. The initial 0 data state
2. What it looks like after the first question is added
3. A fully populated Evaluation
4. The experience of an end user completing an Evaluation within a call.
Putting it all together
After weeks of iterating, testing, and receiving feedback, we collected all of our findings to ship the designs you see below. During the final design phase, we still ran into questions that we didn’t have immediate answers to, so we often leveraged our CSMs as they were closest to our customers on a daily basis. Additionally, I prioritized consistent communication between design, engineering, and product throughout the final phase to ensure that when handoff arrived, there were no surprises.
Final Designs
The Form Builder
This is where Evaluations begin. Choosing between a multiple-choice set, a matrix table, or an open-ended text field, the user can define exactly the questions they want answered for any given call. With the ability to score each question and then later graph the responses, the user can get a high-level understanding of how their agents are performing (in this case from a manager’s POV) with specific queries built around the question’s Evaluations answer.
Building the form
These are the first few steps to building out the form. It is a very simple step-by-step process outlined below.
1. Choose a question type
When first defining the scope of this feature, we talked with CSMs and engineers to figure out what question types users needed/are feasible for a V1 launch. We settled on 3 of the following:
Multiple choice: This is the standard question format that is offered in nearly every survey. It’s intuitive, incremental, easily quantifiable, and for many users all they need.
Matrix Table: This is a question format that is aimed at users who are hoping to delve deeper into the details of call quality. Unlike the multiple-choice format where there may be 5 answers to 1 question, a table can have 5 subcategories to a question with each category having 5 potential answers.
Text entry: Sometimes a user wants to ask a question that is best answered through a written response. Though it’s not a quantifiable question format, it still is essential to give Evaluations that human-centred experience.
2. Evaluation trigger filters
So what do we mean by trigger filters? Take this example: maybe the user made an Evaluation that digs into calls where the Agent Impact Score was low, but instead of manually searching for every call with a low score the user can specify a score threshold with a filter. That Evaluation will then automatically appear on all calls that meet that criteria.
3. An easy way to edit
Editing the text on a form builder is the most common interaction, and we made it exceedingly easy and intuitive to do so. Simply click the zero data state which is also a CTA (e.g. “add question text), and that row will turn into a text field the user can type into. It’s as ergonomic as it gets.
Populating the form
1. Assign points
Tethr’s core strength is in its ability to track, visualize, and analyze data. And we wanted to bring that ability to Evaluations as well. So we allowed users to assign point values to each answer (for both multiple-choice and a matrix table). This allows for a user to measure overall trends of call quality – but from a human perspective.
Users can assign points via two routes:
Assign points to each answer: Allows for weighting of scores (e.g. “Fail” could be 1 but “Needs Improvement” could be 2, while “Exceeds standards” could be 10).
By range: This allows a quick way for the user to equally weight every question. There’s no math involved, all they have to do is define the lowest and highest score and Tethr will calculate the rest.
2. Define required/optional
Some questions will be more important than others, so to ensure that the user can define that hierarchy we allow them to define if a question is required or optional.
3. Permissions
Users can define 3 levels of permissions: Which users can fill out an Evaluation, who can edit it after completion, and who can view it after completion. These are the 3 levels we settled on for one central reason: our customers have privacy requirements that must be met.
Completing an Evaluation
This is Tethr’s Interaction page (a page I had the opportunity to thoroughly redesign to what you see here). Users come here to listen to calls, view high level insights of how that call went, see what kind of categories (e.g. “customer frustration”) were hit, and more. This is where Evaluations ultimately appear.
1. Manually add an Evaluation
Though most Evaluations will appear via the automatic triggers set on the builder screen, the user can manually add an Evaluation as well. It is done through a simple search field which opens a dropdown and the user can add them in bulk.
2. Overall UI
As you can see, the questionnaire form itself is spare. We kept it as simple as possible as this page is already very data-heavy. The multiple choice is simply a dropdown, the matrix table is identical to what the user sees when building the form, and the text box, is, well, a text box.
3. Scoring
Once the user completes a form, an overall score will appear at the bottom. Showing this score gives the user the ability to first evaluate (no pun intended) if the answers they give truly reflects what they thought about the call. Maybe the score was 40% when they actually thought it was a well-handled call, so by having this transparency the user can go back and change their scores before submitting.
4. Versioning
With the ability to edit an evaluation after completion (by any user with proper permissions), versioning was a necessary affordance. The simple link button allowing the user to update to the latest version keeps the versioning convention as simple as possible.
Graphing it all
A user can go into Tethr’s powerful graphing tool and build custom queries around a multitude of Evaluation metrics. Whether it’s the number of Evaluations completed, certain scores of Evaluations, the number of particular types of answers that were given on an evaluation (and much more), visualizing these metrics on a higher level offers the user a deeper understanding of what is happening in their call centers.
How did we measure success?
Retention rate
After releasing Evaluations, Tethr’s retention rate the next year hit over 100% (for clarity, this means existing customers purchased additional features within Tethr and we had almost no churn in general). A very rare stat. Customers, including Apple, often credited Evaluations.
Low customer support requests
Traditionally, Tethr had an enormous number of support requests and we had to hire many CSMs accordingly. But months after the initial release, our CSMs told our team that of any new feature, Evaluations had the lowest number of customer support requests.
Adoption rate
After conducting beta testing and rolling out the product to all our users, we achieved an impressive 93% adoption rate within just one month.
Trail blazed
Within the CX speech analytic world, an end-to-end survey experience like Evaluations wasn’t available. We were the first to do it and, according to the data and feedback we received, it was done well.
“Good produce equals good process”
- Carl Schultz: VP of Product and Design at Tethr
I still refer back to this project for it’s exemplar product development process. It’s success was due to taking the proper steps from idea to delivery. We iterated, tested, kept consistent communication between stakeholders, scoped it out well, and ultimately delivered. Want a good product? Have a good process.