One of the questions in the survey was multiple choice, used for generating a product market fit score. The other three questions were open ended. We aggregated the free text answers and applied our own algorithmic model to extract topics across two dimensions: product strengths and ideas for improvement.
Hundreds of our users found all four questions valuable. But many of them also wanted to create and send surveys with their own, custom questions. Knowing whether their business had achieved product market fit wasn’t enough; they also wanted to understand their customers more deeply.
It seemed to us that topic analysis alone wouldn’t help companies develop customer empathy at scale. We needed to get sentiment and emotion from customer feedback too. While our model could extract topics, it wasn’t designed to conduct sentiment or emotional analysis.
GPT-3 can do sentiment and emotional analysis in addition to topic identification. It also turned out to be more accurate at topic identification than the model we were using for Viable Fit.
For business teams to quickly and easily get summarizations from customer feedback data, we built a system made up of four components.
1. Data ingestion & processing
Viable is set up to ingest text based customer feedback from common survey and helpdesk tools via a direct integration, a “Zap” from Zapier, or manually through our ingress endpoint. Upon ingestion, both our own models and GPT-3 process the data to identify topics and insights, and then assign both a sentiment and emotion to each data point based on its context.
Our in-house system facilitates data import and management.
2. Data storage
Viable then puts that pre-processed data from the ingest step into a database we use to store such data, making it easily accessible when we need it again.
3. Pulling data when a question is asked
When a question is asked, Viable pulls what it needs to answer that specific question from the pre-processed dataset and sends it to GPT-3 for analysis.
4. Answer generation
When GPT-3 gets the data for a question, it writes the summary based on the provided context. In the Viable user interface, Viable displays all the relevant customer feedback data points—which were used to generate the answer—along with the answer to the question. You can look through the individual customer feedback one by one to conduct more in-depth analysis.
Pre-tagging data—by topic, sentiment, and emotion—upon import and storing it for future use helps us streamline the entire process, avoiding congestion.
We use OpenAI’s GPT-3 playground extensively to find the prompts that work for our needs. Specifically, we’ve asked GPT-3 to generate answer summaries to questions based on sample customer feedback data points. The sample customer feedback data we use is representative of what product or customer experience teams would analyze for insights. We also provided an example question and example answer. Below is what these samples might look like.
Sample data point 1
Your low code platform is easy to use when I’m trying to build an order tracker for inbound orders, cancelations, and returns. But I want to connect my order tracker with my CRM and email service provider. When will you roll out easy integrations with the main email apps and add it to the order tracking template? I have to copy and paste new contact names manually. Coding a connection into a custom template is a non-starter for me.
Sample data point 2
How can I import data from Google Sheets and from my Gmail contacts? I don’t see a way to connect these for updating new contacts and other data (like demographics). Btw, none of your documentation has instructions for how to import data automatically from outside apps.
Sample data point 3
The project management template doesn’t have a way to add notes to a task column. It only allows me to enter a check mark. I want the option to enter custom text in the cell in addition to a check mark.
How should we improve our product?
We should improve our product by making it easier to connect apps such as Gmail, Google Sheets, and CRM tools. We should make it easier for users to import outside data automatically. Users would also like to add custom text to the task column.
It took us months of testing and tweaking prompts in the GPT-3 playground to find the right structure and good data.
GPT-3 is only one component of the entire Viable system. Developing and optimizing the structure for ingesting data, processing it, storing it, and querying it also took months of work and lots of testing. We spent time building this system from scratch for the customer feedback analysis use case—this was on top of nearly a year’s worth of lessons to draw from thanks to the initial product market fit engine we built.
Viable ultimately quantifies the qualitative. Product managers and customer experience teams are highly data-driven; many struggle because they often rely on anecdotal evidence to estimate which topics matter most. They told us they wanted a way to measure the relative importance of topics across all their customer feedback datasets.
To address this need, we added metrics to each answer we generate:
How often the main topic of each question appears in the dataset, displayed as a percent of all topics
Within the subset of data points, the percent of additional topics also mentioned
Users also wanted to see the top trends in the volumes of customer feedback data. Our Trends graphs visualize top topics, sentiment, and emotion across all customer feedback channels combined.
We hope this overview gives you a sense of how Viable was designed for customer feedback analysis and the role that GPT-3 plays in the overall design.
If you want to analyze qualitative customer feedback at scale with ease, give us a try for 30 days.
Analyze up to 50,000 data points or 30 days of qualitative data. Connect your customer feedback apps, survey results, and other sources of open-ended customer feedback in just minutes.