Customer Behavior Analysis - Support Messages Evaluation

As part of our quest for insights for our client—a major brand in the creative products sector—we decided to analyze the messages users send to technical support, along with the responses from support specialists.

Our expectation was to uncover patterns and insights that could improve the performance of technical support, detect product quality issues early, and ultimately enhance overall business efficiency.

In the initial testing phase, we developed a script to extract all messages from the technical support system (Zendesk) and processed them using Machine Learning techniques for text analysis (NLP). Additionally, each message was linked to actual orders using client data, identifying the specific products the customer had purchased before contacting technical support, to generate product-based statistics.

All messages were categorized into the following two groups:

1. Problem — related to product quality or order issues.
2. Question or suggestion — related to product usage or inquiries about other brand products that could complement the purchased item).

We identified the following key metrics:

• Sentiment Analysis of User Messages: Sentiment scores were on a scale from 1 to 5, where 1-2 indicated negative sentiment, 3 was neutral, and 4-5 indicated positive sentiment.
• Tone of Support Response and Perceived Resolution: Whether the issue was resolved or not, from the client's perspective.
• Customer Order History Before and After the Support Interaction: An additional crucial metric for evaluating support effectiveness.
• Support Response Time and User Satisfaction: Measuring the speed of response and the user’s feedback on the support experience.
• Common Phrases and Topics: Identification of typical issues in customer inquiries or suggestions.

Based on the data, we established correlations and provided the following reports:

• Customer Interaction with Support by Product + Sentiment Analysis: This report helped identify products with more frequent issues, highlighting areas requiring attention to product quality.
• Product Quality Deterioration Report: A comparison of three-month intervals, showing an increase in support messages or a decline in sentiment, indicating potential product quality issues.
• Technical Support Team Efficiency: An analysis of how effectively support interactions improved the emotional tone of customer messages, indicating customer satisfaction with the resolution.
• Customer Loyalty: Assessment of support effectiveness from the perspective of repeat purchases. This report analyzed how likely customers were to make subsequent purchases after contacting support and the reasons behind it.
• Identification of Common Phrases and Questions: Aiding in the creation of standardized responses and improving product descriptions.

The research results, presented in reports along with our recommendations, enabled our client to:

• Optimize the Technical Support Department: Introduced a new KPI for evaluating support responses, based on how customers reacted to responses and whether they placed orders after interacting with support.
• Improve the R&D Department's Efficiency: Enabled quicker detection and response to product quality declines.
• Enhance Marketing and Content Department Performance: Improved product listing descriptions on Amazon and the company’s website.
• Bundle Creation: Facilitated the development of product bundles based on customer inquiries and suggestions.