USER RESEARCH
To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study.
2 years back, I was conducting user research a messenger application. I was the only designer and the product was beginning to run in an agile methodology - 3. I had strict deadlines and I didn’t have the luxury of an amazing research budget.
After getting approvals from multiple layers in the organization, I first started out by reaching out to the internal user. My strategy was to roll out the application internally, so that the employees use it first. For the actual customers , my logical decision was to recruit users from the existing user-base and use whatever budget was available only for incentives.
The path I chose - Enter research panels.
Identifying users - Reached out to the customer support team:
We wanted to find out who exactly the messenger application's customer is. Data was telling us very little about them. Our numbers only told us how much our customers are paying us, how big their team is, and how long they’ve been doing business with us. And their conversations with our support team only told us about very specific problems they were having. We wanted to know more about the problems they were firefighting every day, what their roles looked like, and what mattered the most to them.
Recruiting UX research participants felt like a massive task. But when I understood the company’s structure and the client base, it made sense to reach out customer relationship managers (CRMs).
This allowed us not only to track the contact information for the participants, where they are in the process, and various communications, but also to add notes and file attachments.
We gathered the list of managers who could provide us access to their team members to participate in the research session.
Identifying users – Reaching out to the customers with a survey
Conducted a mini survey to capture the user types and to understand the NPS (to capture the quantitative data) . I felt NPS surveys are a good place to get hold of customer names. I got some info from the In-product surveys to get participants for my research panel.
Covered the spectrum of users:
Based on the survey info, we came up with a list of customers/end users who could become a part of research panel.
-
Customers who are end users and who use the product frequently
-
Customers who are end users and who don’t use the product frequently
-
Customers who are purchase decision makers
-
Important customers from companies who bring in a lot of revenue
-
Customers who fall under the “Promoters” NPS category,
-
Customers who fall under the “Neutral” NPS category
-
Customers who fall under the “detractors” NPS category
-
Customers who have joined recently
-
Age
-
Geography
-
Department
Challenges I faced in this process
Customer relations team are generally very protective about their customers, especially the ones which rake in the most revenues.
But I made sure to build rapport with the customer relations team and explained the process.
After identifying the participants it was prudent to get a final approval from the product owner and the CRM team, so that panel invitations were not sent inadvertently to customers who have a “do-not-disturb”, have recently cancelled, is in the process of cancellation or in the process of renegotiating the product contract. Once I got the final approval I sent invitation email with NDA.
Reaching out to the users - Emails
After multiple levels of approvals, I had the chance to write mails to the managers about who we are, what are we looking for, how can we help them and how much time would we need?
Email iterations:
Considering the user types and their roles, we composed email (3-4 iterations) to provide info in a crisp manner.
We kept it plain text; we kept it simple. By using their names in the subject, we were able to make a personal connection and get their attention.
Here is a sample we used to email customers asking for permission to talk to us.
“Hi [First name],
I’m _____ from the K Messenger’s user research team. I’m reaching out to some key customers to get feedback on their experience with our product.
What I’m looking for is honest feedback – the good, the bad and the ugly – so we can go back, work with the team and improve how K Messenger works for you every day.
If you’re willing to have the conversation, can you give me around 30 minutes over the next couple of weeks that would work for you?
It’d be awesome if your team members who use K Messenger can join the call as well. Looking forward to hearing from you!”
Building a research panel
At this stage, we felt we need a system to reach out to the customers in regular intervals to capture the data. Why a research panel?
-
Allowed for greater insight
-
Provided better response rates
-
Helped in data-driven decision making
-
Easy access to research participants
-
Provided continues engagement
-
Amazing recruitment cost savings
-
Demonstrates a “we care” attitude for customers
Pre-interview preparation
I partnered with PO and a technical writer- prepared guides and required material for the user interviews.
I felt it was important to do my homework on the participant. I had sessions with customer relationship managers(CRM) to pull out previous feature requests and NPS survey comments.
"Nothing makes a customer more frustrated than repeating what they requested a few months back."
Prepared guides to make it easier for my partners to capture the data.
Assigning roles
To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information such as names of the team members
Pre-interview guide
Focus and time boxing
Note taking guide
After the session
-
After the research sessions, we sent a thank you note to the participants within 24 hours.
-
I made sure to pass on important feature requests to the product team as soon as possible.
-
We reviewed the session recordings so we do not miss out on any important information
-
We wrote down the key points of the session with each participant and sent it via email at the end of each day, along with the link to the recording so that the team has early visibility.
-
I took a week to compile the final report and present it to the product team. This helped the team to start work in the background in the truest spirit of agile and they dint have to wait for the final report.
What did I learn
-
Customers get scared if we mention ”panel”. The general perception is that research sessions take lots of time.
-
Personalizing email makes a lot of difference.
-
Best is to get the sales representatives to send the panel invitations with me in CC. This saved a lot of time and the chances of getting a response from the potential participants was much better.
-
One-on-one sessions were much better that the focus groups, as generally someone with the strongest voice generally dominates the conversation and moreover it leads to other participants getting biased.
-
Research panel participants gave lots of feedback. Running a user research panel is a full-time activity in itself
-
Managing the research panel: There were people changing jobs, dropping out as customers and plain disinterested. Also, the first time we interviewed a participant we got an idea if the person is responsive. I made sure to note my observations and revisit the panel members list from time to time. We sent an email with updates regarding the product road-map to keep the panel members interested and engaged.
-
Persuasion and perseverance pay rich dividends: When I reached out to the CRM, I got a blunt “we don’t have time for this” when we reached out initially. I prepared flow charts of how the user research panel would work and spent time with the heads of CRM to get their buy-in.