Role IQ Learner Research

Challenge

Pluralsight is an online learning platform for technology professionals. After releasing a beta version of the assessment product, the Role IQ product team, had a chance to investigate how customers were adopting the new offering and identify areas of improvement.


Approach

As Product Manager, I led a team that included myself and a UX Designer. The findings were used to help inform opportunities that the team would work on. We used a mixed-methods approach that included learner motivation and value surveys, semi-structured interviews with 24 learners, ad-hoc data analysis, and learner journey mapping.

Key Research Areas

– Gain insights into the original mission/vision for the Role IQ product line
– Understand how learners are interacting with Role IQ and how interactions differ between cohorts (completers vs. bailers vs. B2B vs. B2C)
– Identify areas of improvement that would be most valuable for learners
– Identify any areas of friction in the learner experience

Surveys to understand learner motivation and value

The team sent out two rounds of survey questionnaires to learners who engaged with Role IQ to better understand their motivations and determine whether they have received value from the product. The survey responses indicated differing motivations between different cohort groups (B2B vs. B2C vs. bailers vs. completers).

Semi-structured interviews to understand learner experience

The team conducted 24 interviews to understand how the product experience difference between cohort groups. The interviews allowed us to gain a deeper understanding of learner’s backgrounds, motivations, and experiences.

Affinity mapping to identify patterns

After completing the interviews, the team conducted virtual affinity mapping sessions to identify common themes.

Affinity Mapping for Semi-Structured Interviews

Data analysis to identify usage patterns

Throughout the research project, the team conducted ad-hoc quantitative data analysis to understand usage patterns among the cohort groups better. This analysis allowed the team to answer questions, including:

  • How are learners arriving in the product?
  • What actions are learners taking after engaging with Role IQ?
  • What behaviors are learners taking within their first visit to the product?

Crafting a journey map artifact

The team synthesized our findings into a learner journey map. The map allowed us to identify different phases of the user journey with considerations to the actions they would take, their emotional experience, and identify opportunity areas.

First Time Learner Journey Map

Opportunities

Based on feedback from learners, The Role IQ team determined that they could build upon the existing foundation of the product to provide additional value to learners. 

There was an immediate opportunity to improve clarity for Role IQ learners, with a direct focus on:

  • Increasing clarity around the purpose and value of Role IQ
  • Providing learners with guidance (clarity and support) on how to skill up in a role
  • Providing Role IQ learners with context about how skills relate to a role (relevance and importance of a skill in a role)
  • Providing clarity around Role IQ levels

Outcome

The findings from this study were used to help inform the development of future features in the Role IQ product. As of June 2019, the team released an onboarding experience in addition to adding UX copy to increase clarity around the purpose and value of Role IQ.