Click here to view the analysis presentation.

About The Predictive Index

The Predictive Index is a talent optimization company that businesses use to hire, inspire, and retain talent inside of their organizations. A part of their product offering are instructor-led training classes that introduce the methodology and concepts behind their workplace assessment tools.

The Challenge

After conducting a 9-month pilot of a virtual version of it’s training program, the learning team at The Predictive Index noticed satisfaction scores of the new offering were lower than it’s in-person format. The company partnered with a group of master’s students from Boise State University to investigate the cause for the low satisfaction and identify potential areas of improvement.

The Team

As project lead, I led three other masters-level students through research and analysis of the program.

Key Research Questions and Areas

The analysis focused on four areas, including, program design and delivery, learner application, completion rate, and sales/marketing with the fundamental questions being:

  • Is the program meeting its intended outcomes?
  • What improvements could be made to the virtual training programs?

A Mixed Methods Approach

The team used a mixed-methods approach that included learner surveys, semi-structured interviews, ad-hoc data analysis, literature review, and observations.

Learner Surveys: The project team distributed a web-based survey to learners two months after their completion of attending the virtual training.

Semi-Structured Interviews: I conducted interviews with key stakeholders inside of the organization to gain a better idea of the current state of the program as well as the intended business goals. Based on feedback from the interviews, I developed a graphic to show the inputs and results of the program.

To get a better picture of the training program, the project team reviewed reaction data from learners who attended the virtual training, in addition to reviewing course enrollment and completion data.

Literature Review: The team conducted a literature review of 20 academic articles to understand best practices used in the delivery and design of virtual training programs. The findings were used to create an observation checklist that was used to evaluate the current state of the program.

Observations: Using the observation checklist, the project team observed four virtual training sessions and rated various elements of the program design and delivery.

Example Observation Sheet

Data Synthesis: Following data collection, the project team created rubrics, scored data, and created a dimensional triangulation rubric. The team drew evidence-based conclusions and recommendations using a synthesis rubric.

Overall Themes and Opportunities

The team determined that the overall quality of the virtual program was high. Overall, the program is headed in the right direction, but some gaps should be addressed. The research team provided The Predictive Index with a list of prioritized recommendations (see below) related to the different program dimensions:

Research DimensionsRecommendations
Program design and delivery
Determine the overall strategy of the virtual training program. For example, is it just an online version of the existing instructor-led training, or does it have another mission and purpose?
Learner application
Develop a level 3 (application) survey that is delivered to learners after a specified period (3 months) after completing the program to determine whether they have applied what they learned when they return to the workplace.
Completion rate
Develop a customer-facing website that makes self-registration for learners an easy one-stop-shop. The site should contain a calendar of training events where customers can see availability and register for events.
Sales and marketing strategy
Adapt the marketing strategy to indicate the learning outcomes learners can expect to receive based on their attendance.

Business Outcomes

As of 2019, the recommendations that the research team made, helped to inform The Predictive Index’s “digital first” learning products. The Predictive Index has seen a YoY increase in revenue and customer engagement in their learning experiences.