Donald Kirkpatrick’s four levels of evaluating training effectiveness can be applied to the development of key performance indicators (KPIs) to track strategic performance across organizational Balanced Scorecards (BSCs).
In a not too uncommon scenario, one HR director shared, “Our training courses consistently receive scores averaging above 4.5 on a 5-points scale. We regularly survey department managers to learn what they want. So why did they cut our budget in half this year?”
Donald Kirkpatrick’s four-step framework of evaluation, building on Raymond Katzell's work, "How to Start an Objective Evaluation of Your Training Program," provides steps for answering that very question:
Step One: Determine how the trainees feel about the program.
Step Two: Determine how much the trainees learn in the form of increased knowledge and understanding.
Step Three: Measure the changes in the on-the-job behavior of the trainees.
Step Four: Determine the effects of these behavioral changes on objective criteria such as production, turnover, absenteeism, and waste.
Since its inception in 1956, new titles for these four levels have been adopted and are referred to as:
1: Reaction
2: Learning
3: Behavior
4: Results
The HR Director worked with the enterprise performance management group to create a set of KPIs to track progress against HR's Talent & Technology objective: “Achieve a fully-trained workforce.”
Using Kirkpatrick's framework, the group brainstormed the following KPIs:
Based on their learning from HR team's work, the performance management group realized that the same principles could apply to other strategic objectives. Here are three examples they developed for other strategic objectives on the enterprise’s Balanced Scorecard, using Kirkpatrick’s logic:
The enterprise-level Balanced Scorecard focused on 15 KPIs which measured strategic outcomes for their strategic objectives. Each successive layer below the executive team had fewer KPIs: 12, 10, 8, and 5, respectively. The tables above served as guidelines to help each line or support organization identify their own, critical few KPIs measuring how each team, department, function, or unit supported the layer above them and, ultimately supported the enterprise’s strategic objectives and its associated outcome KPIs. Over time, each organization’s KPIs became more refined to reflect more authentic evaluations of performance using progressively higher Kirkpatrick levels, particularly Learning and Behavior.
Listed below are a few books on measures development worthy of consideration (click on the links to find inexpensive but good hardback copies on AbeBooks):
Evaluating Training Programs, Kirkpatrick and Kirkpatrick
Measuring Organizational Improvement Impact, Chang and De Young
Vital Signs: Using Quality, Time, and Cost Performance Measurements to Chart Your Company's Future, Hronec
The Performance Prism: The Scorecard for Measuring and Managing Business Success, Neely, Adams, Kennerly