The Evaluate Phase
In this Phase, you attempt to answer the question "Was the course effective?" As part of the Training Plan you create in the Design Phase, you should list which Kirkpatrick Levels of Evaluation would be utilized for your project. And no, the correct answer isn't always "all of them!" The levels in the Kirkpatrick model are Level 1 - Reaction, Level 2 - Learning, Level 3 - Behavior, and Level 4 - Results (Kirkpatrick, 2020).
Kirkpatrick Level 1 - Reaction
While the graphic above does a good job of breaking down each level, I would posit that Level 1 includes much more than just enjoyment. As instructional designers, our goal is not to entertain, but to solve learning gaps. As such, your Level 1 should include opportunities for qualitative or quantitative feedback on the whole learning experience, including how effective they felt the course was.
Kirkpatrick Level 2 - Learning
This is 'the test.' It is the summative assessment at the end of the lesson, module, or course that measures a learner's ability to perform against the Lesson Objectives. "Kirkpatrick Level 2 is a key measure of success for the majority of corporate training. It’s not the only measure, but it’s critical to achieve success at Level 2 in order to have success at other levels"(Tribal, 2018, p.1). I would hypothesize that this would encompass not just corporate training, but scholastic learning as well.
Kirkpatrick Level 3 - Behavior
Behavior can be measured in the classroom, but ideally, it is best observed and measured back on the job. Remember the forgetting curve and the 70/20/10 theory? If you can observe the desired behavior days, weeks, or months after the learning event, you can likely surmise that your learner retained the information or skill. How can you evaluate behavior? Try these techniques (Peak Performance, 2020):
-
Conduct learner observations while back on the job
-
Interviews or focus groups discussing their behavioral change on the job
-
Learner self-assessment.
-
Secret shoppers
-
Quality observations
Kirkpatrick Level 4 - Results
The golden question here is "Did the training influence performance?" While this may sound similar to the behavior measurement in Level 3, the focus of a Level 4 Evaluation is to measure performance metrics. Those metrics may be measured at the individual level, at the organizational level, or both. Performance metrics can run the gamut from sales goals to secret shopper scores, to profitability, and anything in between. If a Level 4 is required for your course, these specific performance metrics must be identified in the Analysis Phase as they will impact your Lesson Objectives.
Image Courtesy of KodoSurvey.com
Considerations on Kirkpatrick Levels 3 and 4
Not all Kirkpatrick Evaluation Levels should be completed on every course you create. "Levels 3 and 4, which arguably yield the most useful information for the business, are time-consuming, resource-intensive, and expensive to implement" (Mindtools, n.d. p.1). In most instances, unless the organization or client specifically asks for a Level 3 or 4 evaluation, most eLearning courses, due to their shorter length and limited content, can be evaluated through Levels 1 and 2.
Self-assessment!
How well do you know your Kirkpatrick Levels of Evaluation? How about an ADDIE-tude check with the ten-question quiz below? Don't worry, you can take it as many times as you like!
Resources
Kirkpatrick Partners. (2020). The Kirkpatrick Model. https://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
Mindtools. (n.d). Kirkpatrick's Four-Level Training Evaluation Model. https://www.mindtools.com/pages/article/kirkpatrick.htm
Peak Performance. (2020). Kirkpatrick's Level 3 Evaluation. https://thepeakperformancecenter.com/business/learning/business-training/kirkpatricks-four-levels-evaluation/kirkpatricks-level-3-evaluation/
Tribal Habits. (2018). Measuring training understanding Kirkpatrick Level 2 Learning. https://tribalhabits.com/measuring-kirkpatrick-level-2-learning/