Archive for November, 2010

Assessing Outcomes and Assuring Impact from Learning and Development Investments

November 24, 2010
Last week Thursday I had the opportunity to attend an alumni event offered by my university: a lecture by Dr. Ellen Pruyne, Ed.D, formerly of Harvard and MIT. The lecture was all about ‘Assessing Outcomes and Assuring Impact from Learning and Development Investments’.
Dr. Pruyne invited us to look at such assessment using the Kirkpatrick framework. If you haven’t come across it before, it is basically about looking at evaluation in four stages:
1. Did participants like/enjoy the course experience?
2. What did they learn?
3. How did they apply their learning to transfer it into new behavior?
4. What outcomes/impact occurred as a result of the learning event, subsequent reinforcement and application of learning?
A lot of evaluations apparently tend to get stuck already at stages 1 and, at best, 2.
A good summary of the Kirkpatrick approach is on this website (worth reading page through to the end):
A few more useful observations by Dr. Pruyne:
Training objective: help participants change attitude, skills, behavior, role.
Useful question for evaluators to ask themselves from the outset: “If the program was successful, what would that look like?” (describe that to ourselves as graphically as possible, for each of Kirkpatrick’s four stages)
Useful for course planners to keep in mind, especially re. stage 1: What matters to participants is their perceived ‘return on expectations’ (ROE).

At the same time, ultimately important is: emphasis not just on learning and individual development, but on learning and organization development – how will the organization be impacted because of the individual’s learning? (value contribution)

Accordingly, ultimate question to ask: What is the return on learning and organization development investment?

I found this helpful and offering food for thought around how I plan my own trainings – perhaps it can be helpful for some of you, too?