our research and learning

To pursue our quest of making return on investment a manageable task for customers, we researched other writers and specialists on the subject. Our first influence was Kirkpatrick (1994). He described how, when evaluating a training activity, an assessment should be made at four levels:

• Level 1 involves assessing learner’s reactions to the activity. Relatively simple and cheap to assess (usually in the form of evaluation questionnaires or feedback forms)
• Level 2 assesses what learners have learned (in terms of knowledge, skills and/or attitude) as a result of participating in the activity.
• Level 3 investigates behavioural changes, or the extent to which the learner has applied the newly acquired learning in the workplace.
• Level 4 assesses the benefits of training at an organisational level rather than an individual level, presented in terms of bottom line benefits easily recognisable to senior management.

The model gradually shifts in focus from the learner (Levels 1 and 2) to the organisation (Levels 3 and 4). As they are the most difficult and time-consuming to conduct, the latter levels are often neglected. Yet these are the levels that yield the most valuable data and can provide a thorough justification for future training expenditure.

Jack Phillips (2003) approach to the evaluation of learning activities builds on the work of Kirkpatrick, whom he openly credits with ideas for developing the first four levels of his own framework for evaluation. The significant difference is that Phillips has renamed the first four levels,  and has added a fifth to capture return on investment.

• Level 1 Reaction, satisfaction and planned action: measures the extent to which the participants enjoyed the learning experience and establishes how they plan to apply their learning.
• Level 2 Learning: measures the extent to which the participants have increased their knowledge and improved their skill levels.
• Level 3 Application and implementation: measures the extent to which the participants have applied what they learned from the learning event to their working practice.
• Level 4 Business impact: measures the results that the participants have achieved as a result of applying what they learned from the learning event. This would include measures such as output, quality, cost, time, customer satisfaction and employee satisfaction.
• Level 5 Return on investment: measures the level of monetary benefit against the overall cost of the learning event and then compares the two. Usually expressed as a percentage or cost benefit ratio.

John Darling (2006) developed the concept of embedding learning as an important phase of learning, suggesting that the better distribution of attention in the process moves from the actual training event to both the “before” and “after” phase. He showed the traditional models as seeing the training as the cornerstone of the process, whereas his model shows it as just another phase in several required phases.

Dr. Elwood (Ed) Holton (2000) has written extensively on the factors affecting the transfer of learning as a result of training activities, and has developed a practical model called the Learning Transfer System Inventory. This model identifies a number of potential enhancers or barriers to the practical application of learning, which include motivation, environment, ability and readiness. This model showed us that the transfer climate was a factor that was often missing from learning initiatives which, if not addressed, would significantly affect the results. Learning transfer has become a key phase in our PIC© and a very important part of the holistic process.

Paul Kearns (2005) in his book Evaluating the ROI from Learning, has identified the optimum way to measure value from development activities on the basis of his three box system, with the middle box - value added training - as an area in which organisations should be pursuing the actual value. However, he puts much of the soft skills development, such as leadership and management development, into the box “nice to have” and showed how this area of development could be measured if evaluated in a more systematic manner.

These are some of the key influencers to our work and this learning helped us to develop the Performance Improvement Cycle© as described in the next Chapter.

(this is an extract from my book 'From The Happy Sheet To The Bottom Line'.