Innervation: The structure for learning and evaluation

Conducting a meaningful evaluation of learning has long been a bugaboo in our profession. And it seems this vexing issue is one we have not grown tired of. That’s my conclusion based on a large number of attendees at the recent LDA Insight session.

In this session, Will Thalheimer reviewed the history of the Kirkpatrick/Katzell (K/K) model and offered a critique of his LTEM model. The K/K model moved evaluation forward by acknowledging, through levels 3 (“behavior”) and 4 (“performance”), that training is not just about learning but about performance and results. Its weakness is that it does not guide how learning, and supports for learning, enable quality performance and outcomes.

Will’s Learning Transfer Evaluation Model (LTEM) emphasizes key ingredients for successful transfer of learning. It provides measurement prompts for assessing learners’ understanding, remembering, and application of learning as reflected learners’ ability to recognize when to apply their learning and do it competently. It also reminds us that learners may need guidance and support to apply what they have learned and that they should progress toward unguided performance.

I like Will’s thinking about models. He states that while no model is complete and perfect, good models guide us toward taking more effective actions. Despite the LTEM’s qualities, he critiqued the limitations of his model and offered a glimpse of a new one that is under development. The new model comprehensively maps many factors that influence learning and performance. Only a small subset of these is addressed by either the K/K or LTEM models.

I like thoroughness, but I am wondering about the practical real-world value of such a comprehensive model. This is not the fault of the model but, rather, the design of the environments they describe and the roles learning practitioners play within them. For instance, how does a model that identifies systemic and cultural variables affecting performance help a designer assigned to create training on, for example, a specific policy or tool? As we add complexity to an evaluation model, we begin mapping the organizational system. This can feel defeating to an instructional designer because they are not in a position to influence key parts of the system.

Incorporating these factors into our evaluation models can influence the thinking of others, however, this still seems like a strategy of hope. The problem, to me, is that learning is a separate function, a separate activity away from work instead of within it. We can keep building better evaluation mousetraps but we are not addressing the habitat that produces the rodents. (Sorry mice. Nothing personal. That’s just the best analogy I can come up with.)

Let’s try a different analogy that I think points in a productive direction. Think of learning and evaluation as an organizations’ nervous system. Nervous systems can anticipate experiences and initiate automatic responses to experience. Given some time and some good rules for thinking, these systems can solve problems for current and expected conditions and shape reality through creativity.

In many workplaces, our nervous (learning) system lacks innervation. In a body, strength, agility, power, and flexibility are products of extensive innervation. We limit those organizational attributes if we have rudimentary and sparse learning systems, constrained by limited input, feedback, and coordination with the entire organization. To complete this analogy, we need to grow the learning “nerve pathways” that support the organization's health, performance, and growth.

How do we do this? I have a stem of an idea. I’ll think about it and post later. In the meantime, do you think the nervous system analogy makes sense? If so, where could we begin to design an analogous system that could take full advantage of a complex model for evaluating learning? Please share your ideas in the comments.