Using systems thinking to assess learning needs

Learning (or training) needs assessments (or analyses) are the elephants that are not in the room for HRD. Elephants, because they are big issues that need to be addressed, and not in the room because often they are not be being done particularly effectively. The picture here is confusing. Anecdotally, people I come across in the learning and development field talk about them rarely being done properly, data from the Association for Talent Development claims that 56% of organisations claim to carry out some form of needs assessment, and that these are on the whole ‘thought’ to be effective,[i] but academic research seems to indicate that fewer than 10% of training programmes are supported by a needs assessment.[ii]

Some of the problem lies with knowing how to approach a needs assessment. In recent times I have been working on how to use systems thinking ideas, in particular those aspects of systems thinking which make us question how decisions about training are made. The systems thinking literature talks about decisions as ‘boundary judgements’, and one of the best-developed tools for examining boundary judgements is Critical Systems Heuristics (CSH), developed by the Swiss systems thinker Werner Ulrich.[iii]

A first step is to think about learning to address a perceived organisational problem as ‘a system’, for example, as “a system to develop the necessary knowledge, skills and attitudes needed to achieve X”.

We can then use CSH to help us understand how this system might work. CSH is based around 12 boundary judgement questions, which when answered can help us to understand the system we are thinking about more completely. There are four groups of questions related to motivation behind the system, its control, the expertise that it needs, and its legitimacy. In each of these groups are three questions covering who is involved, what their involvement is, and what their involvement might mean or need.

To explain this more clearly, it will be best to work through an example of how CSH might be used in a generic needs assessment process. The table below shows what these questions might be, but it is important to realise that these questions are not developed sequentially, but iteratively. As we answer each question we may think about what implications that has for other questions, so that the table fills in slowly and somewhat randomly. As a result, there is no one particular order in which to work through the questions, however a good place to start in a needs assessment is Question 2, “What is the purpose of the learning system?” Then see what thoughts that generates and move on from there.

1.      Who should benefit from the learning system?

• Who needs to learn the required KSA?
• Who else in the organisation will be affected by what people learn, and so also need to be involved in some way?

2.      What is the purpose of the learning system?

• What are the learning outcomes for individuals?
• What are the desired outcomes for teams where the individuals work?
• What are the desired outcomes for the organisation?
• What might the learning system change that we do not expect?

3.      What is the measure of success for the learning system?

• What are the measures of success? For individuals, teams and the organisation?
• How will we measure this success?

4.      Who makes decisions about the system?

• Who makes decisions about how learning happens?

5.      What resources and constraints relevant to the system does the decision-maker control?

• How are decisions about learning activities usually made in the organisation?
• Who holds the budget for learning activities?

6.      What conditions of the system’s operation are outside of the control of the decision-maker?

• What is the learning transfer climate in the parts of the organisation where learning will happen? How receptive to new ideas are people?
• How will new KSA interact with existing policies, practices, conditions of employment, etc?
• How do informal learning channels operate within the organisation?
• What group dynamics may support or hinder applying new KSA?

7.      Who should be involved in the design of the system?

• Who will design the learning programme?

8.      What expertise is needed to design the system?

• What expertise is needed for designing the learning programme, both in terms of learning design expertise and subject matter knowledge?

9.      What guarantees of success do the sources of expertise offer?

• How confident are you about the learning design skills available?
• How confident are you about the subject matter expertise available?
• What level of agreement is there that the new KSA needed will actually improve the organisational problem being seen?

10.   Who will represent society and the natural environment acted by the system?

• Who outside the organisation will be affected by individuals practising new KSA? For example, customers or communities.
• What impact will the new KSA have on the natural environment, and who will be affected by this?

11.   What opportunity does the affected society and natural environment have to challenge the system?

• What consultation with external individuals or organisations might be appropriate or advisable?

12.   What is the worldview underlying the design of the learning system?

• How open are you to alternative forms of learning (formal, informal, on-the-job, off the job, communities of practice, and so on)?
• How confident are you that learning is a valid solution to the perceived organisational problem?

 These are not the only or the definitive questions, but can be seen as a good starting point for the process. To me, the most interesting questions are the ones which challenge the assumptions that may easily be made in a more cursory approach to needs assessment: for example, how confident are we that the recommendations for new KSA are reliable, and the importance attached to thinking about the external implications of changes in organisational practice. Too often there is a reliance on the opinions of (not necessarily well informed) senior management about what and how learning should be designed and delivered.

I am using CSH as a key part of a process for designing learning strategies that can help organisations operate more sustainably. The whole process is explained in more detail in a forthcoming book, “Learning strategies for sustainable organisations”, that will be published by Routledge in 2022. There is a little bit more information available about this on my website, http://www.bryanhopkins.co.uk/learning-strategies-for-sustainable-organisations.html.

References

[i] ATD, ‘Needs Assessments: Design and Execution for Success’ (Association for Talent Development, 2018).

[ii] Winfred Arthur Jr et al., ‘Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features.’, Journal of Applied Psychology 88, no. 2 (2003): 234.

[iii] Werner Ulrich, ‘Critical Heuristics of Social Systems Design’, European Journal of Operational Research 31, no. 3 (1987): 276–83; Werner Ulrich, Critical Heuristics of Social Planning: A New Approach to Practical Philosophy, 1983.