Algorithmic Risk Management

The Technological Revolution

In recent years there have been several key technological trends which have converged to stimulate and drive a step change in technological innovation. Computer processing power, data storage and IT infrastructure have continued on an evolutionary trajectory in an attempt to keep pace with the exponential growth in data generation and increasing demand to process and digitise data. The world has become increasingly interconnected with billions of devices now connected to the internet with the scope and application of artificial intelligence (AI) appearing limitless.

In some ways the characteristics of AI are not dissimilar to the coronavirus pandemic in that the technology is proliferating, adapting, evolving and impacting individuals, organisations, markets, society and even the environment. Many of us are still trying to make sense of it, working out how we can exploit it, figuring out how we might need to adapt to simply exist in the ever changing digital landscape.

In recent times AI has also attracted a growing level of scrutiny, debate and fear because there are some difficult issues which need tackling such as privacy, ethical considerations and impact on jobs. Whilst these issues are important, the pace at which this new technology is being adopted appears to be increasing exponentially therefore it makes sense for individuals, businesses and organisations to at least try to understand some of the key underlying principles sooner rather than later. By doing so it may help them to understand opportunities but also help equip them to engage in the broader debate.

Whilst I’m no expert on the subject, the intention of this article is to offer those less familiar with the topic a ‘soft landing’ into the world of AI by defining and explaining a few basic concepts without going into too much detail and trying to use plain English where possible. There is a subset of AI referred to as ‘machine learning’ which is beginning to find new use cases in the construction and engineering sector which will be the main focus. The underlying and ongoing theme of all my articles is how innovation in technology can help create opportunities to improve project performance and outcomes.

Data Science

Whilst it’s true that big data represents a valuable commodity and has commercial ‘worth’ in it’s own right, what in fact makes it precious is the hidden insight and intelligence which can be extracted from it and converted into something useful to support decision making or help with problem solving. The multidisciplinary field known as ‘data science’ is the key that unlocks value from data and can be defined as follows:

“Data science combines the scientific method, math and statistics, specialized programming, advanced analytics, AI, and even storytelling to uncover and explain the business insights buried in data” - IBM

At a more practical level data science starts with the collection and preparation of data which may also involve processing it to improve the quality or transforming it to make it more usable. Certainly within the construction and engineering industry this hasn’t been easy due to a legacy of poor quality data management and data being hidden across different silos. For example, risk data has traditionally sat in MS Excel based risk registers scattered around the business or within a stand alone enterprise risk management system. 

This represents two types of challenges. The first is a technical one which in recent years has been overcome with developments in data storage, IT infrastructure and cloud technology. It’s not uncommon for larger organisations in the sector to have established ‘data lakes’ which extract data out of several core business systems into a central holding area where the data can be easily accessed by other systems. Data from risk management systems, finance, HR, safety and more can be easily accessed. This has subsequently promoted the use of tools like Power BI and Tableau which support improved visualisation of data across the business and are quickly becoming the default reporting tools of choice.

It’s important to understand that ‘data quality’ doesn’t just mean having fields populated and accepted in an enterprise risk management system or risk register. For example, when assessing and quantifying risks on a project the underlying data needs to be relevant, justifiable, accurate and up-to-date for it to be useful. This represents the second challenge which involves improving the characteristics of the data to ensure it’s fit-for-purpose. This can often be a greater hurdle to overcome than implementing new technology. Data feeds both analysis and risk modelling which in turn supports decision making therefore poor data quality can lead to bad choices being made and associated consequences. It’s an important consideration to bear in mind as you read on and a huge topic in it's own right which will be revisited from time to time in future articles.

After capturing, cleaning and transforming the data, the next part of the data science process is to analyse the data. Over the last decade quantitative risk analysis (QRA) has become increasingly popular to help those on construction and engineering projects forecast the likelihood or confidence in achieving cost and schedule targets. This form of analysis is based on a mathematical technique known as Monte Carlo simulation and specialist risk software is used to undertake this. It requires an assessment of the probability of the risk occurring and the cost or time impact ranges usually developed by subject matter experts. Statistical distributions are then (randomly) sampled from and an ‘S curve’ output is usually generated which you can use to infer levels of confidence against costs and dates outcomes.

Whilst a sound mathematical technique is used to undertake QRA, the method by which the modelling inputs are arrived at can be less than scientific and prone to forms of corruption or influence (part of the second challenge mentioned above). Decisions on inputs chosen can be influenced by a range of factors including human judgement, bias, memory and even politics. This can adversely affect forecasting accuracy and has in recent years has attracted much debate and criticism. There are other more ‘top down’ focused methodologies which claim to address some of the above issues such as reference class forecasting (RCF) but personally I feel they offer significantly less insight or intelligence to support proactive risk management, at least on a practical level. This is mainly down to the limited granularity of the data and absence of meaningful analysis from which to understand risks.

Data science offers an alternative and more sophisticated approach to forecasting outcomes. Rather than using parameters derived by human judgement and pumping these into prescriptive mathematical formulae, data science employs a broad range of advanced statistical techniques to identify and explore trends, examine patterns and test relationships across a large volume of historical data.

The data doesn’t have to be confined to risk, cost or activity level data alone which traditional QRA is bound by. In reality the causes of risks and the nature in which they emerge and manifest will be related to many factors therefore having the ability to broaden the data set when attempting to understand, analyse and model risk seems like a sensible and superior approach.

Applying these advanced statistical techniques is a journey of discovery which can help to reveal valuable insights which may otherwise remain hidden, even to people within their respective fields of expertise. The learning from this analysis process is then used to construct predictive models which seek to emulate reality (behaviours and relationships) to forecast project outcomes.

Age of the Algorithm

Data science has been around for many decades and specialist analysis software such as ‘R’, MATLAB and SAS have been popular tools for data scientists. Whilst these tools have supported advanced statistical analysis, a more recent and significant step change has been the emergence and application of AI within the data science discipline:

“Artificial Intelligence is the field of computer science dedicated to solving cognitive problems commonly associated with human intelligence, such as learning, problem solving, and pattern recognition” - Amazon

At a fundamental level AI is simply programming or code, however, what makes it special is that it significantly speeds up and makes the analysis process more efficient through automation which humans are unable to compete with (efficiency wise). One specific way in which it does this is by employing what is termed ‘machine learning’ (ML) which is a subset of AI:

“Machine learning is functionality that helps software perform a task without explicit programming or rules...Machine learning is powered by algorithmic models that are trained to recognize patterns in collected data (such as logs, speech, text, or images)” - Google

The term ‘algorithm’ is a fundamental concept often used in AI definitions which is important to understand, especially since we'll be referring to it regularly in this and future articles. It refers to the mathematical process or set of instructions used to solve problems which defines not only what needs to be done but how (a bit like a cooking recipe). It represents a systematic and logical approach to solving problems which is executed by computer programs.

Algorithms are designed to perform a specific task and as they digest more data over time, they learn and self-optimise with the aim of improving forecasting performance leading to more accurate predictions. Their ability to accurately predict is dependent on the method being used to learn (see below), the volume of data available, a regular supply of new data and it’s associated quality (characteristics described earlier). Determining what level of data is sufficient is usually confirmed through experimentation, testing and robust statistical analysis.

It's very easy to quickly get lost and confused by AI terminology but before moving away from definitions completely, it’s important to have an awareness of the different types of approaches the technology uses to learn from data. This is what sets it apart from traditional methods of analysis and modelling. There are broadly three types of approaches which include ‘supervised learning’, ‘unsupervised learning’ and ‘reinforcement learning’. I’ll refrain from defining these to reduce the word count but would recommend you visit a plain English description which can be found here.

Another relevant subset of AI is ‘natural language processing’ (NLP). This is a computer’s ability to understand and derive meaning from human language (text or voice) in a smart and useful way. It enables the translation of text from one language to another, responds to spoken commands and can summarise large volumes of text.

It’s specifically ML algorithms and NLP which for most of us have become part of our everyday lives. Algorithms usually remain hidden from human sight and interact with us in subtle ways. For example, consider those occasions when you began typing and your words were automatically completed or how online help ‘bots’ answered your queries instantly. We barely batter an eyelid when talking to Alexa or Google in our homes as they move from novelty to our new normality. It’s undeniable that we’re already living in the ‘algorithmic age’ where these algorithms are insatiably feeding off the data we directly or indirectly create through our behaviours, actions and reactions.

ML and NLP have been around for some time and are relatively established across many industries especially in the finance, pharmaceutical and entertainment sectors, however, research has shown that the construction and engineering sector has been one of the slowest to not only adopt this new technology but surprisingly have lacked commitment to future spending (McKinsey and Company, 2018). Some reasons offered are that they appear to lack the capabilities including personnel, processes and tools to implement AI solutions. Other challenges may well include the lack of investment in digitisation of data which is a precursor to being able to deploy the technology as well as having access to a sufficient volume of data for the technology to be able to work it’s magic.

Algorithmic Risk Management

Other industries have demonstrated that there are opportunities associated with pursuing and implementing ML of which the benefits may occur at differing levels within organisations. From a corporate perspective there is a strategic advantage in understanding and exploiting the latest technology ahead of rivals. Early adopters who move fast but take on the risk of failure may also reap the rewards by establishing a strong foothold in a market. For example, suppliers bidding for work may be able to leverage the technology to gain hidden insight over competitors to put together a stronger proposition. They can also demonstrate to a client how they are adopting a more data driven approach to their bid submission which is less prone to bias or inaccuracy.

From a project perspective, exploiting the predictive power of ML may lead to improved confidence in achieving or beating cost and schedule targets by identifying and mitigating the right risks early which may have otherwise remained hidden or influenced by human judgement (see nPlan below). By association, this improved confidence and performance may increase the likelihood of achieving the desired outcomes for stakeholders and clients.

ML led automation can also free employees from repetitive manual tasks which not only increases organisational efficiency (cost, time, less error) but also frees individuals to focus on more value adding and potentially enjoyable activities that humans are better at such as relationships and communication. I know I’ve regularly spent countless hours before, during and after a QRA exercise manipulating data which often feels like “groundhog day”!

Upon reflection of the above and even when performing a casual search for AI on Google it feels like it's only a matter of time before it begins to disrupt almost every industry and sector, possibly to varying degrees. Given the potential significance of the change to come and the rapid rate at which it might manifest I thought it might be useful to define a new term which articulates how project and organisational risk management might evolve:

"Algorithmic Risk Management is an approach taken when projects or organisations chose to leverage the power of data using artificial intelligence to manage risk, reduce uncertainty, exploit opportunities, improve performance and outcomes."

Risk management within the construction and engineering industry has been slow to embrace this new technology but there is evidence to show the tide is now finally beginning to turn. One recent example is where Highways England trialled an AI app on the A14 Cambridge to Huntington improvement scheme (£1.5bn) which sought to identify and predict 'high risk' days (safety incidents). The app was used to record both hazards and good practice and at it's peak 4,500 observations were submitted in one month. Daily risk profiles were then generated based on the data collected identifying which days were higher risk and crucially why. 140 different factors were analysed which could influence risk to workers such as working time, incidents, ratios of supervisors to other roles and schedule activities. The AI pilot achieved 75% success in identifying high risk days. A byproduct of this approach was that by engaging the workforce to report hazards in this way resulted in less harm. Ironically as incidents began to fall the model accuracy fell to 65% as there was less data for it to analyse but it was still 160% more accurate than merely guessing.

In another example Network Rail teamed up with nPlan (an AI software company) to help improve the accuracy of time and cost forecasts for their rail projects. During an initial trial in 2020 Network Rail shared data from two of it's largest rail projects, Great Western Main Line and Salisbury to Exeter Signalling project (over £3bn) for nPlan to test their risk analysis and assurance solution. Their ML algorithms got to work by comparing what was planned against what actually happened on projects at an individual schedule activity level. The trial demonstrated that cost savings of up to £30m could have been achieved had they used their technology upfront. They are now rolling out the software on 40 more projects before scaling up on all Network Rail projects by mid-2021.

These are only two examples of organisations beginning to demonstrate algorithmic risk management approaches. There are some interesting initial observations in terms of how they chosen to proceed with AI adoption:

1. Rather than diving in head first, organisations may feel more comfortable dipping their toes in first by undertaking a small set of AI trials which may showcase potential and offer opportunities for experimentation or tweaking.
2. The time between trial and broader adoption of the technology may be relatively narrow (i.e. 2020 trials to full rollout in 2021 by Network Rail).
3. There is a broad range of both numerical and text based data that can be explored to learn from and build predictive models.
4. The data in scope for analysis doesn't have to be confined to the subject area being explored (in reality relationships exist across many people, processes, systems and stakeholders). Consider the 140 factors Highways England chose to examine.
5. There is a broad range of potential ML applications such as Highways England choosing to forecast over a shorter time horizon and focus on workforce safety verses a longer term (weeks, months or years) for Network Rail.
6. Whilst the technology relies on a large volume of data, clients appear willing to share it when they believe the 'prize' is big enough which opens the door to innovative software vendors. Previously it would have just gathered virtual dust in databases or archived.
7. The implementation of new technology may produce some positive byproducts in terms of peoples behaviours. For example, Highways England staff submitting/reporting more data leading to reduction in incidents. Room for interpretation about cause and effect of course but the principle is sound.

The nature and sophistication of ML analysis techniques combined with automation offer so many exciting opportunities and possibilities especially in the field of project risk management where forecasts and project outcomes have been particularly poor (see my previous article).

Whilst technology alone won't solve all issues which give rise to poor project performance there does appear to be some promising early indicators for success. Whilst the jury is still out on the long term benefits of adopting an algorithmic risk management approach, the above two examples showcase how early adopting organisations are beginning to reap the benefits.

Personally, I remain open minded, excited but also cautious by the nature of this potential change and possibly ‘disruption’ if you choose to call it that. I remain curious about what it means for me as a risk practitioner and how I might need to adapt to remain relevant or what I might need to do to exploit opportunities this new technology presents. I previously outlined the project need case for change and in this article I've provided the technological context for change. Future articles will begin to scratch beneath the surface to explore some of the opportunities and challenges that await us.

As always I welcome your thoughts, comments or opinions and ask that we remain respectful of differing perspectives.