10 reasons not to like the COSO ERM framework a discussion with Grant Purdy
Grant Purdy is a highly respected, veteran risk practitioner based in Melbourne, Australia. He chairs the committee that developed the excellent Australia/New Zealand 4360 risk management standard and has been an active and influential member of the global team that gave us the fine ISO 31000:2009 risk management standard.
In this post, I will share the results of talking to Grant about the merits (mostly the failings) of the COSO Enterprise Risk Management Framework.
Grant told me that recently, after many years, he re-read the COSO ERM framework again, from cover to cover, and discovered that some of his thoughts on its weaknesses have become distorted and exaggerated over time by reading what others had said about it and, in particular, how others have misused the materials to support their own paradigms and agendas. For example, he was convinced that the COSO ERM Framework required risk reporting and did not put enough emphasis on treatment because that is the message that had been put out by many of the large consultancies.
Grant believes that the COSO product has a number of good points but that overall he finds it complex and unwieldy, and can clearly see how many companies would just give up and pay someone to tell them how to implement risk management. He also thinks the cube and the need to keep some alignment going with the Internal Control Framework diagram compromises the flow of the processes given there.
Then, Grant notes there are some big technical flaws that will mean the process being followed will always be deficient and inefficient.
1. “When identifying events, the code mentions external factors; but the majority of the discussion is focused on internal factors, systems, culture etc. The COSO process starts with the internal environment, not the external ones and this fails to reflect the influence that the business environment, regulatory conditions, and external stakeholders have on the risks an organisation faces, its organizational culture, and how they influence its risk appetite and risk treatment priorities. This can easily lead to organisations just focussing inwardly and not actively identifying risks that reflect external factors and circumstances.”
2. “Stakeholders, particularly external ones, are not mentioned and stakeholders’ objectives and their influence on decisions about the significance of levels and types of risk are omitted. This is a critical omission and means the organisation effectively insulates itself from external opinion and stakeholder objectives. Most of the risks we face are caused by an incompatibility between stakeholders’ and our own objectives.”
3. “COSO ERM says that risks are described as events, and events are described and illustrated by examples of sudden, acute occurrences. There is no appreciation of the slow changes in circumstance and situation (for example a deterioration in internal culture or market sentiment) that give rise to some of the most critical risks.”
4. “COSO measures risk in terms of the probability of an event and its “typical” consequences. However, we will not always get the “typical” consequences every time an event occurs. For example, not every time my house is hit by lightning will it burn down. If I estimate the level of risk as the product of the likelihood of the event (being hit by lightning) and the worst consequences (losing my house), I will overestimate it. In fact, there are multiple possibilities: my house could be hit and not be damaged; it could be hit with slight damage; and so on all the way up to being burned down. Each potential consequence would have a different likelihood of occurring.
“Of course, this all sounds rather academic until you actually observe how, in workshops and in life, people who follow the COSO code use a rating system to estimate the level of risk; and they always seem to get it wrong and omit the conditional probabilities that should be applied to the event probability. This means that they always overestimate the level of risk, which prevents individual risks being properly distinguished and compromises any realistic modelling of the effectiveness of controls. The COSO approach to estimating the level of risk reduced the credibility and usefulness of the risk management process because significant consequences are predicted to occur much more frequently than is credible based on historical experience.”
5. “Throughout the document, the term ‘risk likelihood’ is used, but risk does not, per se, have a likelihood. Likelihood is one of the attributes used to measure the level of risk. This is a philosophical trap that can lead the unwary to see a risk as an event and then to use language such as “when the risk occurs”. Risks don’t occur when events occur, risks only ‘exist’ whenever we make objectives. If there are no objectives, then there are no risks. The level of risk (not risk) is described in terms of what can occur (consequences) and how likely they are.”
6. “While there are some concessions to what are called ‘opportunities’, in COSO ERM risks are mostly about losses and risk treatment (response) is about reducing the likelihood and severity of losses. The thinking in the COSO document is not mature enough to appreciate and explain that risk is just the effect of uncertainty in what you set out to achieve and that outcomes can be beneficial, detrimental or both. Certainly, the document does not promote taking risks that have beneficial consequences because you are confident you can treat or tolerate any potential downsides which is, after all, the basis of enterprise: the undertaking of risk for return.”
7. “I find the whole thinking about ‘risk responses’, ‘control activities’ and ‘monitoring’ most confusing and confused and I think most people who read and try to use the code do as well. For example, if you institute an audit regime, this is a good form of risk treatment to reduce the likelihood of unfavourable consequences. However, audit could be required as a matter of policy, could be part of a management process, and could also be part of a monitoring strategy. ISO 31000 clears this all up. Risk treatment refers to the actions you take that lead to the creation of and improvement in controls, and controls are what you employ to modify risk. These controls then require monitoring and review by assurance processes. That’s it.”
8. “The problems with the concept of inherent risk are well-known and the COSO document does not explain why you need to use this artificial, theoretical state where no controls exist, to justify tolerating the present level of risk or doing something more to modify it. In risk analysis it is useful to understand what worse-case consequences could occur if existing controls fail so that we can focus our assurance activities on checking those controls, but this is best dealt with by using the Potential Exposure (inherent consequences) value that does not require any consideration of likelihood.”
9. “The whole area of risk appetite and what COSO ERM calls risk tolerance is handled in a mechanistic and naive way. The thought that before you even do a risk assessment, a board can identify the material risks and tell you how much they are prepared to tolerate puts them on a par with the Gods. What this means in practice is that some Boards may have the ability to think about different types of consequences (not risks) and in some cases they can say how much loss they are prepared to sustain over a period of time compared with the balance sheet and cash flow of the company. However, these are not measures of risk, they are only measures of consequence. For non-monetary consequences, the statements that Boards can make start to get very vague. For example, they might say they never want to kill someone, but they will rarely want to agree on what individual risk of fatality they are prepared to expose their employees or neighbours to.
“Following the COSO prescription of taking these measures of risk appetite and applying them to assess at which level of risk you stop risk treatment is idealised, unrealistic and, in some states, may be illegal. Cost benefit, which is mentioned in relation to ‘response’, is the only way to make this determination, even when it comes to emotive areas like public safety.
“The material in the COSO ERM Framework on risk appetite has led to greater confusion and more wasted consultancy dollars than any other part of the framework.”
10. “The greatest sin – and I’ve left this till last – is that the COSO document confuses and mixes up the framework (the organisational structures, policies, and arrangements put in place to promote, integrate and improve the management of risk) with the process used for risk management, particularly that used for risk assessment, risk treatment and monitor and review. They need to be thought of separately where the framework operates at an organisational level while the process is that which the framework seeks to integrate into all critical organisations processes where decisions are made.”
What’s your view? Do you agree with Grant? If you prefer the COSO ERM Framework to the ISO standard, I would love to hear why. Personally, I like some of the COSO materials (especially the discussion of embedding risk management throughout the organization), but find the cube less than useful and the ISO presentation easier to use.