Poorly defined security KRI or Key Risk Indicators can give your senior management team a false sense of security but is a fear of presenting a potentially negative picture akin to the Hans Christian Anderson tale, ‘The Emporer’s New Clothes’? In this fairytale two tailors promise their emperor a new suit and tell him that those who are unfit for their positions, stupid, or incompetent will not be able to see the fabric of the suit. In actual fact, no suit is made at all, everyone simply believes the clothes are invisible to them. When the emperor shows off his new “suit”, no one dares to say that they do not see the suit for fear that they will be seen as stupid. The story comes to a head when a child shouts out:
“But he isn’t wearing anything at all!”
With Security KRI information security professionals can often find ourselves in a similar situation where we create KRI in order to show how well we are doing, rather than giving the senior management team information that they can use to inform their decisions. If your security KRI do the following then you’re potentially another tailor providing your management team a suit of nothing…
Security KRI are Static
There are hundreds of security KRI that can be collected in relation to information security. Look at what could be reported from Firewalls and Microsoft system events. From Learning & Management Systems (LMS) and BCM Tools. From IAM systems and Office 365. Because there are so many different types of metrics that can be collected, there is a tendency to pick a set of easy to collect KRI and stick with them.
“Once metrics are green in a particular area, look to reduce the frequency of collection and identify new areas where metrics could be useful.”
What isn’t taken into consideration with such an approach is; by collecting a static set of security KRI there is the potential to become blinkered into a narrow focus where the goal is to keep these security KRI green. For example, all the while an information security team is basking in the glory that annual awareness training completion rates are at 99.2%, a piece of malware is slowly egressing data over TOR because no-one is reporting a security KRI related to network traffic.
Security KRI are self-reporting
I see this a lot, metrics are presented to the senior management team with an expectation that the members of the management team will simply understand what they need to do because the metric is a certain colour. Quantitative metrics are often presented with little-to-no qualitative analysis and often with no call-to-action. What exacerbates this situation is that the senior management never ratified the security KRI specification – because the specification was never actually documented. The security KRI were just plucked from an online source and whacked on PowerPoint template without any formal integration or alignment with the organisation’s risk management framework.
The Security KRI specification contains, amongst other things, the actions the Senior Management Team will perform based on the current status of a particular Security KRI. For example, if a Data Processor is 30 days overdue for supplying information relating to their obligations under Article 28.3(h) of GDPR then a recommended course of action should be documented in the KRI specification e.g. Consider giving notice to terminate a contract – in an extreme case of course!
Everything is Amber…
Red is bad! Therefore we don’t ever want to tell the Board that something is Red. When I say red I am referring to the common practice of defining security KRI in terms of Red, Amber, Green – the RAG status – The Traffic Light system. The general idea is that Green means everything is OK, Amber means the underlying situation needs monitoring and Red means immediate action is required. Whatever system is applied, there are a lot of organisations who artificially elongate the amber (other colour schemes are available) section of their traffic lights to hide inaction. These constantly amber KRI come about because they were poorly defined in the first place hiding a point where action should have already been taken. What do I mean? Say you’re monitoring a percentage. Green will typically be a pretty realistic 90%+ but Amber will be 10% to 90% and Red less than 10%. The Amber section doesn’t reflect monitoring and also covers when a risk has already exceeded management approved risk appetite. When I’ve asked infosec practitioners why they do this I’m told, ‘off-the-record’, that: “The CIO has told us we’re not allowed to present Red to the Board”. When I ask why I am usually given the same answer: “Well, it looks bad on the CIO“. Not a good situation to be in.
So how can KRI be defined to avoid this type of situation? Let’s look at an example of a good security KRI which has been poorly implemented:
“Number of Servers operating on Unsupported Operating Systems (OS).”
In the bad example, Green is set at an arbitrary percentage of 10% or fewer servers on an unsupported OS, Amber is set at an amount of 10% to 90% and Red is set at 90+. That means an Amber rating could leave an organisation with 90% of servers operating on unsupported operating systems before any action is taken. For those not familiar with this particular area of security ideally, none of your servers should be on unsupported OS!
A better way would be to define the security KRI so that it reflects risk appetite and the principles of security KRI design. The Amber section can be as long as you want as long as it reflects a monitoring stage. Let’s look at the same example but using a more appropriate KRI. Let’s say we’re defining security metrics for a hospital and there are 2 x 2003 boxes somewhere embedded into two expensive-to-upgrade MRI machines. These two servers form part of a total of 100 servers. These 2 machines have been deemed an acceptable risk due to other compensating controls being put in place. Green is, therefore, set at 98%+ of servers in normal vendor support. Amber should trigger when an agreed percentage of those 98 servers falls out of normal vendor support and into extended support. This would then be monitored as extended support dates can and do change. Red should trigger when any of those servers are within 60 days of the date when extended support will end because a red security KRI should reflect the period of time where the management team now need to take action.
Security KRI – Everything is Red!
This situation often arises because the CISO wants to present a negative picture, commonly at the beginning of their tenure, that everything is really bad and they are going to be the one to fix everything. In this situation, the security KRI is used, cynically, as a means to secure additional funding or, increase the priority of security projects deemed necessary by the CISO. Personally, in the organisations Fox Red Risk work with, the situation is never completed dire. Even in the worst-performing organisations, there are still positive things happening. Security KRI thresholds must be set to ensure the organisation remains within its risk appetite and its legal/regulatory boundaries. Care must be taken that security KRI are not turned into a toxic political tool.
As hinted at earlier in the article, there is a widespread misunderstanding that Red in terms of a KRI has a very different meaning than Red in terms of Key Performance Indicators (KPI). A Red KRI is not supposed to mean something is bad or has been missed, rather that:
“this is the last chance to take proactive action before KPIs are missed.”
It means, just as we do at traffic lights, senior management must stop, understand there is danger ahead if the organisation continues down a particular path, and then take appropriate steps to get going again. Those that deal in reporting security KRI (or any other type of Key Risk Indicator) need to do a better job at explaining this to their management teams!
Everything is Green!
Presenting security KRI to the Senior Management Team showing everything as green is equally as unhelpful. Green KRI should only be presented to the Senior Management Team when the KRI is turning from Amber to Green to alert the Senior Management that a situation has now improved to acceptable levels. Sure, they should be reported for a short period of time but after that, but there is no point in reporting ad infinitum because what action are you asking the Senior Management Team to take? None. Does that mean you should stop collecting the data? Of course not! The data should still be collected and analysed at the operational level within the information security team but there is no point wasting senior management’s time when they have other things to worry about!
Review your security KRI
Security KRIs are a vital tool. What isn’t monitored isn’t managed but monitoring the wrong things and, reporting this information in the wrong way can lead to your senior management team (and the wider organisation) from becoming unnecessarily exposed. They may not be walking down the street naked having rotten vegetables thrown at them (think Cersei’s Walk of Atonement in Game of Thrones) but they may have to one day explain a data breach on the national news or a Treasury Select Committee.
If you are unsure whether you’re collecting the right security KRIs or reporting them in a manner which keeps your organisation secure then get in touch. Fox Red Risk can help.
Contact Us
If you would like to have a conversation to discuss your needs, fill out the form below and we can arrange a time to call you over Teams, Zoom, or Google Hangouts.
About Fox Red Risk
Fox Red Risk is a boutique data protection and cybersecurity consultancy and Managed Security Service Provider which, amongst other things, helps client organisations with implementing control frameworks for resilience, data protection and information security risk management. Call us on 020 8242 6047 or contact us via the website to discuss your needs.
22301 22301:2019 article 28 awareness bcms BIA business continuity calculating risk change management ciso controller cybersecurity data breach data privacy Data Protection data protection by design data protection officer data protection service Data Subject Access Request DPO DSAR GDPR incident management information security leadership monitoring operational resilience Outsourced DPO Privacy processor risk risk appetite risk management ROI security security as a service small business strategic strategy Subject Access Request tools training transparency vciso virtual ciso
