Are Organisations Getting their Pen Test BADLY Wrong?!

The whole point of penetration testing (pen testing) is to identify how vulnerable an organisation’s technology infrastructure is to attack. I suspect, however, anecdotally, that a lot of organisations have lost sight of this goal and are now opting for a; vendor-led, paint-by-numbers pen test consisting of maybe one of two high-profile Internet-facing applications. I’m not saying this is a systemic issue but from my observation over the last few years, it seems this approach to pen testing is pretty widespread and I think we desperately need a return to pen testing’s ethical hacking roots…

In this article, I want to highlight where organisations may be going wrong and how organisations could (should) make their Pen Testers work a little harder for their money!

Anatomy of a Hack

Whilst the complexity of a hack can vary, there are some basic stages to a Hack. For the purposes of simplification, a hack is “gaining unauthorised access to someone else’s technology and then carrying out unauthorised activities.” I’m sure there are better definitions but I’m going with this one for now! The basic stages of a hack are thus:

· Reconnaissance – finding potential targets (including humans!)

· Scanning – identifying vulnerabilities which could be exploited

· Gaining Access – using exploits (both human and technological) to gain unauthorised access

· Exploitation – using access to carry out unauthorised activities (e.g. stealing data)

· Maintaining Access – ensuring access can be attained at a future date for further exploitation

· Covering Tracks – removing all signs of your unauthorised activities

Anatomy of a Typical Pen Test

Now, I’m caveating this section in that I am not saying everyone does this, but I do see this occurring in a significant amount of pen tests.

· Organisations identify the requirement to do a pen test – or are required by a third party

· Cheapest bidder selected, often with little-to-no due-diligence

· Organisation provides full details (IPs, Credentials, etc) of a specific system – the ‘Scope’

· Vendor carries out non-destructive pen test (highly unlikely exploits are actually used)

· A report is produced (that often looks very must like a canned Nessus report)

· All Medium risks (7.0) and below are ignored

· Some (often the more difficult to remediate) High Risks are re-assessed to Medium/Low

· One or two low hanging fruit items remain for remediation

· Re-testing occurs at the anniversary of the pen test

· Repeat the above steps approximately once a year.

OK, so what’s the problem with the above? Well, quite a lot! Here are six of the biggest challenges see in the above approach to pen testing…

Challenge 1 – Who is replicating the Hacker’s reconnaissance?

A hacker conducts reconnaissance because they don’t know what they don’t know. The hacker looks for any way in. Hackers are interested in the company’s most visible, high-profile applications and of course, corporate websites provide lots of useful information. Hackers are also (and often more) interested in that little Internet facing server exposed on ports 21 and 3306 (I’ll let people look those up but seriously!) or the vendor who remotely manages the coffee machine by accessing the internet through the corporate network. The trouble is the typical pen test doesn’t contain a reconnaissance phase, Pen Testers (often lazily) rely on the commissioning organisation to provide them with a target list.

The question thus arises: how does the commissioning organisation know what they don’t know? If the commissioning organisation is getting a pen testing company in, it is highly unlikely they have the organic skills to carry out effective reconnaissance. If the pen test vendor only checks the targets provided, how do these exposed assets ever get detected? The short answer is they don’t – until either something goes wrong internally, or the hacker finds them first.

So, what can an organisation do? Firstly, you can commission a pen testing company to do reconnaissance on your behalf. It may be worth identifying a pen testing company who does this already. There are a lot of low-budget, paint-by-numbers Pen Testers who may not be geared up for this service as it takes more effort. Find someone who has been doing this for a while. Don’t neglect your people too. Nothing is more visceral for senior management when you show them how personally exposed they are to becoming socially engineered!

Challenge 2 – Have you got a good Pen Test Vendor?

It can be hard for a company to select a pen testing vendor as pen testing can often be perceived by organisations as a dark art. This often leads to a choice based predominantly on cost, with vendors pretty much telling the commissioning organisation what they are going to do and the vendor pretty much agreeing. Where additional challenges arise is when a commissioning organisation wants to get a ‘clean bill of health’ to show to their customers and so are often not particularly looking for a robust pen test.

If you’re in the first group, how do you ensure you commission a good Pen Tester? If you’re relying on a third party’s pen test reports, how do you know the test has been sufficiently robust to provide the right level of assurance before you trust this third party with your valuable systems and data?

Let’s tackle what makes a good Pen Tester vendor first. There is no particular law or regulation that prevents anyone from setting up a pen testing company so it’s important to make sure they have been accredited by a reputable body. A good Pen Tester will be accredited to a recognised standard (e.g. In the UK: CHECK, CREST, CBEST for Financial Services). Check the accreditation hasn’t expired too as some will make the effort to gain accreditation but then let it lapse. The next thing you need to check is the people doing the pen testing and their credentials. It’s worth noting a company can be accredited but that doesn’t mean the person you get is competent or qualified. Always insist the Pen Testers testing your environment are qualified and ensure that forms part of your contract with the pen test company. A good pen test company will be more than comfortable providing you with the provenance to support their (and their employees) accreditations – they have spent money and hard graft to get them after all!

The same applies when relying on the results of a third party’s pen test. Insist [contractually] that the pen test is carried out by an accredited pen testing vendor using qualified Pen Testers. Be cautious about vendors who say things like ‘our Pen Testers are self-taught and that makes them better than people who have just taken a test’. It may be true that their self-taught Pen Testers are better than people who have ‘just’ passed an exam but no matter how good they are; they still need to prove that by passing an appropriate exam too. A lot of certifications also require the holder to maintain their knowledge and also sign up to a code of conduct too. No-one who is solely self-taught will have similar requirements to maintain their knowledge or behave in a certain way when testing your systems, nor will you as an organisation have much recourse should they leave their current company and then hack into yours!

Challenge 3 – Is the Scope well defined and appropriate?

Scoping is probably where pen testing goes from conceptually flawed to actually flawed. As mentioned in the reconnaissance section, organisations typically don’t know what they don’t know. In the scoping section this lack of knowledge gets crystallised – essentially the vulnerabilities you didn’t know you have will now not be identified, checked and, worse, will continue to be exposed to attack. That’s the first issue.

Ok, let’s put what we don’t know aside and talk about scoping the test for the assets we do think need testing. ‘Just test the Internet-facing systems’ is the most common. Apparently, hackers only attack a network from the Internet and have limited imagination to attempt to gain access any other way…err no! Organisations must stop thinking of their infrastructure as a perimeter which needs to be defended and take a more risk-based approach. All parts of your infrastructure should be tested but it is how often that should be risk-based. Email is probably the most common way a hacker can gain access to your internal network – how often is email even included in your scope? VPN is another common means of gaining access. Don’t forget the human too, dropping off a malware-infected USB stick with ‘Bitcoin Wallet – Value: 0.456bitcoins’ is almost certain to get plugged into a laptop by some curious person!

Next, think about the types of test and what you’re trying to achieve. Unauthenticated tests have little value these days as all you are likely to be testing is the security of the login mechanism. Authenticated tests allow testing of the internal workings of the application and authenticated admin tests will give an idea of just why you should never expose admin portals to the Internet or, if you must, why you definitely need multifactor authentication (MFA) for your admins.

Finally, think about the newest addition to the OWASP Top 10 for 2017 – logging and monitoring. As an organisation processing personal data, you need to be able to detect a breach as soon as possible so that the impact can be mitigated. As your Pen Testers are testing, your IT and InfoSec teams should be detecting! If your Pen Testers are testing and you can find no evidence of these clear, time-bound, attempts to gain access to your network, then you really have a problem. It is highly likely the Pen Testers are going to ‘noisily’ test. That is to say, the Pen Testers aren’t going to be particularly trying to hide their activity. If you can’t detect a noisy attack, how are you going to detect a hacker doing her best to remain undetected?

So, what should a good scope look like? I’m not going to document a complete scope, but the following components are must-haves.

· Reconnaissance – get the Pen Testers to first go on a discovery mission to find and provide you with a list of potential targets they would want to try and exploit. This is a great pilot exercise prior to engaging a pen testing company to conduct a pen test – some newer vendors may even do this exercise as a loss leader to show you what they can do.

· Risk-based target set – work with the Pen Testers to delve through the reconnaissance material and prioritise which systems are the most inherently vulnerable. Ideally, all systems should be tested eventually but some should be tested more often and more rigorously than others. Don’t neglect internal systems.

· Authenticated testing – there really isn’t much point doing unauthenticated testing. User credentials are constantly compromised because, well, humans are fallible. Work on the principle that at least part of your testing takes compromised credentials into account and, if you have third-party connections, consider the impact if their systems are compromised in order to get to your organisation.

· Test Production Systems in Production environments – If you aren’t comfortable testing your Production systems that alone should ring alarm bells. If your third-party service provider won’t let the production environment be tested, again major alarm bells. Pen testing is not about providing assurance for a specially locked-down test rig that never changes (save for updating patches) – It’s about testing the live environment – the one that will be subject to change and ultimately the environment the hacker is going to hack! If you’re worried about downtime – plan accordingly. Just remember, downtime from a controlled pen test is always going to be better than downtime from a live hack!

· Detection – whilst resources are deployed to attack the organisation (often referred to as the ‘Red’ Team or ‘Red Teaming’), the internal information security resources, including Managed Security Service Providers (often referred to as the ‘Blue’ Team or ‘Blue Teaming’) should be deployed at the same time to detect the Pen Testers activity. For those at the lower end of the maturity curve, it’s probably a good idea to get the Pen Testers to tell you what and when they will be testing but as the detection capability matures the two teams should begin to work blind of each other. When your internal team increases in skill level, consider moving from passive detection to active response (i.e. can your internal team kick the Pen Testers off the network without compromising normal system availability). At the end of the test, the pen testers should provide contemporaneous notes of what they did (with time-stamped logs if possible) so gaps in detection capability can be identified and remediated.

Now there are other things you could include in the scope depending on your risk appetite, such as confirming identified vulnerabilities using benign exploits or live social engineering on your employees. These additional items should only be considered once you have got the core scope well locked down.

Challenge 4 – Does Reporting & Remediation Advice have context?

This is where a Pen Tester, working collaboratively with the organisation, can really add value – but rarely does this collaboration happen. Pen Testers typically write reports independently of the organisation, providing base CVSS scores straight out of the box (read Nessus). Anecdotally, I have not seen to this day a Pen Tester who has adjusted these base scores to take into account organisational context (e.g. factoring in the CVSS Temporal and Environmental scores). This means most risk ratings attached to the base scores are already inaccurate and potentially misleading.

Contextual reporting is critical to ensuring the right risks are remediated. Take this example of a vulnerability (CVE-2018-1000100) with a CVSSv2 base score of 6.8. It is common for organisations to take a risk-based approach, whereby anything with a Common Vulnerability Scoring System (CVSS) score of say 7.0 or below is risk accepted (and is thus unlikely to be remediated). Anything with a score of 8.0 or above would be graded as High Risk and anything 9.0 or above would be graded as Critical – it is these vulnerabilities which are then tracked and, where possible, remediated. The flaw in this approach is that this doesn’t take into account any organisational context. When you add the Temporal and Environment components specific to an organisation, this 6.8 could easily become 8.7 and a risk that would have been accepted under the base score alone, now becomes a viable remediation candidate.

Here is a link to how the above was scored: https://nvd.nist.gov/vuln-metrics/cvss/v2-calculator?vector=(AV:N/AC:M/Au:N/C:P/I:P/A:P/E:F/RL:W/RC:C/CDP:MH/TD:H/CR:M/IR:M/AR:H)

In contrast, the following vulnerability (CVE-2018-1000118) has a CVSS base score of 9.3 – this would likely have everyone’s attention if it were identified during a pen test! Now apply Temporal and Environment scores and this vulnerability could now be scored as low as 1.4! This means your organisation could be remediating vulnerabilities unnecessarily at the expense of other important work.

Here is a link to how the above was scored: https://nvd.nist.gov/vuln-metrics/cvss/v2-calculator?vector=(AV:N/AC:M/Au:N/C:C/I:C/A:C/E:U/RL:OF/RC:UC/CDP:L/TD:L/CR:L/IR:L/AR:L)

Now, these are extreme examples but they serve to show just how easily context can significantly change the risk scoring. So, what should reporting look like? Basically, the report should contain both the inherent ‘base’ score and, the residual ‘Temporal & Environmental’ score so that CISOs and IT Leaders can make informed decisions about how the vulnerability remediation will be managed. The most important factor in this reporting is to use a recognised calculator (such as the one at the links above) to perform the calculations. Using the ‘finger-in-the-air’ approach is clearly to be avoided as it will lead to questions and challenge which most likely can’t be defended! There are lots of other things that should be in the report too (executive summaries, remediation advice etc.) but if the decision support information is not good enough, the rest of the information is going to be of limited value.

Challenge 5 – Are you remediating and retesting the right things?

Ok, so we know what our vulnerabilities are and, we have gone through a scoring mechanism to determine the correct vulnerabilities requiring remediation. Again, this is where Pen Testers could add further value – but they don’t! What tends to happen is the recommendation in the pen testers report comes straight out of the box (again read Nessus). This is, of course, one approach that would likely reduce, or remove, the vulnerability – but that is not what the organisation commissioning the pen test needs.

What organisations need are options based on their specific organisational context. Rather than telling an organisation this is what Nessus (which I think is quite a good tool by the way) thinks should be done to remediate the issue, Pen Testers could add value by better advising the organisation based on what they have learned during the testing e.g. as your architecture is currently set up in Y configuration, if you make 2 architectural changes, the vulnerability would be completely architected out. To give a more visceral example, if for some crazy reason, you have an architecture which allows IT Admins to connect directly to a Bastion Host from the Internet, and that host’s applications contain a number of exploitable DDoS vulnerabilities; instead of remediating all the different software vulnerabilities, the potentially more sensible (and better in the long term) approach could be to simply stop exposing the Bastion Host directly to the Internet and instead forcing IT Admins to first connect to a VPN and then connect to the Bastion Host from the Internal network. Nessus, or any other automated tool, is never going to be able to make these contextual suggestions – but a Pen Tester who has taken the time to understand the client’s infrastructure could!

Which, finally, leads to retesting. Make sure the pen test includes a retest within a reasonable period (i.e. no later than 3-4 months after the original test). The retest can be streamlined to cover only the vulnerabilities that your organisation has agreed to remediate but, should also include some wider regression testing to confirm any remediation efforts haven’t inadvertently exposed the organisation to additional (perhaps worse) vulnerabilities.

Challenge 6 – Are you being transparent?

A pen test is an assurance activity and so the results of testing need to be made available to those seeking assurance – in a sanitised form. Sanitisation should strip out certain information, that if it was exposed to the wrong people could cause harm – but the sanitised report must still be useful to those seeking assurance. So what does transparent look like?

Details of the testing organisation and the scope of the testing should always be made available to those seeking assurance because this shows what has been tested, and how rigorously. Results should also be made available to the extent that Base scores and Temporal/Environmental scores are included but with limited information on the specifics of any particular vulnerability – remember a hacker is going to do their own reconnaissance whatever is included on a pen test report but we don’t need to make it easy for them. Information on what is going to be accepted and what is to be remediated should also be made available to those seeking assurance, including confirmation of remediation via retesting. This is so those seeking assurance can make an informed decision about the risks to which they would be exposed. If you are relying on a third party’s pen test report as assurance of their security and any of the above information is not available – or worse they won’t provide it – then you must insist on the information and escalate if necessary. Remember if you are using the third party provider to support the processing of personal data, then you are legally obligated to ensure they maintain a secure processing environment. Additionally, the third party is legally obligated to provide sufficient guarantees their environment is secure – so there really isn’t an excuse not to be transparent.

Summary

Organisations would benefit a great deal from reviewing their pen testing approach and beginning the process of getting a better level of service from their pen testing vendor. (Re)Introducing a reconnaissance phase, improving the scope of testing, improving the quality of reporting through the inclusion of specific organisational context, better remediation and retesting and, finally, demonstrating better transparency to those seeking assurance will all add real value that sadly has been eroded over the years as pen testing has deviated, considerably, from its ethical hacking roots. Let’s get things back on track!

About the Author:

Stephen Massey is the Managing Director of Fox Red Risk, a boutique Cyber Security and Data Protection consultancy, which in addition to offering GDPR advisory services, provides vCISO and Data Protection as a Service. Stephen has also authored the popular book “The Ultimate GDPR Practitioner Guide” which is available on Amazon in both 8×10 paperback and Kindle eBook

About Fox Red Risk:

Fox Red Risk offer a vCISO service which, amongst other things, helps Client Organisations with commissioning penetration testing. Fox Red Risk does not conduct pen testing and does not currently partner with any penetration testing companies. Fox Red Risk will never recommend a penetration company unless it fits our clients’ needs – and never as a result of receiving financial inducement.

ciso information security penetration testing Pentest risk security security as a service vciso virtual ciso vulnerability scanning