The penetration testing lifecycle: What you need to know

Understanding the penetration testing lifecycle 

If you were to search for the phrase “penetration testing life cycle” you’d be treated to about eleven million results. Having read a few, it is clear that many are remiss of the how and, more importantly, the why behind the stages of the pen test lifecycle.  

If you’re reading these articles because you’re making a business case for penetration testing, then the how and the why is crucial to understand. Likewise, you might have found yourself in the position where you’re already on the hunt for a penetration testing partner and you’re trying to arm yourself with as much information as possible.  

Or maybe you’re considering becoming a pen tester yourself. Either way, let me add a little more colour to the subject. I’m a staunch believer in balancing out the information asymmetry between client and supplier because we all end up better off when we’re a little more informed. 

Now, my goal isn’t to teach you to perform a penetration test for yourself.  

You can’t get that kind of knowledge from a single article, and that would be like trying to teach someone to be a GP in a day. Rather, I’m going to give you the insight and explanation that will mean that when your pen tester says something mildly indecipherable like “I found a few potentially interesting things on whilst I was doing some OSINT on Shodan that I’m keen to take a look at later on” – you’ll know that they’re talking about the reconnaissance stage of the pen test and that everything they’ve found is publicly available.  

Let’s take a look at these pen test stages and understand what’s important, from a pen tester’s perspective. Before we get into the meat and bones of the article, I’d be remiss if I didn’t spend a little bit of time making sure that we’re working on the same definitions for the rest of the article and giving a quick recap of what pen testing is.  

If you’re already clued up on the definition of pen testing and understand that it has nothing to do with biros, then you can safely skip the next section, the penetration testing lifecycle stages.

What is penetration testing? 

Imagine that you’re a plumber and you’ve just bought yourself a brand-new van. However, you’ve heard from your fellow plumber peers that there’s a fair amount of crime against plumbers’ vans. People are breaking into vans and stealing thousands of pounds worth of tools and equipment. 

You get some recommendations about how you can make your van a much safer, much harder target for thieves, and you sensibly invest in some heavy-duty upgrades: deadbolts on all the doors, GPS tracker, aftermarket immobiliser and the ubiquitous “no tools are left in this van overnight” sticker.  

You also learn about other ways to reduce the chances of criminals finding your van in the first place. Now, despite being vigilant, you’re still wondering how all these security features you’ve implemented would hold up against a targeted attack.  

You talk to a security systems friend of yours who suggests that she spend a few hours giving your four-wheeled safe room a once-over. She knows the tactics, techniques, and procedures that criminals use, and she’ll try a whole host of these to try and bypass your van’s security, stopping short of bending doors, cutting wires, and breaking windows.  

She’ll even write you a list of things that she finds and how important they are to fix, so you can easily prioritise what you need to fix, change, or update. And, once you’re done, she’ll re-test your efforts to see if they’ve worked. This is a great idea, and after a lengthy negotiation, a payment of beer is agreed upon. 

What you’ve just commissioned is a penetration test or pen test. It’s a security test in which a security consultant will assess the strength of your security by attacking it in the same way that a real criminal would.  

In cybersecurity this means finding and exploiting vulnerabilities in computer systems, be it networks, websites, mobile applications, APIs, or any other computer system you own or use.  

It’s one of the most direct and popular ways to assess the real strength of your security systems and, among the many benefits, it allows you to get fast, detailed feedback about any vulnerabilities you might have. It also shows you how individual vulnerabilities might be chained together to create larger, more potent ones.  

Penetration tests are a form of ‘ethical hacking’ which sounds much cooler than it is if you’ve only seen Hollywood depictions of ‘hacking.’  

We seldom get to evade the police by running down the fire escape stairs and jumping into conveniently placed bags of rubbish. But we do sometimes wear hoodies and yes, you can sometimes boost your Wi-Fi reception range using a Pringles can.  

Either way, take it from me, it’s a lot of looking at black screens with white text waiting for your script to finish running before erroring out. Ethical hackers are a curious bunch, often obsessively so. And we love nothing more than opening things up, finding out how they work and figuring out what else you can do with the thing… 

Pen testing wrapped up, let’s move swiftly on to the various stages of a pen test. 

The penetration testing lifecycle stages

Depending on who you ask, you’ll be told that there are between five and seven stages of the pen testing life cycle. The difference between them really comes down to preference, but any number of stages will cover the same things.  

I’m a fan of the seven-stage approach, so that’s what we’re going to do here. If this isn’t the first blog you’ve read that explains pen testing, or if you feel like you’ve got a good grounding of what the stages are broadly speaking, then feel free to skip ahead. 

Nonetheless, the between-five-and-seven stages of a pen test lifecycle are: 

1. Pre-engagement and scoping 
2. Reconnaissance 
3. Scanning and enumeration 
4. Exploitation and maintaining access 
5. Post exploitation 
6. Reporting 
7. Remediation and re-testing 

Or, for the sake of brevity, we can also call these stages of a pen test: 

1. Plotting 
2. Snooping 
3. Poking 
4. Hacking 
5. Cleaning up 
6. Bragging 
7. Fixing 

For the rest of the article, we can consider these terms to be interchangeable.  

Stage 1: Planning the engagement 

When can we get started? you might ask, only to feel that slow sense of deflation when the response is “We just need a little more information from you….” I know, everyone wants to skip the planning and get right on into the action and start hacking and finding vulnerabilities. 

But skipping over this phase of a pen testing engagement is quite literally setting up to fail.  

Far from being unavoidable, getting the pre-engagement part of the test right can be the difference between a successful test and an unsuccessful one.  

When you buy a pen test, you’re buying a series of activities and expertise to be applied to specific systems within a specific period. They’re time-bound exercises and they need to be testing a system outside of the agreed time tends to be looked at dimly by the police. Something to do with the Computer Misuse Act, which was another thing we learned about on day one in hacker school.  

In any case, it’s vitally important that we get all the key information about the test upfront so that you get your penetration test scoping correct and in turn, ensure you’re getting the most from your pen testing budget.  

This might mean various levels of effort on the client side depending on what is being tested. I test web applications and carry out API tests more than I test internal infrastructure, so a lot of my tests require the client to create development or ‘test only’ versions of the application.  

This will also include things like setting up users (in the case of authenticated tests) and API keys where necessary.  

Infrastructure

In terms of infrastructure, this will mean creating low-privileged user accounts as well as delegating whatever access might be needed. And by this, I mean, how am I going to access your network? Do I need VPN access? RDP? Or you can deploy one of our virtual machines inside your network? 

I can’t labour the internal infrastructure testing points enough. There are so many moving parts on a modern network that can be easily overlooked if the time isn’t spent upfront making sure that all bases are covered.  

Subnets

Take network subnets, for example. There is a whole body of attacks against networks that involve relaying traffic flowing between different computers on the network. When we do this, we’re listening in to parts of the encrypted, digital conversations that the devices are having with each other.  

We’re being sneaky and sometimes getting involved in those conversations, masquerading as harmless, innocent devices that just want to be involved in the network shenanigans. Mostly we’re just trying to steal credentials though so we can pretend to be you or your network printer or whoever else is accessing network resources at the time.  

But one thing that will put paid to all of this is if we’re not directly connected to a network that has devices talking to each other. There are a lot of reasons why you might not want or be able to achieve this type of setup, among other constraints, but knowing something like this upfront will help in the planning phase.  

If I’m told before the engagement that I’m not going to be on the same subnet as the rest of the users, then I’ll know that I might need to spend more time trying to find ways around it or focus on other types of vulnerability exploitation.  

As a pen tester, I’m not always involved in this part of the engagement, but if I am, then I’ll have a little more time to start thinking about the test and plotting the ultimate demise and complete control of your computer system as it falls to my near godlike computer hacking abilities. Well, that’s what I tell myself I’ll do anyway. 

Stage 2: Having a good ‘ole snoop around 

This stage in the pen test lifecycle is often overlooked in a lot of tests. There are a couple of reasons for it, the least of which is that we’re not always clear on what we define as ‘reconnaissance.’  

If you look at a five-stage model of pen testing, then recon will include the ‘active’ and ‘passive’ phases. An analogy for this is thinking of your IT systems as a neighbourhood–even a small town or hamlet. Passive recon would be driving down the road and having a look at the houses. Or even better, doing a virtual drive-by on Google Street View.  

Passive snooping is anything that doesn’t interact with any devices, while active snooping is akin to going up to doors, looking through windows and trying back gates to see if they are locked.  

Personally, I think that as soon as you are sending traffic down the wire and interacting with a device, you move from the recon stage into the scanning stage. In my mind, if anything I do has a chance of ending up in a server log then that is active. 

That said, I understand why people might want to put active snooping into the recon phase. I’m no lawyer but I feel like people don’t get put in prison for seeing if someone’s front door is locked. There’s the whole intent side of things, but overall, you might just be being a good friendly neighbour and checking that everyone’s security is up to par. I digress…  

Stage two is passive reconnaissance. This is the most fun on web and API tests because the gods saw fit to bless us with search engines like Google and oh boy is it a treasure trove.  

In the recon phase of a test, we’re trying to build a picture of the computer system we’re trying to exploit. This means gathering information. The thing about information is that humans create and organise information and we often show more of it than we intend to.  

We put things in places of convenience and then forget that they were ever there. This is true enough in the real world where it’s easy to shout to your neighbour across the street “Hey Alice, your back gate’s open” or “Bob, you left your keys in the front door.” But in a digital world, it’s much harder to see the systems that you’ve left unlocked and vulnerable. 

Are you an oversharer? Information disclosure vulnerabilities 

There is a whole class of vulnerabilities called ‘information disclosure’ vulnerabilities: these happen when computer systems tell the user more information than they’re meant to. This covers a broad set of things, from the slightly more technical “you left your cloud storage container set to world readable” to the simply perplexing “I found a list of usernames and passwords in a folder called ‘secret’ on your website.”  

That last example isn’t a horror story that systems administrators tell their children at bedtime, I’ve seen something alarmingly similar happen on a test. I, myself, have found internal documents that have been uploaded to a website so that they could be more easily shared with employees.  

Innocent at first, but the problem with putting anything on the web is that Google is exceptionally good at two things: one, finding things; two, telling you where it found the thing.

Nonetheless, in the second stage of a pen test, we’ll be having a very good online search of your organisation, looking for all the PDFs that have been uploaded to your website, what internet devices are associated with your external IP addresses, any text files that have been indexed and even, if needed, we can pull personnel lists from places such as LinkedIn. 

The fun here is endless, and whilst we may not use everything that we find online, we might just find some piece of information that allows us to guess a username, a password, find a hidden admin page, database server or something else that you wouldn’t want people snooping around. 

Stage 3: Poking stuff and listing things 

This stage in the pen test lifecycle is where we start to pop up in access logs because stage three is part of a pen test when we begin the process of asking your computer systems questions and recording the answers. During this phase, we’re breaking out our scanning tools and actively asking computer systems what have you got out there? 

This phase is most exciting when you’re on an internal network because you already know an awful lot about what you expect to find on a network, so there’s a little rush of excitement when you do find it.  

You know that at the very least, there are going to be workstations, routers, and a domain controller or two. It gets fun when you query the DNS server and find out that there are routes to other networks that are connected to yours, and those networks have interesting-sounding devices on them like BACKUP01 or FINANCE.  

But we’re not just looking out for what stuff calls itself when we’re sifting through a network: the tools that we use are good at collecting information about what software the machine is running.  

Finding information like operating systems is a crucial part of the scanning and enumeration process because the mere existence of certain operating systems or other pieces of software is a vulnerability in and of itself.  

For example, if you’re running Windows XP–which support ended in 2014–this is a huge vulnerability to a network, especially if that machine has any kind of privileges in the domain.  

Unsupported and vulnerable software is always seen as a vulnerability, and we’ll always classify it as ‘critical’ in our reports. 

There’s so much information that you can generate during this phase. There are also a lot of ways that this part of the process can send you on a merry chase. Remember, we’re asking these computers questions and cataloguing their answers to get more information and decide on the next questions that we ask.  

But what if you find yourself talking to a web server that simply says ‘yes’ to every question you ask? This is a common approach to some types of infrastructure, as it slows down an attacker’s scanning and enumeration process.  

Web servers can be configured to respond to invalid or malicious requests with an ‘OK’ response and helpfully serve you up the website’s home page. I’ve seen this on many occasions whilst running an automated scan and trying to discover any hidden directories, only to have my tool excitedly tell me that it found every vulnerability it knows about. 

Likewise, network firewalls can be configured to say that every one of the 65,536 ports on a given IP is open and ready for business.  

This whole phase is about identifying vulnerabilities and coordinating them into potential chains of exploits.  

There’s a lot of intuition and experience that kicks in here. We’ll be looking for things that are often only slightly different from the standard. Sometimes it’s something as small as a naming convention being different. It might seem like nothing, but if we’re looking at four variations of the same web app and one of them calls their login page “login-admin” rather than “admin-login” then that tells me something; it means that whatever the rules and procedures are for the other three apps is likely to be different. 

I was helping a colleague on a web app test who was wondering why he’d found a configuration file that had nothing to do with the application he was testing. I reasoned that the application had been migrated at some point, but they forgot to do a proper cleanup, so he should look for more of those files as they could have potentially sensitive information in them. He did, there was, but unfortunately ,the things he found were out of date and couldn’t be used.  

An attacker wouldn’t necessarily care about that, and they’d try those details in a bunch of other places or use it as some kind of leverage. Better not to give them the opportunity in the first place! 

Stage 4: Let’s hack! 

Finally, we’re ready to hack I hear you say, elated at the prospect of our many hours of careful preparation and years of practice breaking like waves against your multi-layered, sophisticated, defence-in-depth security posture that you’ve honed over countless deployments.  

Or perhaps you’re in a different situation, and you’re building the business case for greater organisational investment in cybersecurity, and secretly hoping that a pen test finds some serious vulnerabilities, giving you the license to print off the report, roll it up and use it as a blunt tool to remind the board about the importance of upgrading and maintaining your IT.  

Either way, this stage in the pen test lifecycle is all about action: we’re actively exploiting the vulnerabilities that we’ve identified in the previous phase(s) and, if we’re lucky, we’ll be able to chain together a few vulnerabilities to give us even more access and even more control over systems that you don’t want people having control over. 

The specifics of this part of the test vary wildly depending on what’s being tested because vulnerabilities are always specific to the target platform. There’s no point trying to run a Windows-based vulnerability against a Linux machine. Likewise, PHP-based exploits aren’t going to work against a JavaScript application.  

But it’s outside the purview of this article to go into excessive detail–as much as I’d love to–I’ll run through a few things that we’ll look for on a prominent level. And only in terms of web apps and networks too for similar reasons. 

Let’s start with some network (infrastructure) type stuff. Attackers will always go for the low-hanging fruit wherever possible, so the first ports of call are going to be the things that come back as obvious vulnerabilities. Finding things like Windows XP is effectively an open door and a typically low-effort, high-reward target to attack.  

Infrastructure assessments 

On infrastructure assessments, we’ve got the luxury that we don’t have to be too sneaky when we’re scanning a network, so we’re able to use vulnerability scanners which will deliver us a report of all the in-scope machines and any potential vulnerabilities they might have.  

This is just the first stage, an automated task that gives us a nice overview. It’s then for us to determine which sets of vulnerabilities might lead to further access. Some things that would be low-hanging fruit would be network shares that don’t require a username and password to access.  

Misconfigurations 

We’re also going to look at the configurations that are responsible for giving users permissions and access on your network. Well, configured, it becomes quite difficult to exploit. But there is a whole host of common misconfigurations that can very quickly lead to the complete takeover of your domain via privilege escalation attacks.  

These types of attacks are so commonplace that they’re categorised by number: at the time of writing there were fifteen techniques for privilege escalation in Active Directory environments, called ESC1-15. This family of vulnerabilities are well documented, including how to mitigate them, so it’s always worth having a look at your own network to see if you might have any. 

Privilege escalation attacks 

Privilege escalation attacks are also possible via overprovisioned user rights. There are a few tools that we’ll run against a network which will map out the user privileges and how they relate to each other. Sometimes we’ll find accounts that won’t have high privileges themselves but are part of groups that do have high privileges.

Think of a network printer that is set to have no privileges itself but has been accidentally added to the same group as administrators. If an attacker can gain control of the printer, or find a way to impersonate it, then they already have a significant advantage when it comes to furthering their attack. 

Web app testing 

With web apps, it’s another story altogether. Web is one of those strange areas where there’s an overlap between infrastructure and application. If you ever get a web app test and an external infrastructure test, then don’t be surprised if you see some of the same vulnerabilities on both reports. Leaving aside similar things like outdated software and vulnerable operating systems, web app tests will focus quite heavily on understanding the business logic of an application and trying to find ways to find flaws in it and circumvent it. 

Most web apps that we test are interactive, fully blown applications. I’m talking about the types of things where users log in and use the application to complete tasks in an interactive way.  

Think of a popular application like Canva, as an example. With applications like these, there are so many user inputs – and a user input isn’t a thing that you should blindly trust, because when you receive input from a user you often process it on your hardware. That means that you’re doing things in your application with things that someone else has given you.  

Why is this risky?  

Say you have a quite simple (albeit pointless) application that ask for someone’s favourite number, and you give them back that number, but you multiply it by 10. If you just blindly trust that the person has given you a number and do no further checks to verify it is a number, then at the very least your application will throw an error when someone writes their number in letters instead of digits. Threat actors will try to abuse this and find ways to get your application to do things it wasn’t designed for, like calculating incredibly large numbers so your server locks up or trying to get your server to run some of their own, malicious code. 

In application testing, we’re spending a lot of time looking for user inputs and seeing if we can get malicious things into them. Sometimes we’re able to upload scripts that will run in the users’ browser. This is a good attack vector for a threat actor because if there are weaknesses in other areas like how the application sets its cookies, then you might be able to steal a logged-in users’ current session.  

There are bonus points if that user is an administrator. We’ll also try bypassing login screens, or, if there’s an ecommerce element to it, trying to get through the checkout process without paying any money. 

Stage 5: Cleaning up 

The cleanup process does what it says on the tin. Whenever we exploit a system, we’ll often want to leave things so we can regain access to that thing without having to go through the whole attack chain again. This could be webshells if we’re able to get access to a web server, but it might also simply be the malicious files that we’ve uploaded to an application.  

In networks, it’ll involve resetting permissions if you’ve managed to gain the ability to do that. There’re also new users that you might have created if you’ve compromised the domain completely, DNS records for example. More than simply ‘leave it as you found it,’ we’re trying to make sure that if we’ve found a vulnerability, we don’t leave the door open for anyone else to come snooping around. 

Stage 6: Reporting, a.k.a bragging rights 

As much as we IT professionals like to talk shop and trade war stories, there’s precious little point in telling the client about how elite you are as a hacker if they’ve no documentation about how you managed to compromise their system.  

No head of IT is going to tell their engineers that “the pen tester was insanely talented and danced around our security like it wasn’t there, so everyone’s got to do a whole lot better starting from now.’ That gives no one information about what they need to do to plug the gaps in their security systems, or how to prevent similar vulnerabilities being added into new systems that they create.  

This is where the report comes in. The report is the only thing that makes penetration testing viable as a service. It’s the product for us. We pen testers take great pains in making sure that our reports are as detailed as possible, so you get the most value out of it and make your systems safer. You can read our full article on what a good pen test report should look like, so I’m not going to go over covered ground, but I’ll talk briefly about the four main sections of a pen test report. 

Executive summary and test Information 

This part of the report is where we give a high-level summary of what we found during the test. The Executive Summary is written in a style that we imagine will be read by the business leaders and other (assumedly) non-technical audiences. This isn’t an exhaustive description of the vulnerabilities found, but rather a concise look at the most important ones.  

We’re all about prioritisation in cybersecurity, and whilst there are often multiple conflicting priorities, things become a whole lot easier to manage if they’re put into some kind of prioritised list. More on that later. 

We also include the test information that we were given at the start of the test, and we’re careful to note what might have changed during the test as well as any constraints and/or limitations that we experienced during testing. 

Test findings 

This is typically another summary section where we give summaries of vulnerabilities, recommendations, and prioritised recommendations. This is about getting a little more actionable detail without overwhelming the reader.  

Many of the vulnerabilities found are self-evident, such as Outdated Apache Server 2.2.11 so you can read the brief description and go and remediate. Our prioritised recommendations are also beneficial to you, as this is your tester’s view of what updates and upgrades are essential. 

Detailed findings 

This is the meat of the report, an exhaustive list of what was found on the test, with all the findings classified and ranked in terms of severity.  

The most severe vulnerabilities are always first, ranked Critical and High. Following that are Medium, Low, and Informational findings. Informational findings are things that aren’t vulnerabilities, more so curiosities or notable deviations from best practice. 

This is where we show the gory details about the vulnerabilities that we’ve found. If they’re complex, multi-stage vulnerabilities like how a Domain Administrator account was compromised, then expect to see a step-by-step guide to compromising that network. The same is true of application testing and seen more often in reports as there is usually the necessity to show the specific requests that are made. 

Appendix 

This section outlines the tools we use and any further information about methodologies, which can change depending on the test. If there’s also a vast number of affected hosts for a given vulnerability, then we might also put them in an appendix to keep things neat. 

Stage 7: Remediation and re-testing 

We pen testers won’t fix the things we find, nor would you, or your engineers want us fixing it. These are your systems, not ours. But the remediation of issues can be hugely important, especially if they’re critical or even high severity vulnerabilities.  

These levels of vulnerability are extremely important to get fixed in a prompt fashion. From an organisational standpoint, having known, serious vulnerabilities in your system and not making reasonable efforts to fix them isn’t a great look.  

Unlike what Hollywood will have you believe, those of us who work in IT and security much prefer quiet, drama-free days at work instead of running between forest fires with buckets of water. As such, many pen testing companies offer free re-tests of high and critical vulnerabilities within a certain period. This also includes reissuing the report so that it accurately reflects the vulnerabilities. 

This is a great incentive and will give you peace of mind, knowing the vulnerabilities we’ve found are being fixed and retested. 

Wrapping up 

So, there we have it—an in-depth look at the penetration testing process with original insights from a pen tester. My hope is that there’s enough information to give you the knowledge you need when you’re engaging with a pen test partner and help you get the most value out of your penetration tests. 

Need help?

If your organisation needs help running a penetration test on an application or infrastructure, we’re here to help. We can assess your environment and run a full penetration test. We can also advise you on any follow-up actions or remediations from our findings. Contact us for a friendly chat. 

It’s also important to note that a regular penetration testing regime can help ensure you are NIS 2 compliant.