In this blog, Principal Security Consultant for Evalian, Keith Sharp, takes a deep dive into API security and why it should be a crucial part of your organisation’s security strategy. But first, let’s cover the basics…
Table of Contents
What is an API?
An application programming interface (“API”) is a software intermediary between different applications. Using a set of defined rules enables systems to communicate with each other. You can think of APIs as connective tissue or messengers that run between systems, allowing data transfer from application to application. The two most common types of APIs are Representational State Transfer (“REST”) and Simple Object Access Protocol (“SOAP”).
What is API penetration testing?
An application programming interface (“API”) penetration test is a security assessment carried out by a penetration tester to validate that the APIs in scope are appropriately secured. The tester uses the same tactics, tools, and techniques as would be used by a real-world attacker. The objective of the pen test is to discover vulnerabilities that could impact the confidentiality, integrity, or availability of an enterprise’s data or infrastructure – and provide steps for them to be remediated.
An abundance of APIs
It is likely that in the interconnected digital world the average person owning a smartphone, or laptop or using a computer for work, interacts with multiple APIs daily.
You may not know precisely what an API (Application Programming Interface) is, and much like many terms used in everyday jargon in the tech world, such as OS (operating system), Cloud, AI (Artificial Intelligence), or algorithm – they can all be defined in many ways, in different contexts and be generally confusing to the layperson. However, what most people do not realise is that they use APIs every day, and without realising.
Most interactions on your smartphone will exchange data with an API to pull information from the Internet, either directly from a data source owned and managed by the application’s creators, or from a 3rd party service (e.g., an app that uses data from Google Maps will do so via Google Map’s API). API traffic accounts for the lion’s share of Internet traffic today due to digital transformation and application development increasingly becoming cloud-based.
Similarly, when interacting with websites, most of the heavy lifting in terms of data retrieval is commonly supported by the integration of API technology, this technology varies and is often tied to the needs of the application. That is, APIs are not simply a defined and regulated web standard like HTML, but more of a general construct, which is why, like terms such as “Cloud,” everyone has different understandings of what APIs mean.
Why do we use APIs?
APIs are, as a core definition, a software development tool which allows complexity to be abstracted, to make life easier when giving a system functionality (which otherwise could be laborious to create from scratch). Needless to say, APIs are immensely popular with 90% of developers using them in some capacity. APIs take the headache out of writing complex code, by allowing developers to simply write directives that can be understood in a few keywords strung together with a logical syntax.
APIs in Web and Mobile app development
The most typical usage of APIs today is their integration into web and mobile app development and can be used to do a great many things in the context of application processing. It is worth bearing in mind that APIs can retrieve data from systems locally:
- They can interact with data stored in a web browser session or by a mobile app
- They can retrieve and manipulate files stored on your device and
- They can process audio or video from the hardware of the local device and a myriad of other sensor input data.
But for the purposes of this article, we will be focusing on APIs that send and retrieve data via remote systems. APIs on remote servers are used by applications to achieve specific functional tasks (most typically, creating, reading, updating, and deleting data (CRUD)). and is typically where the greatest risk lies today for an organisation implementing APIs as part of their internal operations or their online service offerings.
Understanding the threat landscape of API attacks
It is common to find web and mobile application-related APIs exposed directly to the Internet. This means any organisation invested in deploying APIs as part of their B2B, B2C services or internal operations, should note they come with their own set of unique security-related threats.
The motivation for a detailed understanding of an API’s operating definition is key to understanding the unique threats facing APIs versus those affecting applications. To interact with APIs, you need to know the precise rules and restrictions that create a successful set of requests.
APIs can expose the underpinning operational logic of an application; this is often evident in the online documentation that organisations – or 3rd party API providers – may publish. In fact, this is how numerous API security tools operate today, by feeding them the blueprints of the API, to which skeleton API requests are then formulated, ready to be fired towards the related service.
Once the basics of an API’s requests are understood, the next layer of abstraction to get right is knowing the type and format of data that needs to be issued in the requests. Accurate information is vital for the requests to be processed successfully by the API.
Furthermore, the construction of certain well-known API enumeration tools, at their core, operate using wordlists and request structures taken from analysing thousands of publicly exposed API documentations, including API routes from well-known API-driven frameworks, such as Flask, Rails, Express, Django as well as more bespoke projects found in the wild.
Unless instructions are provided it may not be clear what type of data and in what format should be placed into a request parameter. Every parameter value sent to an API, from date formats, policy numbers, and user identifiers to PIN codes – must be issued correctly or the API will not be able to successfully process the information it receives.
Some APIs, such as GraphQL APIs, have built-in mechanisms that allow the entirety of the API’s schema to be downloaded and investigated by attackers with certain tools, like obtaining blueprints to a building prior to attempting a break-in. This behaviour can be advantageous to threat actors. If they can determine any misconfiguration or vulnerabilities associated with those areas, they can target parts of the APIs that look to be the most lucrative or disruptive.
That being said, sometimes a user must have an account and access to a system to gather such data, therefore API information leakage to the public could be limited to authenticated users. Phew, risk diverted? Not quite. It is important to remember that authenticated users are as much a threat as unauthenticated users with regard to API security. In certain sectors, large numbers of users need an account to use an online service and can therefore self-register – this is commonly the case with social media platforms, retail, travel, and professional services (e.g., recruitment). Therefore, hiding APIs or details of their inner workings is not a strategy that can work to stop attackers from attempting to manipulate them.
APIs handling personal data (PII), have become a valued target for attackers. Mining data or enumerating information about individuals once authenticated to an application with API configuration weaknesses, is a widespread practice.
Another common threat includes the bypassing of authorisation controls to APIs which are intended to only be reachable behind paywalls. Attackers recognise that in complex application architectures, an API is often something that can be inconsistently configured, with human error leading to unexpected application processing or weakness in the intended access controls. You may find our recent article useful on Cloud security misconfigurations and how to prevent them.
Access attacks against APIs have their own unique set of threats, commonly stemming from the unauthorised sharing of API keys, leakage in application responses, or keys being hardcoded into mobile applications. Attackers are aware of these bad security practices and can even use tools to hunt through application responses or lines of client-side code looking for these items.
The bottom line is, the more information an attacker has about your API, the more likely they will be able to find security holes in it.
The growing threat of API attacks in 2023
Recent reports such as Imperva’s “Quantifying the Cost of API Insecurity” show trends indicating that average losses per year against API-related cyber security are the greatest against the companies with the highest revenues, with 25% of cyber events experienced being attributed to API insecurity. So, if your organisation relies heavily on APIs and you are a highly profitable business, attackers are probably going to go hunting for APIs looking for exploit gold.
Additional trends in the above report determine that technology-dependent sectors experience higher frequencies of API-related cyber events and not all sectors are as likely as others to experience API-related cyber-attacks. Industries, where API-related cyber threat events are highest, include Information, Professional Services, Retail, and Finance. A conclusion in the report states that in the US alone, up to 23 billion (USD) in losses could be avoided annually if companies maintained perfect API security. API security appears to be a major security gap not being adequately addressed even by the largest corporations with the deepest pockets.
Notable API breaches
Organisations with large databases are prime targets for hackers and therefore any APIs designed to get data in and out of them, by association are going to be sought out by them. Notably in the past year 200 million user profiles were extracted from Twitter (2022), and in the same year a study by CloudSEK showed that thousands of mobile applications were leaking valid Twitter API keys and their secrets.
The Australian telecommunications giant Optus suffered an API-related breach in September 2022, which could have been easily prevented. Data relating to 10 million customers was exfiltrated from Optus databases, including customer names, dates of birth, phone numbers, email addresses, home addresses, driver’s license information, and passport numbers. The attack was said to stem from the ability to access the affected database via an unauthenticated API. An unauthenticated API takes no effort to breach, although it may not be obvious to find.
Other breaches in the public domain from last year included attacks on API flaws in the automotive sector. Specifically, KIA and Ford were found vulnerable to APIs exposing customer data, but in addition, vehicle location information was being leaked alongside data that could allow the complete compromise of the car’s remote management system. The breach of these automotive APIs had the potential to allow malicious actors to locate and unlock vehicles and start and/or disable the engines.
As more and more physical objects in the real world become smart enabled, their likelihood to connect to cloud services driven predominantly by APIs will follow, along with their potential for misconfiguration leading to unauthorised use or abuse.
The laws of API attraction
The reason attackers go after APIs is obviously related to the proliferation of APIs within the web and mobile applications, we use every day. The complex integrations of APIs in large application architectures are the reason they are successful and have more to do with the human factor as opposed to the technologies used in the development of APIs.
The larger the system’s surface, the higher the probability of mistakes creeping into deployments that rely heavily on APIs to interconnect components. Within the last year at Evalian we have seen much smaller, self-contained web and mobile applications using APIs in insecure ways. It is becoming apparent that the most common and most devastating cyber incidents are simply due to misconfigurations often relating to API integrations in web and mobile applications.
Common API vulnerabilities found by Evalian in 2023
So far, during 2023, we have seen multiple instances of the following types of vulnerabilities when performing tests that involve APIs (i.e., during web application penetration tests, mobile application pen tests, and API security assessments).
A common High risk we frequently observe during API penetration tests, is API calls being unintentionally available to unauthenticated users. In observing such oversights, we have been able to retrieve data about users (typically in B2C solutions), and even data about different client companies (typically in B2B solutions) in multi-tenant environments, without requiring any valid access or session tokens to do so.
Broken Object Level Authorisation
One of the most common API vulnerabilities reported relates to finding improper restrictions on data. Applications often must break down data into discrete sets of data, and these sets can have a unique identifier that is used to fetch the data efficiently. For example, if an application retrieves the contact details for an organisation, it may use a specific organisation ID value to do so. This frequent practice is well known to attackers and attempts to enumerate such ID values by looking for misconfigurations is a common strategy. This year we have seen many instances where we have been able to break out of the expected API restrictions and gain access to data in multi-tenant applications simply by manipulating such object identifiers.
Embedded API keys
It is common for 3rd party API keys to be seen embedded in code used by an application or pulled from an application in an application response. Recently we succeeded in finding an API key that was hardcoded into an exposed component within an application architecture. This, alongside the fact that the API was also improperly protected, meant anyone with the key, could gain administrator-level access to a 3rd party service, and lead to the exposure of extremely sensitive and personal information.
Our testers have uncovered a compromise in the integrity of data being sent to APIs due to a lack of strict verifications or validation. As a result, our pen testers have been able to mirror the actions of a threat actor by altering pricing information being carried by an API and changing it to different values. An attacker could have used this method for monetary gain.
What has been interesting across our API security findings identified in the past year is that, typically, the API-related findings feature as unusually high-risk issues amongst an otherwise robust implementation of a web or mobile application and/or API.
The API vulnerabilities discussed above usually stem from authentication and authorisation issues which are largely related to the configuration of the API’s data processing at the server side. It comes as no surprise that the top issues in the latest OWASP (Open Web Application Security Project) Top 10 for API Security are related to these types of oversights.
OWASP Top 10 – API Security
Many people are familiar with the OWASP top 10 as it relates to application security, which is updated every few years, based on the most common types of vulnerabilities actively facing web applications. If an attacker can get access to your API, even partially, it could lead to the theft of data, unauthorised access, or disruption to services.
OWASP define API Security separately, as a subset of application security – and the key threats typically align to those relating to data breaches, or misuse. As a security baseline, these threats are covered in Evalian’s manual-driven testing approach with regard to API penetration tests, and web and mobile application penetration tests, where APIs are in the scope of the assessment.
OWASP has published the API project ahead of the official release of the API Security Top 10 2023 which lists the vulnerabilities likely to be included.
Within the API tests performed over the last 12 months at Evalian, we can see definite correlations with the issues at the top of the OWASP list being the most prevalent in Critical and High-risk findings.
To assess an API’s security posture effectively, some work is required up front by the test provider, to understand how the API operates in situ. Authentication and authorisation are key tasks to cover during a test, so a tester needs to ensure that the scope of an API test is clear from the outset.
There are also some other factors that come into play when determining how an API test is approached. These can include restrictions or preparations that may need to be implemented prior to the testing.
Penetration tests should form a core part of your API security programme. The development industry is, broadly speaking, “shifting left”, meaning there is more focus on security by design before launching APIs. However, even in these instances, it’s frequently possible that errors and vulnerabilities remain. For this reason, we recommend performing an API penetration test just before launch – and then at least annually after that.
In cases where significant changes are made to the web application relating to the API, you should also conduct a penetration test. If you’re unable to organise a penetration test prior to the API’s deployment, we advise getting one as soon as possible thereafter.
Scoping an API assessment
Assessing the security of APIs can present some challenges and limitations because often it can be difficult to concisely define the perspectives of how an API is used within its wider application architecture. Non-developers, who are not close to the technical details of the API’s integration and use cases may not know how to describe its exposure to users, and/or other systems or providers of penetration testing services.
We work closely with our clients to understand these perspectives to accurately scope and complete API penetration tests.
Evalian’s expertise lies in assessing the following common API architectural deployments utilised heavily by web and mobile applications:
- REST APIs – which are essentially API implementations that use HTTP communications sent in either JSON, HTML, XML, or plain text format which interact with data stores on the server side. These are the most popular API architectural deployments in use today.
- SOAP APIs – This is an official API protocol maintained by the W3C, which uses the SOAP protocol and messaging format and is XML based. The SOAP format uses a WDSL (web service description language) to define the API. SOAP is less popular/common in public-facing web applications and mobile applications but has a history with respect to enterprise-level developments, due to its reliability in the context of database transactions.
- GraphQL APIs – GraphQL is a query language, queries are issued via the client and validated and then executed on the server side returning the response. GraphQL has use cases where it excels in terms of performance, typically when dealing with fetching data from multiple sources. It also supports controls with respect to managing data fetching and regulating bandwidth for devices with less processing power.
Clients often state a requirement to test their APIs but may not realise that their API is simply the mechanism by which their web or mobile application does all its work. The approach to testing would be typically testing the API through its sole-intended integration into the web and/or mobile application. Therefore, we test the API by testing the respective mobile or web application and this can apply to REST, SOAP and GraphQL APIs.
When independent API testing is required (that is, testing an API outside of the application or mobile app context), the process of testing requires more preparation than required from testing through a series of web or mobile application screens. Without any buttons, input boxes, forms or visual indications, an API can be difficult to assess.
Usually, we find a discussion between ourselves, and the client is required to determine an accurate scope for API testing. This can often, however, be a straightforward process depending on the level of knowledge a client has on their API and helps us to determine the timescale, effort to test and necessary approach.
Given that the scoping exercise with our client is such a crucial step in the process and ensures an effective and efficient API test, we aim to understand the context by looking at the following key areas:
- Complexity – What type of API is the target – REST, SOAP, GraphQL, what is the technology stack of the hosting server(s), how many services are in scope, how many API calls does each service support and what type of operations and data do they encompass.
- Exposure – Is the API exposed to the Internet or is it shielded from direct access? This is important to determine certain details of how the target will need to be assessed.
- Modes of Access – What authentication processes or other access controls are utilised to protect some or all the API’s functionality (i.e., what is the proportion of authenticated vs. unauthenticated endpoints).
- Integration – what and/or who consumes the API and under what controls.
- Restrictions – can the API be reached remotely, are there request limits, or rate limits? Are restrictions placed upon access tokens (e.g., short expiry), or is access controlled via IP address? Do special setup processes need to be completed out-of-band for consumers to use the API, e.g., paywalls, subscription processes, identity, or background checks?
In some cases, we may need to dig deeper to understand finer details during the scoping of an API test. We may need to ask things like “Where is the API hosted?” or if you allow others to use your API. A good penetration testing provider will talk you through these and support you in understanding exactly what information is needed at the scoping step so that there are no surprises later down the line in terms of additional costs.
We can help you understand more about how penetration testing is scoped and how much a pen test costs in the UK.
If you would like to hear more about how we support organisations like yours with our specialist penetration testing services, visit our case studies page:
How does an API pen test work?
If certain aspects of APIs are not clarified prior to an API assessment, testing APIs can often be like trying to assemble furniture from Ikea without having any instructions! Remember, an API Therefore, if we do not have access to or familiarity with the application or mobile app that consumes the API, then it can be extremely difficult to perform any meaningful security tests. Comprehensive testing will not be possible unless the tester has access to schema, definitions files, sample API requests/responses and test data.
API penetration testing is not a black-box exercise. generic vulnerability scanning will not find anything other than common server configuration issues or locate common API routes. It is not until a human being reviews the requests and responses of the API processes that any meaningful or methodical security audit can begin and so APIs should be approached, when feasible, from a grey or white-box perspective. At Evalian, we always aim to test an API from a manual perspective in the main, driving our toolsets by knowing all the available API requests and their expected responses so we can identify any API calls that could be accessed or abused by threat actors.
Furthermore, the different types of API (REST, SOAP, and GraphQL) can present their own unique classes of vulnerabilities, so we use specific tools and software to allow us to be efficient against assessing whatever the underlying supporting technology is.
The stages of an API test
An API penetration test occurs in five stages: preparation, reconnaissance, vulnerability analysis, exploitation and reporting. A specialist penetration tester can assess all kinds of API implementations for potential vulnerabilities. These tests are an essential way to determine the security of your APIs and are necessary to identify and manage exploitable weaknesses.
For an overview of different types of penetration tests, read our guide to penetration testing.
At the end of the penetration test, the tester should provide you with a list of the vulnerabilities discovered, along with guidance on how to remediate them in a report. The report should be followed by a ‘wash-up call’, where you can discuss the findings, vulnerabilities, and recommendations in more detail.
A typical penetration test is structured as follows:
- Preparation: Firstly, the testing provider and you will agree on the scope of the test, which will include identifying your testing goals, going over the rules of engagement and confirming the project’s scope and timeline.
- Reconnaissance: Before beginning testing, the tester will gather as much information as possible about the target API, including details such as authentication credentials, IP addresses, URLs and example test cases.
- Vulnerability analysis: Armed with the information they have gathered, the tester will look to identify vulnerabilities in the target API – looking at both the application and network layers. To do this, they will log machine names, network sources and application services to gain a deeper understanding of the API and its potential weaknesses. The tester will use a mixture of automated vulnerability scanning and manual techniques here. As they go, they will note which vulnerabilities present the most risk and prioritise their testing based on these findings.
- Exploitation: During this stage, the tester will test the vulnerabilities they have already identified to confirm they are exploitable. They will determine which vulnerabilities truly pose the most risk and document their findings. This phase is necessary for deciding whether vulnerabilities are real and true positives, which may lead to further discovering vulnerabilities that are only visible post-exploitation.
- Reporting: Once this is complete, it’s time to report the findings to the client. This will take the form of a detailed report, often along with the option to have a debrief meeting to talk through the results and answer any questions. The testers will advise which vulnerabilities to prioritise fixing based on their risk factor.
APIs are clearly a viable attack surface for hackers, with high profile breaches involving APIs often being the result of significant oversights in basic security measures. APIs are used in abundance by individuals and businesses alike. Given that their operational parameter is abstract, can lead to a weakening of an organisation’s security posture from things like improper implementation, configuration, or inadequate testing at various stages in their integration within mobile and web application architectures.
That being said, with the appropriate level of understanding of how your APIs are used, and their operational parameters, it is possible to cover security baselines such as the OWASP top 10 – API Security and eradicate the potential for immediate exploitation of common misconfigurations.
Learn more about Evalian’s web and mobile app pen testing services:
Get a fast quote for API testing
API Security can be improved by putting APIs through rigorous penetration testing, using the right supporting details and information to allow such assessments to walk the functional surface of the API successfully.
At Evalian we can work with you to understand how best to approach the testing of your API integrations, whether through a web application client, a mobile client, or independently using appropriate sample data. As we have seen in the past year, there can be a lot to lose from an API Security event.
Contact our expert team today for a friendly discussion about how we can best support you with our penetration testing services, whether it’s mobile application pen testing you need, web application testing, or external/internal infrastructure testing, we’ve got you covered. We offer remote or on-site testing at affordable rates, a detailed pen testing report, a wash-up call and a free retest for remediated issues.
Image by storyset on Freepik