AI-powered cyber threats: The ICO’s Five Steps to protect your organisation
As artificial intelligence (“AI”) becomes increasingly ubiquitous, it is becoming commonplace across many sectors and, for many organisations, it is now part of their “business-as-usual” operations. AI tools, when used effectively, enable companies to work more efficiently and productively than ever before, completing tasks in a fraction of the time that a human could.
AI is also becoming increasingly affordable, driven by technological advances and growing market competition, making AI tools accessible to a wider range of organisations. In a world where time is money, AI is fast becoming an attractive proposition for organisations seeking to improve efficiency and productivity.
However, AI is not only being used for enhancing the services businesses provide to their clients by optimising processes, speeding up production and improving the quality of deliverables. It is now also being used by cybercriminals to carry out AI-powered cyberattacks, and this is an increasingly difficult problem to address.
As Ian Hulme (ICO Interim Executive Director for Regulatory Supervision) identified in his blog, Five steps to protect your organisation from AI-powered cyber threats , AI-powered attacks “…are faster, more advanced and harder to detect”. As a result, organisations need to ensure that they have a robust security posture in order to avoid succumbing to such attacks.
In its blog, the ICO sets out five key steps that organisations can take to strengthen their posture, protecting against AI cyber threats:
Seven key cyber threats organisations should be aware of
The ICO highlights seven key cyber threats that organisations need to be aware of, namely:
| RISK | ||
|---|---|---|
 | 1. AI-enhanced phishing | We are all familiar with phishing emails, but AI-generated phishing emails are much more sophisticated than we have ever seen before. The telltale signs that helped us in the past to recognise that an email was not genuine, such as spelling mistakes or poor grammar, are no longer as common when AI is used. The messages can be flawless, containing no spelling or grammatical errors. They can also mimic the writing style of your trusted contacts, creating personalised text that is very convincing. The end result is that AI-enhanced phishing attacks are much more difficult to spot. For further guidance, see our blog: How to Identify Phishing Emails. |
 | 2. Deepfake social engineering | Deepfakes are AI-generated videos, pictures or audio of people which look and sound real, but are fake. It’s often easy to identify a deepfake, as they may have strange eye movements, their hair may look frozen, their lip movements may not match the audio, or their voice may sound robotic. However, with the advances in AI, they are harder to distinguish from the real thing. Also, it is becoming easier and quicker to create them. For example, previously, it was necessary to collect several hours of someone’s voice before AI could mimic it, but now a ten-second voice clip is sufficient to clone a voice. The deepfake, once created, can be used by cybercriminals to impersonate senior officials within your business to trick staff into releasing confidential data or transferring funds, creating a huge risk. |
 | 3. Automated vulnerability scanning and exploitation | Part of your organisation’s basic technical and organisational security measures should include vulnerability scanning in order to identify vulnerabilities in your computer systems and take steps to address them, such as installing patches. Cybercriminals also use vulnerability scanning, but they use it against organisations to find weaknesses and, once found, they will exploit them by launching a cyberattack, effectively breaking into your system. For further information, see: Best Practices for Vulnerability Scanning and Penetration Testing methodologies and Best Practices for Vulnerability Scanning. |
 | 4. AI-powered malware | Another part of your organisation’s technical and organisational measures should include anti-malware. This is used to prevent malware from infiltrating your systems and causing havoc. However, AI-powered malware is extremely difficult to detect as, rather than operating under a fixed code, it can adjust and adapt to its environment by rewriting its own code. As a result, the usual anti-virus tools will not be able to prevent the malware from infecting your systems. |
 | 5. Credential stuffing and password attacks | Credential stuffing happens when cybercriminals try usernames and passwords, which they have already obtained, across multiple websites or apps. This works if people have used the same login credentials for different platforms or services. According to the ICO, it is estimated that up to 65% of people use the same password across multiple sites, leaving themselves, potentially, vulnerable to cyberattacks. When AI is used to support this type of attack, cybercriminals can access vulnerable accounts in seconds. AI also enables cybercriminals to crack weak passwords at a phenomenal speed by simply using brute force (i.e trying different combinations until finding the correct one). For further information in relation to password security, see World Password Day and Cyber Essentials and Passwordless Authentication. |
 | 6. Data poisoning | Data poisoning occurs when cybercriminals use AI to alter the data your organisation is using to train AI models. The result is that your AI model learns from corrupted data, leading to outputs which are biased or manipulated to cause harm. The flaws are often very difficult to identify, and the risk of harm is significant, especially where, for example, the poisoned AI model is used in the healthcare or finance sectors or in automated vehicles. You might find this blog informative for further reading: AI SEO Poisoning: Attackers Deliver Malware & ClickFix Payloads. |
 | 7. Indirect prompt injection attacks | There are two types of prompt injection attacks, namely, direct prompt injection and indirect prompt injection. The former happens when a cybercriminal injects malicious content directly into the AI tool. The latter occurs when the malicious content is contained within documents or emails that the AI tool will access. For example, it may appear within a document that your organisation receives but, as it is invisible to the naked eye, you will be none the wiser. Perhaps the document is a lengthy report which needs to be summarised, so it is copied and pasted into an internal AI tool for this purpose. However, once in the AI tool, the malicious content is released. |
2. Get the basics right and layer your defences
Consequently, the rise of AI has created a multitude of cybersecurity risks, all of which have the potential to cause extreme harm and distress, whether that be physical, mental or financial.
In terms of protection against cyberattacks, the ICO advises, amongst other measures, that organisations should implement the five technical controls set out within the Cyber Essentials scheme and the actions described in the Cyber Governance Code of Practice Cyber Governance Code of Practice – GOV.UK
In addition to this, the ICO recommends that organisations take a layered approach to guard against cyberattacks so that, even if the cybercriminals manage to overcome one obstacle, there are several more to penetrate before they are successful.
3. Restrict access points
The ICO highlights the importance of implementing the principle of least privilege. In practice, this means that access to data should only be granted to those who need it for their job role. For example, there is no need for a receptionist to have access to payroll data. Applying this principle will help to prevent cybercriminals from moving around your IT systems so easily in the event that they do gain access.
The ICO recommends that audits should be carried out on a regular basis to ensure that access rights are still appropriate and access should be disabled immediately if it is no longer required, such as when an employee leaves.
The ICO also reminds organisations to consider third-party suppliers and recommends implementing multi-factor authentication when using third-party platforms, wherever possible.
Further, the ICO advises on conducting due diligence on your supply chain to ensure that they have implemented appropriate technical and organisational security measures and that any access they are given to your systems is proportionate for the purpose.
As a controller, full details of the technical and organisational security measures should be set out in the Data Processing Agreements with your processors.
Protect Your Business From A Data Breach
4. Improve your detection, monitoring and incident response
Whilst it is important that the above-mentioned measures are implemented, the ICO explains that it is just as important to ensure that constant security monitoring is in place so that any threats or vulnerabilities can be promptly identified and addressed.
The ICO advises that comprehensive security monitoring will include looking for any suspicious activity such as unusual login patterns and unexpected transfers of data, and AI is a useful tool to assist with this (providing it has not been tampered with by cybercriminals, of course).
Frequent vulnerability scanning and penetration testing will help identify any issues. For further guidance, see our blog on Best Practices for Vulnerability Scanning.
Further, the ICO identifies the benefits of having a robust and regularly tested incident response plan in place, insofar as this will help your organisation to become familiar with how to respond if any such events do occur. For further information, see our blogs: Complete Guide to Cyber Security Incident Response and ISO Compliance Audit Guide.
5. Protect personal data
Every cybercriminal will have their own agenda. For example, they may wish to manipulate the data you are using to train your AI model, lock data with ransomware, cause disruption etc. However, the target for some will be to steal personal data which they will then use for criminal purposes, such as fraud or identity theft.
The ICO suggests applying the following to protect personal data:
Data minimisation and storage limitation – Ensuring your organisation only collects and processes the personal data you need for the identified purpose and only retaining it for as long as is necessary for that purpose. If you don’t have the data, it cannot become a risk.
Audits – Conducting regular checks to ascertain what data you have, where it is and who has access to it to ensure that it is still appropriate and making changes, as necessary.
Training – Ensuring staff who have access to personal data undergo training, not only in data protection, but in AI-powered cyberattacks so that they are aware that the present threat landscape now includes voice cloning, deepfakes, sophisticated phishing attacks, etc. The training should be regularly updated to reflect developments.
AI Governance – Thorough AI governance is imperative if your organisation uses AI. This includes carrying out data protection impact assessments and implementing appropriate safeguards. The ICO recommends that organisations should follow the government’s AI Cyber Security Code of Practice.
Encryption and pseudonymisation – The ICO also suggests that encryption and pseudonymisation are implemented so that data is protected in the event that a personal data breach does occur.
Whilst the above suggestions have been highlighted by the ICO, they should all be part and parcel of your data protection compliance plan, in any event.
Therefore, the ICO highlights some of the main AI-powered cyberattacks that organisations are presently at risk of, together with an outline of the steps which can be taken to guard against them. The ICO acknowledges that cyber security has always been a risk, but adding AI into the mix only increases that risk. Ian Hulme says:
“None of this is new, but AI brings a renewed urgency and greater speed. Organisations can prepare themselves for future cyber threats by establishing robust security fundamentals early, applying layered defences, and ensuring human oversight in their detection and response processes.”
How Evalian can help
If your organisation needs any assistance with implementing appropriate security measures against AI-powered cyber threats, such as AI phishing attacks, understanding what is needed from an AI governance perspective or complying with the data protection legislation, we would be happy to help.
