The EDPB’s opinion on AI Models and data protection

Article 64(2) of the General Data Protection Regulation (“GDPR”) provides a mechanism under which supervisory authorities can ask the European Data Protection Board (“EDPB”) for an Opinion in relation to matters that will affect more than one Member State. Using this procedure, the Irish Data Protection Authority (“DPA”) asked the EDPB for its Opinion on the use of personal data for the development and deployment of Artificial Intelligence (“AI”) models.

The request was made on 4th September 2024 and on 18^th December 2024, the EDPB published its response.

The DPA responded by welcoming the Opinion.

What information did the Irish DPA want clarified?

The request from the DPA asked the EDPB four questions, namely:

1. Under what circumstances may an AI Model be considered as ‘anonymous’?;
2. How controllers may demonstrate the appropriateness of legitimate interest as a legal basis for personal data processing to create, update and/or develop an AI Model;
3. How controllers may demonstrate the appropriateness of legitimate interests as a legal basis for personal data processing to deploy an AI Model; and
4. What are the consequences of an unlawful processing of personal data in the development phase of an AI model on the subsequent processing or operation of the AI model?

The EDPB addressed each of the above questions in its Opinion, providing guidance and criteria, together with three scenarios in answer to the fourth question.

EDPB’s response to the questions

A summary of the EDPB’s Opinion in respect of each question is set out below:

1. Under what circumstances may an AI Model be considered as ‘anonymous’?

In its Opinion, the EDPB indicates that AI Models trained on personal data would not, in every case, be classed as anonymous and each one would need to be assessed on a case-by-case basis, taking into account the following:

• The likelihood of identifying, whether directly or indirectly, the individuals whose personal data was used to develop the model; and
• The likelihood that personal data can be extracted from the AI Model through queries.

In making the assessment, the EDPB advises that it is necessary to take into account “all the means reasonably likely to be used” by the controller or another person to extract personal data. If it is not possible to identify an individual, link or infer information from a dataset, then it will be considered anonymous.

However, robust documentation would need to be provided to demonstrate how the AI Model is anonymous and, in their Opinion, the EDPB provide a non-exhaustive list of methods that may be used in support. The EDPB also states that the WP29 Opinion 05/2014 on Anonymisation Techniques should be considered when assessing anonymity.

In applying the above and making the assessments, organisations must bear in mind that, potentially, processes within an AI Model could be reverse engineered such that, in certain circumstances, elements of the training data (i.e personal data) can be reconstructed, leading to the possibility that individuals could be identified.

This risk can arise due to the advanced pattern recognition capabilities of AI and the increasing availability of auxiliary datasets, emphasising the need for robust data protection and AI governance compliance measures.

2. How may controllers demonstrate the appropriateness of legitimate interest as a legal basis for personal data processing to create, update and/or develop an AI Model?

The EDPB addressed questions two and three together. See below.

3. How may controllers demonstrate the appropriateness of legitimate interests as a legal basis for personal data processing to deploy an AI Model?

First of all, the EDPB, emphasises that there is no hierarchy between the different lawful bases and the most suitable one should be applied in each case. That said, in considering the suitability of legitimate interests as a lawful basis to develop and deploy an AI Model, the EDPB reiterates the importance of conducting a thorough legitimate interests assessment with three clear stages, namely:

1. Identifying a legitimate interest;
2. Establishing that the processing is necessary for the legitimate interest; and
3. Balancing the interests of the controller or third party against the interests, fundamental rights and freedoms of the data subjects.

See our recent article on legitimate interests for further guidance on this lawful basis.

The EDPB highlights that, whilst all the GDPR principles and data subjects’ rights are important, the most significant principles and data subjects’ rights to consider in relation to the use of legitimate interests are:

• Accountability
• Lawfulness, fairness and transparency
• Purpose limitation
• Data minimisation
• Right to object

The EDPB lists a number of mitigating measures that may be implemented to limit the impact of the processing on data subjects, which include:

• Technical measures such as pseudonymisation;
• Measures that facilitate data subjects’ rights, such as unconditional opt-out;
• Transparency measures such as media campaigns and other publicly and easily accessible communications; and
• Mitigating measures in relation to web-scraping, such as excluding the collection of data from websites that object to scraping, excluding the collection of sensitive data and data that involves risks to vulnerable people,e such as those who might be subject to abuse if the information were released publicly.

4. What are the consequences of an unlawful processing of personal data in the development phase of an AI model on the subsequent processing or operation of the AI model?

In addressing this question, the EDPB provides three scenarios:

• In the first scenario, the EDPB considered a situation in which a controller processed personal data unlawfully in order to develop an AI Model, retained the data within the AI Model and then processed it again when deploying the AI Model.

In this situation, the personal data is not anonymous because it is retained within the AI Model, and the GDPR, therefore, applies. The EDPB states that the unlawful processing of personal data in the development stage could impact the subsequent processing.

For example, if the personal data was collected without a valid lawful basis in the development stage, the individuals affected could request that their data be erased (subject to conditions) or the supervisory authority could order the controller to delete the data. This could, therefore, significantly impact many organisations.

For example, a provider of an AI system could potentially have to go back to the drawing board and create an entirely new model where the appropriate lawful basis for processing has not been applied for data collection, as they may not technically be able to delete information from their system.

Also, if an organisation is deploying an AI system where a valid lawful basis has not been applied for collection, this could significantly disrupt an organisation’s operations if they are using an AI system to deliver services and their AI system is deemed non-compliant by a supervisory authority.. The EDPB advises that each supervisory authority would need to assess the position on a case-by-case basis.

• In the second scenario, the EDPB considered a situation in which one controller processed personal data unlawfully in order to develop an AI Model, retained personal data within the AI Model, and subsequently, the personal data is processed again, by a different controller, when deploying the AI Model. This scenario is very similar to the first, but in this case, the personal data is processed by a different controller at the deployment phase.

In this situation, the EDPB highlights the importance of ascertaining which party is responsible for which obligations under the GDPR, ensuring appropriate assessments are carried out at material times, and that robust documentation is in place to demonstrate that the obligations have been complied with.

For example, in order to satisfy the Accountability Principle under Article 5(2) of the GDPR, the controller wishing to deploy the AI Model would need to show that it had conducted a thorough assessment to ascertain that a valid lawful basis was identified for processing the personal data at development stage and that the GDPR Principles under Article 5(1) were complied with.

The EDPB also comments that an EU Declaration of Conformity, which is required under the EU AI Act for high-risk AI Systems, would be taken into consideration, but that this would not necessarily be unequivocal proof that the AI Model complies with the GDPR.

• In the third scenario, the EDPB considered a situation where one controller unlawfully processed personal data at development stage but ensured that it was anonymised before the AI Model was deployed by another controller.

In this case, the EDPB confirms that, as the AI Model does not contain personal data at the deployment stage, the GDPR does not apply and that the initial unlawful processing would not impact the AI Model once deployed. However, the EDPB commented that merely asserting that the data has been anonymised would not be sufficient and that this would need to be assessed on a case-by-case basis, and the controller would need to provide documentary evidence to prove what processes they followed to anonymise the data.

Therefore, whilst the EDPB has provided guidance, the legality of the AI Model would very much depend on the specific circumstances of each case. However, in relation to each of the above three scenarios, the EDPB highlighted the powers open to supervisory authorities to take action against organisations that fail to comply with their obligations under the GDPR.

Next steps

The EDPB has indicated that there is more guidance to follow. However,  at this stage, we would suggest that it would be prudent for organisations to take the following steps:

1. Identify and document your personal data processing activities that involve AI;
2. carry out a detailed Data Protection Impact Assessment, where necessary, such assessment to include ascertaining whether the personal data was lawfully processed at the time of collection;
3. If relying on legitimate interests as the lawful basis for processing, conduct a detailed legitimate interests assessment;
4. ensure that data subjects’ rights are supported;
5. apply appropriate technical measures where possible to ensure robust data minimisation techniques are applied when processing any personal data within an AI model and anonymise/pseudonymise data where possible; and
6. Prepare and retain appropriate documentation to support accountability requirements e.g. policies/procedures and technical documentation.

If you would like any help in understanding any of the above or would like to discuss how we can support your organisation from an AI governance perspective, please get in touch using the form below or visit our AI Governance Services page.