Must AI systems comply with the GDPR?

AI Act or GDPR: which legal framework applies to my AI system?

• The GDPR applies wherever an organisation processes personal data—whether for communication, commercial or analytical purposes.
• The AI Act applies where an Artificial Intelligence System (AIS) is implemented.

The AI Act defines an AI system as “a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments”.

• Where AI systems process personal data, the two regulations are both applicable and complementary.

The remainder of this article outlines best practices for ensuring that AI systems processing personal data remain compliant with both regulations.

Phase 1: establishing dedicated governance

Before developing an AIS, the organisation responsible must implement a structured governance framework and establish a steering committee to oversee compliance with both the GDPR and the AI Act throughout the system’s entire lifecycle. The committee should assess the need for the AIS, map data processing activities, develop an action plan, and monitor the implementation of legal recommendations.

Phase 2: defining the roles and responsibilities of stakeholders

The development and deployment of an AIS typically involves multiple stakeholders—developers, vendors, end users, providers of training data, and so forth. For each phase of the system’s lifecycle, it is essential to identify the role and responsibility of each party with regard to data processing. This requires a detailed analysis of the AIS lifecycle to assign responsibility—either as data controller or processor—for the stages of development, training, deployment, and operation.

Let’s take a practical example: if an AIS is developed according to a client’s specifications for internal use, the client is likely to be the data controller, while the development team acts as the processor. Conversely, if a company develops an AIS and its functionalities independently for commercial sale “off the shelf”, the company will be considered the data controller during both the design and training phases.

Phase 3: determining the purpose of processing under the GDPR

Article 5 of the GDPR sets out the core principles governing the processing of personal data. Any use of personal data must meet the criteria of lawfulness, data minimisation, proportionality, and transparency—ensuring that individuals can modify or withdraw consent. The use of data must be governed by a clearly defined purpose. This purpose may remain constant throughout the AIS lifecycle or evolve during its development and use.

Phase 4: selecting the legal basis for processing

Once the purpose of the AIS has been defined, it is necessary to select the right legal basis for processing under the GDPR. These legal bases are the following : consent; contract; legal obligation; vital interests; public task; or legitimate interests.

In the context of AIS development, legitimate interest can be a suitable basis, provided that the requirements of the GDPR and the former Article 29 Working Party (G29) are met—namely, restricting data use to what is strictly necessary, facilitating the exercise of data subjects’ rights, and ensuring that the AIS is free from discriminatory bias.

The use of third-party datasets requires heightened diligence, particularly to verify the compatibility between the original purpose of data collection and the intended use. Where such compatibility cannot be demonstrated, data usage becomes problematic. As a general rule, it is strongly recommended to confirm that third-party datasets do not contain sensitive information and were collected in compliance with the GDPR.

Phase 5: informing individuals and facilitating the exercise of their rights

Transparency is a core principle of the GDPR. Organisations must therefore develop a clear strategy to ensure that individuals receive accessible and comprehensible information about the processing of their personal data by AI systems—at every stage of the data processing lifecycle.

Furthermore, to ensure compliance and avoid penalties, AIS must incorporate, by design, features that enable users to exercise their rights of access, rectification, or removal of data.

Phase 6: applying the principles of data minimisation and accuracy

AI systems require substantial volumes of data to function effectively—an apparent contradiction with the GDPR’s principle of minimisation. Nevertheless, where personal data is involved, only data that is strictly necessary should be collected and used. In addition, the AIS must offer guarantees concerning data accuracy and reliability, in order to avoid bias.

To that end, French CNIL recommends the following best practices:

• Implementing pseudonymisation, data aggregation or synthetic data generation;
• Ensuring that data is deleted once it is no longer required;
• Continuously verifying data accuracy throughout the AIS lifecycle;
• Combining automated alert systems with human oversight to assess data sources and their accuracy.

Phase 7: Securing data in AI Systems

Data protection is a critical requirement under the GDPR. Failure to comply with security obligations—or to report breaches—can lead to severe penalties.

Data protection should be ensured through both technical and organisational measures, including:

• Data encryption;
• Strict access controls;
• Regular security audits.

Where data is transferred outside the European Economic Area, contractual safeguards must be in place to guarantee a level of protection equivalent to that required by the GDPR.

If the AIS presents a high level of risk, a Data Protection Impact Assessment (DPIA) must be carried out before deployment.

Fostering a culture of compliance and documenting all actions

Ensuring that an AI system complies with both the AI Act and the GDPR should not fall to a select few. It must be a collective effort, supported by all stakeholders involved in the system’s design and implementation. Each actor should carefully document all phases of the AIS lifecycle, specifying the types of data used, and the respective roles and obligations of all parties.

It is also advisable to organise awareness sessions on the requirements and good practices related to the AI Act and GDPR, for anyone involved in data collection or system development.

Disclaimer

The opinions, presentations, figures and estimates set forth on the website including in the blog are for informational purposes only and should not be construed as legal advice. For legal advice you should contact a legal professional in your jurisdiction.

The use of any content on this website, including in this blog, for any commercial purposes, including resale, is prohibited, unless permission is first obtained from Vaultinum. Request for permission should state the purpose and the extent of the reproduction. For non-commercial purposes, all material in this publication may be freely quoted or reprinted, but acknowledgement is required, together with a link to this website.