Applicable to: AI providers and deployers.
Date of application: 2 February 2025.
Objective: prohibit the use of AI systems that exploit the vulnerabilities of an individual or a specific group of people, causing them significant harm.
Regulation (EU) No 1689/2024 on Artificial Intelligence (AI Act) of 13 June 2024 introduced a series of prohibitions within the European Union on practices involving the use of AI systems that pose an ‘unacceptable’ risk (the highest level assigned to certain AI systems in the risk-based approach adopted by the AI Act).
In this article, we will analyse the practice covered by Article 5(1)(b) of the AI Act, under which the following is prohibited:
the placing on the market, the putting into service or the use of an AI system that exploits any of the vulnerabilities of a natural person or a specific group of persons due to their age, disability or a specific social or economic situation, with the objective, or the effect, of materially distorting the behaviour of that person or a person belonging to that group in a manner that causes or is reasonably likely to cause that person or another person significant harm.
The elements that characterise the prohibited practice are:
1) the fact that the AI system is placed on the market, put into service or used in the European Union
2) the exploitation of the vulnerabilities of a natural person or a specific group of persons, due to age, disability or a specific social or economic situation
3) that the techniques are intended to have, or have the effect of, materially distorting the behaviour of a person or a person belonging to such a group
4) that the behaviour distorted by the prohibited practice causes, or is reasonably likely to cause, significant harm to that person or to another person.
Vulnerability may depend on age (for example: children or older people), disability or socio-economic circumstances.
On this point, the Guidelines approved by the European Commission on 29 July 2025 clarified that children are particularly susceptible to manipulation, due to their limited ability to critically assess and understand what is real and the underlying objectives of AI-based interactions. A toy based on an AI system that encourages a child to undertake dangerous challenges (such as climbing furniture) would therefore fall within the scope of the prohibition in question.
Similar conclusions should be drawn with regard to a predictive algorithm used to target, on the basis of postcodes, advertisements for predatory financial products at people living in low-income areas and facing financial hardship, thereby taking advantage of their susceptibility to such promotional messages and causing them significant financial harm.
The prohibition applies both where the system is used with the intention of distorting users’ behaviour and where such distortion is merely a consequence of the adoption of the technique that exploits vulnerability.
The concept of ‘significant harm’, which is necessary for the prohibition to apply, is not defined within the AI Act. The Guidelines have, however, specified that it must result in a significant negative impact on a person’s physical or psychological health, or on their financial and economic interests; or, in the case of a practice prohibited on the grounds that it exploits vulnerability, on another person as well.
Articoli Correlati