Applicable to: AI providers and deployers.
Date of application: 2 February 2025.
Objective: prohibit the use of AI systems to assess or predict the risk of a natural person committing a criminal offence, based solely on the profiling of a natural person or on assessing their personality characteristics.
Regulation (EU) No 1689/2024 on Artificial Intelligence (AI Act) of 13 June 2024 introduced a series of prohibitions within the European Union on practices involving the use of AI systems that pose an ‘unacceptable’ risk (the highest level assigned to certain AI systems in the risk-based approach adopted by the AI Act).
In this article, we will analyse the practice covered by Article 5.1 d) of the AI Act, under which the following is prohibited:
the placing on the market, the putting into service for this specific purpose, or the use of an AI system for making risk assessments of natural persons in order to assess or predict the risk of a natural person committing a criminal offence, based solely on the profiling of a natural person or on assessing their personality traits and characteristics; this prohibition shall not apply to AI systems used to support the human assessment of the involvement of a person in a criminal activity, which is already based on objective and verifiable facts directly linked to a criminal activity.
The elements that characterize the prohibited practice are:
1) the fact that the AI system is placed on the market, put into service or used in the European Union
2) that the AI system assesses or predicts the risk of a natural person committing a criminal offence
3) that the assessment or prediction is based solely on one or both of the following:
- profiling (a concept in relation to which the AI Act refers to Article 4.4 of Regulation (EU) 2016/679 –GDPR)
- assessment of personality traits
On this point, the AI Act has clarified that, in line with the presumption of innocence, natural persons should always be judged on their actual behaviour and not on AI-predicted behaviour based solely on their profiling, personality traits or characteristics, such as nationality, place of birth, place of residence, number of children, level of debt or type of car, without a reasonable suspicion of that person being involved in a criminal activity based on objective verifiable facts and without human assessment thereof (Recital 42).
The Guidelines approved by the European Commission on 29 July 2025, have identified, as example of prohibited practices relevant to the case in question, the use, by a law enforcement authority, of an AI system to predict criminal behaviour for crimes such as terrorism solely based on individuals’ age, nationality, address, type of car, and marital status. With that system, individuals are deemed more likely to commit future offences that they have not yet committed solely based on their personal characteristics.
However, this prohibition does not apply to AI systems used to support the human assessment of the involvement of a person in a criminal activity, which is already based on objective and verifiable facts directly linked to a criminal activity.
The Guidelines cite, for example, a case in which the police is investigating the risk of a possible armed robbery and suspects two individuals. If the suspicion is based on verifiable and objective facts (such as verifiable participation and dialogues in dark web chat groups for purchasing arms), the use of an AI system combining geospatial predictive policing information and automated number plate registration information of vehicles belonging to the suspects serves solely to support human assessment and is therefore not covered by the prohibition laid down in the regulation.
Articoli Correlati