Applicable to: AI providers and deployers.
Date of application: 2 February 2025.
Objective: prohibit the use of AI systems intended for the evaluation or classification of natural persons or groups of persons on the basis of which a social score is assigned that results in them a detrimental or unfavourable treatment.
Regulation (EU) No 1689/2024 on Artificial Intelligence (AI Act) of 13 June 2024 introduced a series of prohibitions within the European Union on practices involving the use of AI systems that pose an ‘unacceptable’ risk (the highest level assigned to certain AI systems in the risk-based approach adopted by the AI Act).
In this article, we will analyse the practice covered by Article 5.1 c) of the AI Act, under which the following is prohibited:
the placing on the market, the putting into service or the use of AI systems for the evaluation or classification of natural persons or groups of persons over a certain period of time based on their social behaviour or known, inferred or predicted personal or personality characteristics, with the social score leading to either or both of the following:
|
(i) |
detrimental or unfavourable treatment of certain natural persons or groups of persons in social contexts that are unrelated to the contexts in which the data was originally generated or collected; |
|
(ii) |
detrimental or unfavourable treatment of certain natural persons or groups of persons that is unjustified or disproportionate to their social behaviour or its gravity. |
The elements that characterise the prohibited practice are:
1) the fact that the AI system is placed on the market, put into service or used in the European Union
2) that the AI system is intended for the evaluation or classification of natural persons or groups of persons over a certain period of time based on their social behaviour or personality characteristics
3) that the social score leads to either or both of the following:
|
- detrimental or unfavourable treatment of certain natural persons or groups of persons in social contexts that are unrelated to the contexts in which the data was originally generated or collected; |
- detrimental or unfavourable treatment of certain natural persons or groups of persons that is unjustified or disproportionate to their social behaviour or its gravity.
The Guidelines approved by the European Commission on 29 July 2025 clarified that profiling of natural persons under the GDPR (EU Regulation 2016/679) may fall within the scope of the prohibition in question (see, in this regard, the judgment of the CJEU of 7 December 2023, C-634/21, SCHUFA I, concerning a creditworthiness scoring system).
They also provided some clarification regarding the two conditions that must be met, either one or the other, for the prohibition in question to apply.
An example where prejudice arises in social contexts unrelated to those in which the data were originally collected is the use by a national tax authority of an AI system to analyse taxpayers’ social media posts for the purpose of selecting tax returns for inspections.
However, a situation of unjustified or disproportionate prejudicial or unfavourable treatment may arise when an AI system is used to profile households in order to identify children at risk based on criteria such as the parental mental health and unemployment, if, on the basis of the score, children are taken from their families even in cases of minor transgressions committed by the parents (for example, occasionally missed medical appointments).
