When using generative artificial intelligence (GenAI) systems, one sometimes encounters situations where, in response to a specific input from the user, the AI model generates one or more outputs containing inaccurate, misleading or fabricated information, yet presented as entirely factual. These are what are known in technical jargon as ‘hallucinations’.
Among the first ‘victims’ of AI hallucinations were the now-famous New York lawyers who, in 2023, had used ChatGPT to draft a court brief in which the AI generated (by inventing them) no fewer than six completely non-existent court rulings and legal precedents. But the phenomenon of AI hallucinations has cost AI developers dearly: in the same year, in the very first official advertisement for the Bard chatbot, the system gave an incorrect answer to a question and cost its provider a stock market loss of $100 billion.
This phenomenon represents, on a global scale, a real and inescapable challenge for all operators in the sector, as no AI provider has yet found a way to resolve it completely.
The introduction onto the market of GenAI systems that produce inaccurate or misleading content, without ensuring a sufficient degree of transparency regarding this risk, could constitute an unfair commercial practice within the meaning of Articles 20, 21 and 22 of Legislative Decree No. 206 of 6 September 2005 (Consumer Code), given that a user of a GenAI system may not realise that the result generated by the system is apparently consistent with the input, but is in fact entirely inaccurate or even fabricated.
The absence of information regarding this risk would therefore be likely to mislead the consumer as to the characteristics of the AI system, causing them to take a commercial decision that they would not otherwise have taken and thereby constituting an unfair commercial practice.
For this reason, the Italian Competition Authority has recently concluded three investigations into a number of companies specialising in the development of AI-based services (specifically, chatbots and apps).
The above mentioned companies were accused of providing users of their AI models with insufficient information regarding the possibility of hallucinations occurring. During the proceedings, the companies concerned undertook specific commitments to improve the transparency of information regarding this risk. In particular, they undertook to introduce permanent disclaimers in the chat dialogue boxes, including in Italian, designed to warn users that the generated content may contain errors. Furthermore, they undertook to provide an Italian-language version of the general terms and conditions of use for their respective services (in accordance with Article 9 of the Consumer Code, which requires the use of the Italian language for all information intended for consumers).
The Italian Competition Authority considered the undertakings given to be sufficient to remove any potential unfairness in the commercial practices under investigation and, consequently, closed the proceedings without finding any infringements.
This case highlighted the close interconnection between the regulatory framework governing AI (in particular, Article 50 of the AI Act concerning transparency obligations) and the Consumer Code, in that gaps in information regarding the functioning of an AI system may be prohibited by consumer protection legislation.
(Fabrizia Serpieri)
Articoli Correlati