Technology and (Dis)Trust: AI between confidence and controversy

Held on October 2 and 3 under the theme "Technology and (Dis)Trust: AI Between Confidence and Controversy," Lawtomation Days 2025 provided a forum for reflection on the role of law in digital governance and the legitimacy of AI-driven decision-making. Leading experts, academics, and students explored the delicate balance between social trust and technological skepticism. 

The two-day event, organized by the Jean Monnet Centre of Excellence for Law and Automation, was hosted in Madrid’s IE Tower. The opening panel, chaired by Antonio Aloisi, Professor at IE Law School and Co-Director of Lawtomation, and included keynote speeches from Michèle Finck, professor of law and artificial intelligence at the University of Tübingen, and Veena B. Dubal, professor of law at the University of California, Irvine School of Law. Finck’s opening presentation, titled "The AI Act as deregulation: a constitutional perspective on the EU regulatory approach towards AI," offered a thought-provoking look at the AI Act as a potentially deregulatory tool. 

Finck posited that far from a robust safeguard, the Act may function as a form of deregulation, subtly shifting power from public institutions to private powers. As she noted, "The general portrayal is that the AI act is thought of as an international outlier in its stringent approach to regulating AI—my main point is that it’s actually deregulatory as it is a pre-emption of member state control of AI."  

She underlined how the AI Act’s scope extends only to providing harmonized rules for the placing on the market, putting into service, and the use of AI systems. Legislators’ primary intention was to prevent member states from adopting their own rules in relation to AI in a very broad manner. Finck argues that this point has been underappreciated in discussions around the Act and explains that the "open-ended nature," and "undefined legal terms", means there is significant uncertainty about what national enforcement will look like. Ultimately, she concludes, it will be "political factors that determine how things will play out in years to come."

Professor Veena B. Dubal’s keynote, "Data Rights at Work: A Comparative Perspective," explored the growing impact of algorithmic management and surveillance technologies on labor rights. Her talk placed the spotlight on the power imbalance between employers and employees in digital workplaces and reflected on how data collection and algorithmic decision-making can undermine autonomy and dignity. 

Using the example of gig-economy workers in San Francisco, she observed how the algorithms determining employees’ constantly changing pay are themselves constantly changing. With algorithms now deliberately designed to modify behaviors through fluctuating wages or targets, traditional data-protection laws—which rely on individuals understanding how they are being assessed—falls short: "Collectively understanding the logic of decision-making systems then, will not help them advance in their jobs, as the systems may be designed to learn about and treat individuals differently."

The closing panel, chaired by Francesca Palmiotto, Professor at IE Law School, included Mathias Siems, Professor of the European University Institute. He explored the challenges and opportunities posed by AI in legal research. Siems proposed a risk-assessment framework for AI use in academia, likening it to EU food labeling systems with a color-coded scale for different practices. Rather than banning AI, universities should promote literacy, individual responsibility and disciplinary diversity, helping scholars assess when AI use is appropriate. 

Fellow panelist Ignacio Cofone of the University of Oxford picked up on many issues similar to those highlighted through the conference. He discussed the relational nature of modern data: "As soon as you download an app and use it for a minute, companies get information on you because they profile you according to the behavior patterns of people who came before." In practice, "companies can now infer your sexuality from your buying habits."

Therefore, "in a world of unpredictable inferences, the model of giving people the power to choose breaks down," as there is simply no way for individuals to know how the individual elements aggregate. So, as Cofone argues, regulators will have to reframe their approach to accountability: "Data protection should not just regulate individual choices—it is about holding the powerful accountable for the consequences of what they do." He concluded simply that in practice, any credible claim that could affect people’s lives should be subject to legislation. 

As AI systems become more deeply embedded in our societies, one thing is clear: the legal field has a duty to adapt—and fast. The insights shared at this year’s Lawtomation Days will have a big role to play in shaping a legal response that prioritizes human dignity and meaningful trust.