Can you give a brief overview of the EU AI Act and explain what its main objectives are?
The EU AI Act started being discussed back in 2021. So that was before ChatGPT was all over the news. It was even before my mom was aware that there is any such thing as AI that was not science fiction. This EU AI Act forms part of a digital strategy within the European Union that goes back probably to 1995, with the data protection directive. But as computer technology and consumer electronics got more prevalent and ubiquitous in our society, the EU thought, and I’m not judging whether they were right in doing so or not, they had a duty to preserve their democratic values. I’m not saying democratic values generally, like as an absolute, but theirs. That has been guiding their legislative process.
Sometimes there’s also a little bit of economic and technological preservation that is underneath those regulations. The EU AI Act is probably the most popular or the most well-known element of the EU digital strategy, but you also got the EU Data Act, DORA and NIS2 which, directly or indirectly, will also impact AI companies or users. There’s not a single week where we don’t have 200 pages dropping from Brussels to review. The idea is that technology developments right now are mostly centered in the US and in China, there’s a lot of investment being done, and the EU, despite having a lot of big brains, despite having a large single market, does not necessarily have the means to compete and has a late start.
We see the economic value moving away from the EU to other countries, so there is on the one hand, the guiding principle of maintaining the democratic values and on the other hand, ensuring that companies in the EU get a chance to develop themselves.
The EU was very happy with the result of GDPR that was voted in 2016, applicable since 25 May 2018. It kind of shook the technological world by forcing companies to be more mindful about how they processed personal data. Now, with AI, it’s basically the same general philosophy of accountability: before you do something, you need to know what you’re doing rather than doing everything and thinking after the facts.
You know that quote that they attribute to Mark Zuckerberg: “Move fast and break things”? That’s not the EU philosophy. The EU is: “be mindful about what you’re doing, how you’re doing it and why you’re doing it.” Once you understand that philosophy and that approach to regulation, then the AI Act is fairly logical within that frame of reference.
Which forces do you identify in this strategic conflict between the democratic values the EU wants to protect and the technological preservation?
There is a fantastic book by Kashmir Hill from the New York Times, called “Your Face Belongs to Us,”. which is an amazing read on facial recognition. Facial recognition, especially in real time, in public spaces, has a very dystopian feel to that, because this is something that we’ve seen done in China, just like social scoring. It basically became public at the same time as a Black Mirror episode addressing the same issue — and Black Mirror was supposed to be science fiction anticipation! We saw the technology and the tv show meeting at the same point, as if anticipation was already late to the reality. These are typically the kind of things that the EU AI Act had wanted to regulate. There’re four tiers in the EU AI Act.
There’s basically what is banned, there is what is heavily regulated, and then there are the more trivial AI usages with limited or no impact on individuals. The core focus of the EU AI Act is on the first two and the first “banned” tier corresponds to what is not acceptable in our society, but only include a limited set of uses.
As part of the final discussions surrounding the EU AI Act, France and Germany, but especially France, were really adamant that they should minimize the list of banned AI once again to potentially preserve some economical consideration at the national level, which seems to conflict with the big principles of enlightenment and democratic values.
In order to demonstrate that your AI is not banned or is not heavily regulated, companies (either providers or deployers of AI systems) will still need to assess all their AI systems. As part of the accountability framework, they will bear the weight of such demonstration that their AI is falling within a given tier and subject to certain regulatory requirements.
It’s a bit of a reality check. It tells companies (either developing or deploying AI systems) that, regardless of who they acquired such systems from (a European company, a US company or a Chinese company), before they use or create it, they need to know how it works and they cannot merely rely on commercial pretenses. They cannot simply rely on the black box approach, i.e. “I input something in the AI, and I get results, it’s magic”. They must know how the AI system works and document that the way it works is not impairing the freedom and the democratic value of the EU.
The list of banned or regulated AI system can be revised down the line but getting a status quo among the 27 member states is going to be a complex task.
I think the list right now is okay. At a personal level, I think it could have been expanded a little bit to effectively align with the projected EU values. For instance, the ban on emotional recognition is strictly limited to labor and to education. That means commercial companies can still do that. While it is not banned, it is likely going to fall into the “heavily regulated” tier, but it’s still feasible. Nevertheless, I feel the EU Ai Act struck a balance in drawing the line where AI practices were not acceptable. We didn’t want to turn into a dystopian society. This is the outer frontier that the EU will not cross, and the rest is fair game. At a personal level, I think it’s not perfect, as no regulation can be, but I think it strikes a good balance to put some limits to what is and what we can see done.
Can you explain the different risk categories and give some examples?
The first tier, the banned tier, is limited. I think there’s a grand total of eight types of AI that are banned. Then at the lowest tier you’re going to have things that are providing no risk whatsoever that you can freely use, and they’re going to be benign type of AI. It’s going to be like a spam filter, is going to be basically the automation that we’re seeing already in place, and that does not create any risk of affecting individuals. Ten two middle tiers might be a little bit complex, because one of the prisms for assessment of your AI is the risk to individuals. Chatbots, for instance, should not create any assumption for the individual users that they’re talking to a human. You need to make it clear that this is an AI they’re talking to. I’ve seen some bots where it’s clear it’s not a human. But sometimes I’ve also met some people performing lower than a bot could, so you never know. In any case, there is a requirement for transparency, and not only assessing the intent of the AI, but also assessing the actual effects as well as potential effects on individuals.
So even if a company has a noble heart and say that they are creating some AI system to “make the world a better place”, but there is an edge case where it can generate harm to people, it can deprive them some benefits, or it can materialize physical or emotional harm, the company will still need to do their homework and thoroughly review and update the consequences of unleashing their AI system into the world. Because even though machine learning and auto improvement of an AI system is not a prerequisite for an AI System to be defined as such, if there’s some changes over time, because there’s some improvement, then companies will need to update their assessment. It’s a living picture of how your AI system is evolving, and you cannot help but think about Skynet. At some point, even though you can start with an AI that is effectively benign and made to improve the life of the people, it may evolve over time and wants to kill Sarah Connor or humanity. You need to check regularly where you stand on that scale. It’s also a moving picture because the EU can proceed with delegated acts to update the various lists of the various tiers. There is also an interplay between the EU AI Act and existing placing on market regulation. The EU AI Act, unlike GDPR, is not a self-contained regulation. It calls on other pieces of European legislation. For instance, I’ve been dealing with autonomous cars for the past nine years now. That’s been a long time and what I was saying at the time is that autonomous cars regulation was a blueprint for AI, because it is an AI. It’s computerized, it makes decisions autonomously, and it takes some inputs of what it sees around it (e.g. the current situation on the road) and creates some outputs (e.g. whether to brake, to accelerate, or to turn). This is the exact definition of an AI System under the EU AI Act: (i) computer based, (ii) autonomous and (iii) able to create an output from an input.
It was before there was a definition for an AI system. The EU AI Act does create a general framework that companies will need to abide by, but it does also refer to other car regulations. Like for instance, the type-approval to put a vehicle on the market. These more specific, sectoral, requirements will come on top of the general requirements for AI. It’s an interplay, and it all starts with being accountable, doing your homework, knowing what AI you’re using and how you’re using it. For instance, if your marketing department decides on using an AI, and legal and the C suite are not aware, that’s going to create an issue for the company. Companies need a registry of all the AI systems that they are using to abide by the accountability principle.
So, the political regulations always must follow the much faster technological developments?
It’s been like that forever. Look at the e-privacy directive. It was adopted in 2002, before there the whole world of behavioral advertising came into existence. It was before YouTube and Facebook. We were supposed to have an ePrivacy regulation to update that framework since 2018, but it never saw the light of day because of a lot of reasons, but it shows that law is a reactive tool. Maybe in some very limited situations, looking at France, like for instance, on death penalty, right to abortion or same sex marriage, there might have been pieces of legislations passed despite the majority of the people being against it at the time and maybe five years later, people were okay with that. But it’s rarely proactive, it’s mostly reactive, because of the principle of a democratic society is if it’s not prohibited, then you can do it, so there’s no real need to regulate it.
With the EU AI Act, it was a regulation that was adopted in two years, which is extremely fast within the European context. In these two years, we saw technology evolve between the first draft of the EU AI Act and its adoption. GenAI came on the market, and the EU instances had to create additional definitions and specific provisions to address something that was not even perceived at the original draft stage.
The EU AI Act will basically enter into force within a year from its publication to the official bulletin. And we can only image what can happen in that meantime, because we’ve seen what happened between 2021 and 2023, and Moore’s law tends to showcase that improvements can develop at an exponential rate. That’s why a good law has to be general and impersonal. If you tailor provisions that are targeted towards certain companies, this is doomed to fail because they will work around and find another model. You have to create baseline principles, as only those principles can stand the test of time. To that extent, I feel like the EU AI Act is striking a good balance between general principles and very technical definitions.
We’ve discussed how the definition of an AI system is quite broad, yet there are also specific provisions in place, such as the computing power threshold for GPAI measured in flops. These figures are conservatively set for now but are expected to evolve as technology advances. In theory, this approach combines the best of both worlds: it establishes foundational principles while allowing for periodic adjustments to address technological changes. In practice, we’ll have to see how effective this strategy is. This is a new direction for the EU, which typically takes between five to seven years to enact legislation. By the time it was enacted, it was sometimes already outdated. We’ll see how it unfolds. For example, GDPR was passed eight years ago and implemented about six years ago, and only recently did they update some procedural aspects. The EU AI Act, through the delegated acts, may allow for quicker updates. However, whether it will keep pace with rapid technological advancements is questionable. Likely, it will still be playing catch-up, as regulators and legislators are not technicians and depend on available information, often without deep immersion in the tech scene. It would be interesting to revisit this conversation in two years to see where things stand.
Could the EU take a leadership role with giving guidance to further technological developments?
When the EU AI Act was first introduced in 2021, I was surprised. I felt like it might have been a media coup where the EU wanted to address something that I did not necessarily think, at the time, was a priority. I’ve already talked about the ePrivacy regulation. It’s something that should have been adopted six years ago, and is still nowhere to be seen. In my view, this was and remains a priority, and they should have done that before other things. I believe they EU may have decided to prioritize AI because what they saw was that drain of resources and capital being poured into startups elsewhere. And they wanted to do something to level the playing field, to give European companies fighting chance before it got too late.
I’m pretty sure that they saw a possibility to become the light of the world in terms of regulation. It is the first comprehensive AI regulation in the world. Other countries are going to play catch up. And if you speak first, if you’re the first to regulate, especially when you got a regulation that has extraterritorial reach, you’re setting the rules. The other countries and regions are going to be required to play catch up if they want to access the European market. That’s still more than 300 million people. It’s a strong economy. If you want to access this market, you’ll need to play by EU rules. I think there’s been several aspects that came together:
- the financial and economic aspects, how do we create and facilitate an ecosystem that will be beneficial to European companies?
- How do we protect human rights and civil liberties; and
- a little bit of vanity too.
We did that with GDPR and we’re the privacy leader in the world. We did that again with AI and we’ll see once again where it leads. But I think there was a bit of everything all at once. The agenda under which the EU AI Act was adopted is also providing context: Commissionner Thierry Breton was very clear that they wanted to finalize the text before the European elections. And that’s in June. The EU AI Act is to be the legacy of this European parliament. This is going to be the legacy of that European administration, so everybody was hard pressed to agree and get in line and get their ducks in a row. It might have been a vanity drive to get there so quickly, but it does not really matter in the end: the EU AI Act is there.
Imagine yourself in 2035. When you look back at the EU AI Act and its further development, how do you think their legacy and the EU AI regulation will be shaped in this future?
I’m starting with a topic that genuinely concerns me, not just professionally as a lawyer, but personally as well: voice cloning. The advancement is so pronounced that I’m considering setting up a safe word with my mom, to be shared face-to-face and away from any digital devices, as a security measure in case either of us questions the other’s identity during a conversation.
The quality and accessibility of technologies like voice cloning are reaching a point where they pose a real threat to democracy. With rampant misinformation, particularly noticeable during various global conflicts (wars in Ukraine and the Middle East) and political cycles (such as the one in the U.S. four years ago and the upcoming European elections), these tools have the potential to significantly disrupt democratic processes. Looking ahead, the technological landscape is likely to evolve, bringing even more advanced and potentially more disruptive technologies into the mainstream. If we consider a more pessimistic scenario, these advancements could fundamentally alter or damage the fabric of civilization as we know it within the next decade. This represents the worst-case scenario, where the potential exists for technology to unravel societal structures. I’m not overly optimistic, contrary to the view that AI will magically solve all global issues, like curing cancer within a year or eliminating the need for human labor because machines will take over our jobs. It’s conceivable that in ten years, my next interview or even my role as a lawyer could be fulfilled by a chatbot. It’s possible that neither of us will be engaging in this conversation in our current roles due to AI advancements, which isn’t necessarily negative if it means we can derive income from alternative sources without working. On the flip side, the optimistic scenario involves AI freeing us from labor, allowing us to benefit from other revenue streams. However, balancing these extremes, I hope for a middle path influenced by regulations like the EU AI Act, where AI is used responsibly to enhance societal well-being. This path may lead to significant changes in our professions—perhaps we’ll still have jobs, or maybe we’ll have income without traditional jobs, which doesn’t sound too unfavorable.
Within our firm, we are currently utilizing AI tools that can perform tasks typically assigned to interns. This shift raises questions about the future role of interns. They now need to learn how to operate these AI tools while also acquiring the foundational skills necessary for their career advancement that machines cannot replace. If they overly rely on AI without developing these essential skills, they won’t be adequately prepared for more complex legal work in the future. The road ahead offers a range of possibilities, from encouraging to daunting, and it’s crucial to consider how we can harness technology to benefit our clients, ourselves, and future generations.
I sincerely hope that the EU AI Act will hold companies to a higher standard, ensuring they deploy AI thoughtfully and with a clear understanding of the implications. However, similar to what we’ve observed with GDPR over the past six years, this act should also empower individuals to hold these companies accountable. Just as Max Schrems has been a pivotal figure in enforcing GDPR, we may see someone—perhaps even Schrems himself—emerge as a similar force in the realm of AI regulation. By establishing these regulations, the EU is effectively informing its citizens of the rules everyone must follow and empowering them to enforce these standards and hold violators accountable. This regulation not only addresses corporate responsibility but also levels the playing field for ordinary individuals.
How could we ensure that our democratic values and ethical use of AI is realized by the providers and deployers?
Drawing another parallel with GDPR, it was one of the first European regulations to attempt extraterritorial reach. It established that if you are operating within the EU, you are subject to GDPR.
However, challenges arose with companies that processed data outside the EU but handled information pertaining to EU residents. GDPR addressed this by creating specific extraterritorial provisions: companies targeting their activities towards the EU or monitoring the behavior of EU individuals fall under its jurisdiction. Applying these principles to the EU AI Act involves complexities, particularly because technology developed outside the EU might not directly target EU individuals but could be deployed within the EU by local companies.
This distinction made it necessary to clarify the various roles in the regulatory language, much like the shift in vocabulary observed from the 2021 draft to the final draft of the EU AI Act. Initially, the term “deployer” wasn’t used; instead, everyone was broadly categorized as “users”—from individual users like someone interacting with ChatGPT to corporate entities integrating it into their systems. The introduction of a dedicated term for “deployer” in the final draft helps clearly distinguish between different types of users, facilitating a better understanding of the layered compliance responsibilities. If you’re a developer of an AI System elsewhere than in the EU and you’re not touching the EU market, you can do what you want. The EU does not have either the capacity or the willingness to tell other countries how they should develop AI. If you’re in a totalitarian country and you develop the social scoring of your citizen, it’s not good, but the EU does not have the capacity to care for that. However, ss soon as you try to get that technology into the EU market and make it available in the EU, then they needed some safeguards to ensure that AI system was abiding by those rules. Either it’s going to be the publisher themselves that will be subject to the EU AI Act, or it’s going to be the deployers.
The corporate users—companies that integrate AI technology into their products—must adhere to the accountability provisions of the EU AI Act. This means that AI publishers, regardless of their location, must provide deployers with necessary information and safeguards to determine the classification tier of the AI system. If the AI falls into a banned category, EU deployers are prohibited from using it, but they must be equipped with the relevant data to make this assessment independently. This approach is quite strategic and economically motivated, aligning with the EU’s goals of maintaining a free market flow. Essentially, you can operate as you wish in your part of the world, but as soon as your activities intersect with the EU, you must comply with its regulations.
How does the EU enforce these regulations?
We’ve identified a significant vulnerability in the EU AI Act. Commonly in the EU, hefty fines are imposed based on the company’s turnover or a fixed penalty: 7% for violations concerning the use of banned AI systems, 1% for providing misleading or incorrect information to regulators, and 3% for intermediate infringements. The majority of cases will likely fall into that middle category, but enforcement poses a challenge.
For data protection, enforcement was straightforward. Data protection authorities (DPAs) have been established for over 30 years since the directive, making them a natural choice for GDPR enforcement. However, AI regulation does not have a clear enforcement body yet. Member states are tasked with appointing an authority, which could be an existing body with expanded roles or a completely new entity. Many DPAs are campaigning to oversee AI regulation, although AI does not always involve personal data, and there isn’t a complete overlap between personal data and data used for AI.
I am skeptical that DPAs, already stretched thin and under-resourced for GDPR enforcement, would effectively manage additional responsibilities in AI. Alternatively, some countries might assign AI oversight to competition or consumer protection bodies. This leads to potential fragmentation in how the EU AI Act is interpreted and enforced across different sectors—whether it’s from a data protection, competition, or consumer protection standpoint—risking a balkanization of enforcement across the EU.
Despite all 27 member states being subject to the same regulations, enforcement could vary significantly. An AI Office has been established to streamline this process, but it will take time to become effective. This variability could undermine the uniform application of the AI Act across the EU.
Six years after the implementation of the European Data Protection Board (EDPB), we are still awaiting some guidelines, indicating that the enforcement of the EU AI Act is also likely to lag. This delay could lead to uneven enforcement across the EU, potentially creating de facto safe havens for AI companies. For instance, a company might choose to operate in Ireland if the regulator there is a competition authority known for its business-friendly stance. Conversely, companies might avoid France where the regulatory environment is perceived as less accommodating to business interests. While some EU countries may benefit from attracting companies and generating economic value, others might find themselves at a disadvantage due to stricter enforcement. This could lead to an uneven playing field within the EU, impacting the overall effectiveness of the EU AI Act. In my opinion, this is probably the weakest link in the EU AI Act: a lack of streamlined enforcement across the EU. And there’s a lot of wiggling room for member states to come up with an enforcement theory.
Given this vulnerability in the regulatory framework, what outcomes might we anticipate? How are companies likely to respond? Do you foresee the emergence of new institutions or roles to address these challenges?
Starting with new roles, it’s clear that companies will likely need to establish the position of Chief AI Officer. This is becoming more evident in discussions within the International Association of Privacy Professionals (IAPP), where data protection officers are increasingly being tapped to also manage AI-related responsibilities. However, just like the administrative challenges seen in regulatory bodies, these professionals are often under-resourced, which could make this an overwhelming task without adequate support.
Companies will need to centralize and empower compliance efforts in some form, and it seems that this responsibility won’t typically fall to the Chief Compliance Officer, given the specific technical and compliance knowledge required for AI that may be outside their expertise.
As for the regulatory landscape, the prospect of establishing a dedicated AI regulatory body seems unlikely in the current macroeconomic climate, which favors reducing government expenditures. Whether the EU AI office can establish an effective working group or framework to harmonize enforcement rules remains to be seen. Given that the European Data Protection Board (EDPB) has struggled to develop a unified approach to GDPR enforcement across national regulators, there’s a real challenge ahead. While it would be beneficial for such coordination to occur, there’s a cautious outlook on the likelihood of achieving this integration.
The EU plans to introduce regulatory sandboxes to provide an environment that supports guidance and compliance. What are your thoughts on these sandboxes, and how do you see them being utilized in the coming years?
I believe this approach to co-regulation is very characteristic of the Anglo-Saxon model, reminiscent of the UK’s approach to fintech after leaving the European Union. It’s a promising initiative, but a major concern remains the regulatory bandwidth available to implement these sandboxes. According to the EU AI Act, there should be at least one sandbox per country, but there’s no guidance on how many companies or AI systems can actually participate in these programs. This could result in a scenario where a sandbox is merely symbolic, supporting only one company’s compliance journey, rather than fostering a broad and dynamic environment of innovation and regulation.
Unless there’s a genuine commitment at the political level, which would need to come from the heads of state, the potential of these sandboxes may not be fully realized. For sandboxes to be effective, there must be a political will to establish robust environments that attract not only the most promising companies but also those most committed to complying with the AI Act. Additionally, these sandboxes can serve as tools to identify the shortcomings of the EU AI Act—highlighting where it fails to meet its promises or imposes excessive burdens on companies.
Ideally, this feedback could be gathered, and, through the delegated act, adjustments could be made to the EU AI Act or comprehensive reforms could be implemented within six years, similar to what we’ve seen with GDPR. If fully leveraged, sandboxes could be an invaluable tool for monitoring AI development within the EU, potentially fostering the rise of leading tech companies—or “unicorns”—within the region. As for co-regulation, I’m a strong advocate for codes of conduct, particularly in relation to GDPR, and I see them as a vital path forward. Essentially, these codes represent a collaborative effort within the industry, functioning like an enhanced sandbox involving an entire ecosystem rather than just one company. Through this setup, professional associations could streamline management, facilitate enforcement, and maintain open communication with regulators about industry practices and compliance.
This approach can be especially effective due to the resource constraints that regulators face, which I mentioned earlier. It allows for a sort of privatized enforcement where regulatory bodies can delegate oversight responsibilities to associations overseeing the codes of conduct. This not only alleviates some of the burdens on regulators but also provides clearer guidelines and assurances for market participants. Although it took around 18 to 24 months to finalize the EU cloud code of conduct, the outcome has proven to be quite impactful. In essence, adopting codes of conduct for AI could create a win-win situation by providing predictability for market players and reliable assurances for regulators, thus making it an underutilized but highly promising strategy for the future of AI regulation.
I’m looking forward to how these sandboxes and usage and regulations will develop in the real world. What kind of loopholes or open questions do you see which must be answered in the future?
Loophole? That’s a challenging question. After reviewing the EU AI Act, I’ve identified some weaknesses, as we’ve discussed. However, when it comes to an outright loophole, nothing specific has stood out to me yet. I’m examining the chain of liabilities among publishers, deployers, and end-users, and it appears to be consistent.
While the weaknesses are present, the overall structure doesn’t seem to show glaring loopholes. Nevertheless, as we’ve touched on earlier, the enforcement will be crucial because the EU AI Act was designed with strong economic incentives in mind. If enforcement lacks predictability, it won’t deliver the economic growth initially anticipated. Hopefully, a coherent enforcement strategy will be developed swiftly to address these challenges.
Is there anything didn’t talk about that still needs to be outlined?
No, I think the point I really wanted to make was about code of conduct, because I really think it’s a way forward. I talked about the tremendous consequences that a lack of foreseeability on the enforcement would create code of kind of could create that foreseeability, could achieve that virtuous system. So I really think it’s a way forward. Another thing is how technology is going to evolve, and it’s going to evolve very quickly. Two years ago, we had generative AI coming on the scene like a tidal wave. We still have some issues with the visual AI LLMs, like stable diffusion, still cannot draw fingers, still cannot draw letters. When you tell them, design me logo with those words on it, it cannot do it consistently. Part of me is convinced that they know how to do it. They just don’t want to disclose it because that could be way too problematic for society. And now we got the voice clone that are keeping me awake at night and everything in the span of two years. So next up is going to be real time video, like deepfakes. Deepfakes have been addressed by the EU AI Act. I think it’s a very noble inclusion. Not necessarily talked about too much, but we’ve seen Georgia Meloni already acting on it because they made some deepfake porn videos of her, but she’s cannot yet use the EU AI Act, and has to rely on existing regulations instead. It reminds me a lot of that movie with Arnold Schwarzenegger in the eighties, the Running Man , where you can basically create your own reality and people can face the consequences of that. There’s going to be distrust.
Discussing AI often recalls various dystopian sci-fi films, with “Johnny Mnemonic” particularly relevant when considering the reliability of information in an era where anything can be fabricated. There’s a real concern about the societal issues that might emerge through AI’s evolution, and the EU AI Act aims to preserve the status quo and prevent the most dystopian outcomes. It’s an admirable attempt, and it will be interesting to see how other countries respond. Will they take inspiration from the GDPR, resulting in a patchwork of regulations that complicate global AI governance strategies? Or will there be a more streamlined approach? The EU has already made progress by adopting OECD definitions for AI, showing deference to technological expertise in its terminologies. Hopefully, this will lead to some level of standardized enforcement, even if minimal. With so many uncertainties extending beyond mere compliance, the challenges of AI are as much about societal impacts as they are about regulatory responses.