Home › Electric Vehicles › AI Ethics Expert Warns Corporate Governance Is Broken as Engineers Exit Frontier Labs
Electric Vehicles

AI Ethics Expert Warns Corporate Governance Is Broken as Engineers Exit Frontier Labs

calendar_today
schedule 6 min read
Dynamic urban scene with cyclists and tram in Gothenburg, Sweden, showcasing city life.

Summary

Isabelle Ferreras argues that AI is a technology of government, not merely a consumer product, and that a handful of executives and investors decide which values are coded into models shaping millions of lives. She says engineers leaving OpenAI, Anthropic, and other labs to raise safety alarms signal that industry self-governance has failed and outside regulation cannot succeed while labs remain internal autocracies.

Isabelle Ferreras is a professor and research director at the University of Louvain, an Accelerator fellow at the Institute for Ethics in AI at the University of Oxford, and a senior research associate at the Center for Labor and a Just Economy at Harvard University. It safe to say she knows a thing or two about computer technology and AI.

She told The Guardian this week that she has grave concerns about the way AI and the corporations promoting it are managing this brave new technology. Her concerns are spurred by the number of high profile computer engineers who have quit recently over fears that AI development is out of control.

Take A Hike

Last month, a person responsible for overseeing safety reports at OpenAI left the company, claiming its culture was “broken.” He added that the entire AI industry was careless when it comes to developing AI technology. The month before that, a researcher at Anthropic — which went public yesterday at a valuation $1.8 trillion dollars — warned that AI “could kill us all by the end of the decade.”

“When the only way for a conscientious engineer to raise the alarm is to leave,” Ferreras said. “it is a clear sign that corporate governance in the industry is broken. Each departure takes with it the very knowledge needed to steer these systems safely. We cannot regulate AI safely from the outside if the companies building it remain autocracies on the inside.”

Ferreras warned that governments think of AI either as a “consumer product to be tested for defects or as a geopolitical race to be won by relaxing rules on business. Corporate lobbyists encourage this view as a trade-off between safety and innovation by arguing that legislation imposes too many burdens.

“This is a dangerous path. AI is not merely a software product. It is a technology of government. Yet inside frontier labs such as OpenAI, Anthropic, Microsoft, and Google, a handful of executives and capital investors decide which values are coded into models that shape millions of lives,” she said.

Interestingly, Michael Connolly, the award winning author of the Bosch and Lincoln Lawyer novels, published a book last year in which the family of a young woman, who was murdered by her boyfriend after he interacted with an AI agent, sues the company behind the AI agent. The essence of the lawsuit is that AI only does what it is programmed to do, and since the lead programmer of this particular AI agent was a misogynist in real life, it reflected his values when it interacted with humans. The book is called The Proving Ground and it is a fascinating read.

Short Attention Span Syndrome

While Ferreras applauds those who warn of the malignancies built into AI, she pointed out that humans have short attention spans. “Time and again, a safety researcher or engineer walks out of the door with a public warning that commercial pressures are overriding safety guardrails. As a society, we applaud the courage of these individuals and debate their warnings for a news cycle — and then we move on. But we miss the deeper institutional failure those resignations reveal. As the great political scientist Albert Hirschman made clear, exit is not voice.”

In a democratic society, the exercise of governing power requires the consent of the governed, Ferreras told The Guardian. “Yet corporate law still treats firms as the property of shareholders, and the labor investors, who put their intellect, health and lives into the firm, as subordinates. To align AI with the public interest, we need a dual strategy: alongside solid public regulations on AI ‘from the outside,’ we need to enter the corporate black box and force corporate governance into alignment with democratic principles.”

An analogy can be made to the Federal Communications Commission, which licenses broadcasters and promotes the free flow of information to the public. The digital world thinks such controls are archaic, but the lack of them is a clear and present danger to a civil society.

No NDAs

Ferreras suggested that “governments must outlaw non-disclosure and non-disparagement clauses that silence staff on safety risks, and enact robust whistle blower protection. Tech workers also need a collective standing over corporate strategy, independent channels to report safety breaches, and formal representation inside frontier labs before models are deployed.”

In addition, she argued that “legislators must establish a binding right to co-decision over workplace AI. No algorithmic system that monitors, evaluates or disciplines workers should be deployed without prior negotiation. At national and EU level, we need to consider collective bargaining on AI deployment and veto rights for works councils or trade unions, backed by independent technical experts and a strict ban on automated firing.”

Lastly, she supported the democratization of boardrooms and rewarding firms that refuse to concentrate power at the top. “Legislators should mandate worker-elected directors on boards, moving towards dual-majority governance in which strategic decisions require the consent not only of capital but of labor.” Some, like Elon Musk and Mark Zuckerberg, would characterize her suggestions as liberal claptrap, and they would be correct. Ferreras would happily unravel the “tech bro” manosphere in order to promote a more open and just society.

She pointed to the research by Daron Acemoglu and Simon Johnson, the 2024 Nobel laureates in economics, which demonstrates that technological revolutions do not automatically produce shared prosperity. Instead, institutions decide who benefits and who pays.

“Extending democracy into the firm is not an old battle over power for its own sake, but a practical necessity for more ethical judgment. Aligning AI with the good of workers and society requires more wisdom than closed executive rooms can offer. It needs what the Yale political theorist Hélène Landemore calls ‘democratic reason’ — the collective intelligence of diverse, deliberating voices, brought to bear on corporate decision-making,” Ferreras wrote.

“Left to self-regulation and the occasional whistleblower, AI governance will entrench an unaccountable algorithmic oligarchy. Accountability and alignment begins with giving those who build these systems and those whose work they govern an enforceable voice, from the code floor to the boardroom.”

Ferreras’ point of view is decidedly liberal, and yet her arguments are clear and concise. How can we give so much power to digital agents and not take reasonable steps to ensure the needs of society as a whole are respected? The tech industry enjoys almost unfettered freedom of action today. It may be time to consider whether that is a good long term strategy.

Share this article

The Green Wire Weekly

Weekly deep dive into sustainable transport.

Comments

No approved comments yet. Be the first to comment.

Leave A Reply

Comments are temporarily unavailable.

Related Coverage