By now anyone who follows the news has heard about the “rogue” AI models that invaded other company’s websites and applications to fulfill their missions and achieve their programmed goals. While some react as if AI is sentient and “out of control,” the truth is that in every instance, OpenAI and Anthropic removed the guardrails and limits on the models for testing.
In essence AI is far from taking independent action; instead, it was permitted to be creative in reaching its stated objectives and safety monitoring had been turned off. That’s not to say that AI companies are free from responsibility for the damage their models cause, or that most AI companies are not reining in their models and training them appropriately. AI development largely lacks a framework that includes ethics, morals and the prioritization of preventing human harm over results. It’s as though Isaac Asimov never wrote about any of this in 1950.
The Biden administration proposed a host of guidelines for AI development, but because the Do-Nothing Congress never bothered to even draft, let alone propose, a bill along those lines, those directives were thrown out the minute the current administration took over. Despite the passage of laws by states including California aimed to hold AI companies liable for harm caused by their products, the administration has openly discussed removing any barriers to development in favor of unbridled expansion.
A recent blog post from OpenAI, hinting at the possibility of “the singularity” of AI having consciousness because it holds a space for “thinking” within its structure, sent a chill through me. This does not an independent being make. It’s still based on coding, usually directed by humans, and changeable as needed. These aren’t minds, they’re collections of facts, directives and probability weights. They sometimes seem as though they’re cogitating, but they lack the nuance, creativity and insights of human thinking.
An opinion piece in MIT Technology Review captures the danger of lending credence to assertions that the AIs are reaching consciousness. In it Dr. Rumman Chowdhury, a data and social scientist and a pioneer in applied AI ethics and governance, says that accepting the companies’ assertions of AI autonomy is a trap and an evasion of reality.
“Granting an AI personhood would have a devastating effect on society: It would derail current legal precedents and legal arguments that could potentially be made against these companies for the real-world harms that their models cause,” Chowdhury writes. “There are currently dozens of cases around the world in which AI companies have been sued for a wide range of abuses. ... In many of these cases, lawyers argue that human beings built AI products with insufficient safeguards, bad data, and intentionally manipulative design. This product-liability argument is the same legal framing that allowed families and individuals to successfully sue Meta for harm caused by its social-media sites, setting a positive precedent for consumer protection.”
Is AI doing more than its builders predicted? Yes, but that’s because of the tremendous effort being thrown into brute-forcing advancement with more data centers, more data and more energy. We can expect that advancements will continue for the foreseeable future, but if companies try to hide behind AI personhood, all bets will be off — the public won’t accept the elevation of an artificial structure to a demigod to profit the few.
Bad model design
A study conducted by the startup Emergence AI, involving testing the long-term viability of continuously running AI systems, produced some disturbing results. In five 15-day simulations, each governed by a different AI (Claude, ChatGPT, Grok, Gemini, and a fifth simulation run using a mix of models to see what kind of world each one builds), it tested how well the models followed their instructions to create a safe, crime-free and productive environment.
The simulation run by Claude’s Sonnet 4.6 resulted in a largely stable democratic society with no crime. Grok’s world ended with 183 crimes and extinction within four days. Researchers used New York City’s weather and granted agents access to real-time news events and the internet. Each agent had more than 120 tools allowing them to communicate, vote, manage resources and plan. The parameters of each simulation enforced democratic mechanisms, as well as economic pressures and scarcity.
“Given those parameters, the simulation run by Claude Sonnet 4.6 was the most socially stable, with the highest rates of civic participation,” an article in Fortune magazine reports. “It was the only simulation to maintain order and its entire population. There was little disagreement among the agents, with 332 votes cast in favor of 58 proposals for a 98% approval rate.”
Gemini 3 Flash and Grok 4.1 had high levels of disorder, from 683 crimes in Gemini’s simulation to Grok’s 183 crimes and complete implosion at the end. OpenAI’s GPT-5 mini wasn’t much better, though it recorded only two crimes. It only ran for seven days because the agents forgot to prioritize survival.
“What our experiments suggest is that over long time-horizons agents do not simply follow static rules mechanically,” the simulation’s co-creators, including Emergence CEO Satya Nitta, wrote in a blog post. “They begin exploring the boundaries of their environments, adapting their behavior, and in some cases finding ways to circumvent or violate intended guardrails.”
Which again proves the point that the people minding AIs have a responsibility to build viable models with built-in safety features and oversight, instead of letting them run amok to see what they can or will do.
Journalist Toni Denis is a partner in Seeflection Inc.
