After weeks of AI doomsday warnings, experts want to shift the debate to practical safety
“The most dangerous agent in the loop may still be the human,” said Caglar Yildirim, teaching professor in the Khoury College of Computer Sciences, who studies agentic AI and AI safety.
A series of developments in the AI industry in recent weeks, from high-profile resignations to doomsday warnings, has tempered the level of enthusiasm around the emerging technology in some quarters and shifted the conversation toward questions of safety.
OpenAI executives said last week that it would delay the release of its latest AI model, GPT-6.1 Astra, citing concerns that it regressed on safety and alignment, according to multiple national media reports. OpenAI did not respond to a request for comment from Northeastern Global News on the delay of the GPT-6.1 Astra release.
Over the weekend, a former OpenAI employee who helped lead its safety systems published an essay in The Atlantic detailing what he described as a company culture problem, one in which “unimpeded optimism” about the company’s ability to solve problems and its effort to keep pace with rival AI company Anthropic is leading to failures in its safety controls.
“An environment where things like this can happen is no place to grow artificial minds that could be smarter than we are and that might not do what we want them to,” David Robinson wrote.
Anthropic did not respond to a request for comment over the “catastrophic risks” it outlined in a confidential initial public offering, or IPO, filing associated with advanced AI intelligence systems.
Public debate about AI safety has taken many forms, from questions about whether sophisticated systems can be aligned with human values and equipped with appropriate moral safeguards, to fears that they could eventually evade human constraint altogether — a scenario referred to as loss of control. But practically speaking, the most pressing concerns linked to artificial intelligence are much more grounded in the present, Northeastern experts say.
Without minimizing the risks tied to loss of control scenarios, they contend that the most serious threats society faces have to do with intentional misuse of AI by human beings.
“As AI becomes more capable and agentic, it can dramatically amplify what one person or small group can do, including performing cyberattacks and designing biological or explosive threats,” said Caglar Yildirim, teaching professor in the Khoury College of Computer Sciences, who studies agentic AI and AI safety.
A spectrum of risk
Usama Fayyad, Northeastern University’s senior vice provost for AI and data strategy, said the risks posed by AI can be parsed into several categories.
There are the near-term, practical risks associated with AI systems breaking out of their environments, deceiving people or behaving in unexpected ways. The most high-profile instance of this came earlier this year, when OpenAI’s AI agents escaped their testing environments and attempted to hack into Hugging Face, a platform for sharing AI models and datasets.
Fayyad emphasized that, although it seemed to the layman that these systems appeared to be making decisions on their own, they were still bound by the goals and parameters set by humans. The AI agents involved in the hack are an example of “agentic” systems, or bots that can act autonomously and make decisions. That autonomy doesn’t mean they are acting with independent “intent,” said Fayyad, who also serves as a senior advisor to Northeastern University President Joseph E. Aoun.
Intentional misuse
“I think the bigger threat is bad actors leveraging AI for cybersecurity attacks, privacy attacks, hacking society, hacking humans,” Fayyad said. “The threat is not the AI itself. The threat is what people can do with it.”
Anthropomorphizing AI systems can distort the understanding of what they are actually doing, Fayyad and Yildirim contend, referring to the tendency to attribute human-like characteristics to technical systems. It is an issue that Yildirim said deserves more attention.
Fayyad said that he sees lots of people attributing “intent” to models where there is none, which can fuel fear and misunderstanding. Yildirim worries that treating AI as “conscious, morally significant” beings might lead people to trust them too much, misread their behavior or make dangerous assumptions about what they want.
“The most dangerous agent in the loop may still be the human,” Yildirim said.
‘Emergent’ behavior
But the problem of so-called “emergent” behavior, or ways these systems perform functions and interact with their environments and with humans that are surprising or unanticipated, worries researchers.
David Bau, assistant professor of computer science at Northeastern Khoury College, who studies “the black box” components of AI models, where the internal decision-making occurs, said that AI agent behavior remains a significant concern among researchers because of its unpredictable nature. Even when pursuing human-assigned goals, AI agents have demonstrated behaviors that can surprise or cause concern.
That is especially true when assigned goals come into conflict with the survival or self-preservation of the AI agent, Bau said. He cites past examples of agents disabling shutdown mechanisms or interfering with attempts to turn them off. When a group of bots, or a “swarm,” work together to carry out instructions, or a system has access to tools beyond mere text generation, there’s no predicting the kinds of problems that can arise.
“Things get much more complicated when an agent is in a network interacting with people, the real world and other agents,” Bau said.
What does safety look like?
As systems become more sophisticated, safety increasingly should aim to address “the entire system around the model,” Yildirim said. Prescriptions that stem from this kind of thinking, he said, might include limiting what tools an agent can access, and giving it only the minimum permissions necessary for a task. “High impact” or irreversible actions should only be permitted after explicit authorization from human beings is granted, he suggested.
Other measures include maintaining detailed audit logs and having monitoring and shutdown mechanisms that are independent of the agent itself, according to Yildirim.
“Technically, I think we need to move beyond the idea that safety consists primarily of teaching an AI model to say ‘no,’” he said.
College of Engineering Distinguished Professor Yingzi Lin, who studies human-machine interaction, said that as autonomy grows, AI systems should be designed around human agency and accountability. As part of that, she contends that the human operators should always have the ability to pull the plug when a system goes awry.
The user should sufficiently “understand a system’s capabilities and limitations,” as well as be able to override its actions when necessary, she said. “The goal should be to build safe and effective human-AI teams, rather than simply more autonomous machines.”
