'We're losing control,' says AI pioneer Yoshua Bengio

The Canadian computer scientist considered one of the founders of AI, Yoshua Bengio, told AFP that humanity is "losing control" of the technology and the world needs guardrails similar to nuclear arms controls.

Fears about the dangers of AI have intensified in recent weeks, as workers from major companies have resigned with warnings about potentially existential threats to human life if the pace of technological advancement is not checked.

Bengio is a University of Montreal professor who won the 2018 Turing Prize, computing's Nobel, alongside Geoffrey Hinton, widely known as the "godfather of AI."

For Bengio, "there's a reason that companies are saying this is going too fast, that we're losing control."

"People like me have been expecting this for a long time," the 62-year-old told AFP on Tuesday, saying security discussions surrounding AI have not been adequately robust.

There was broad-based alarm after OpenAI disclosed that a swarm of AI agents -- programs that can take actions on their own -- had carried out an unauthorized breach of the open-source platform Hugging Face in July.

Such autonomous action is the central threat, Bengio said, warning of a situation where AI agents "have the ability to get through cybersecurity barriers and enter any company."

Last year, Bengio founded the non-profit LawZero aimed at creating a new form of advanced AI that is "designed to be trustworthy and safe."

Bengio told AFP people need to be aware of the risks ahead.

AI agents may foster "an ability to establish an individual connection and persuade humans to act in ways that suit it but are not necessarily good for all of us," a capability that could have sweeping political consequences, he said.

Longer-term, "AI might not even need human beings anymore to do real things in the physical world," he added.

In that scenario, "one extreme could be the destruction of humanity," he warned.

"That's an extreme," he acknowledged, but noted that even if a banking system were shut down, "that's still very, very serious."