During the United Nations General Assembly, Microsoft (NASDAQ: MSFT) Vice Chairman and President Brad Smith spoke with Alan Murray about whether frontier large-model developers should hit the brakes to ensure the technology they build does not bring catastrophe to humanity.

"The real question is: if you believe you should slow down, then don't keep racing at full speed, and don't wait until external forces make you stop," Smith said.

On the same day he made these remarks, Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman spoke at the UN, calling on countries to establish AI safety guardrails so that AI development does not outpace humanity's ability to control it.

Smith, who has years of industry experience, said: "If you are a company CEO and you judge that your own product has safety risks, should you still bring it to market? I think the answer is obvious — no. Companies have a responsibility for public safety."

Even so, he added: "Some regulatory measures will eventually be inevitable."

Historically, many products only became safe through regulation. For example, milk was once contaminated, "so we set up agricultural inspection departments and introduced relevant laws. Now when you walk into a supermarket, all kinds of milk are safe to drink."

When Smith raised the idea that there should be the ability to "slow down operations or directly shut down AI systems," host Alan confirmed that he was referring to an AI emergency kill switch, and then asked: "Do you think that's a good idea?"

Smith replied: "Of course. ... Model developers should have the authority to shut it down; cloud infrastructure providers like Microsoft should have that authority; and the agencies responsible for governing AI agents should also have shutdown capability. Even if legislation mandates this mechanism, that would not be a bad thing."

Alan asked him how corporate managers should handle AI safety issues within their companies.

"My view is, don't stop using AI. AI is a profoundly fundamental innovation and an important tool for boosting productivity. But always remember that technology needs governance. ... Governing this technology is no less important than the technology itself."

He also offered a reminder: "I don't wake up every day thinking there is a 10% chance of human extinction. But I would rather we worry too much than do too little. After all, what we are discussing concerns human survival. For God's sake, this matters more than the latest technology, and we must treat it seriously."