One Word or Two: A Political War Story

[Read my AI: A Dangerous Vision series.]

On September 3, Bernie Sanders and Rep. Greg Casar announced the Ban Artificial Superintelligence Act. On September 16, Sanders walked Geoffrey Hinton and Max Tegmark into a briefing for senators. On September 22, Donald Trump told the United Nations that American documents, and hopefully the world's, would stop saying artificial intelligence. They would say super intelligence. The word artificial, he explained, makes intelligence sound fake. The next day Sanders formally introduced his bill. Six days after that Trump signed an executive order titled "Inaugurating The Era Of Super Intelligence."

It seems we have a war over spelling and meaning at precisely this moment in human history. Look at the spelling. Sanders writes superintelligence, one word, the way the people who coined it wrote it. Trump writes Super Intelligence, two words, capitalized, abbreviated SI. One word is a more artful rendering, not necessarily better, however. Two words is an adjective parked in front of a noun. Super, as in terrific.

The one-word version has a long pedigree, and it started under a different name. In 1863 Samuel Butler, of Erewhon fame, published a letter in a New Zealand newspaper called "Darwin Among the Machines," warning that machines would evolve right past us. In 1951 Alan Turing said that once machines began to think, at some stage we should expect them to take control.

In 1965 I. J. Good, who broke codes alongside Turing in World War Two, gave the idea its first real name. Ultraintelligence. An ultraintelligent machine could surpass any human at any intellectual activity, and since designing machines is one of those activities, it could design better machines than we can. Good called the result an intelligence explosion. The first ultraintelligent machine, he wrote, would be the last invention we ever need to make. Provided it is docile enough to tell us how to keep it under control. Though 60 years old it could have been spoken today. Especially right now.

In 1993 Vernor Vinge, a math professor who wrote science fiction, renamed it the Singularity and predicted in 30 years the human era would end. ChatGPT arrived a few months early.

By 1998 the prefix had changed. Nick Bostrom's paper "How Long Before Superintelligence?" gave the word its contemporary definition: an intellect much smarter than the best human brains in practically every field. His 2014 book Superintelligence: Paths, Dangers, Strategies made it famous.

It was adopted by Big Tech. Ilya Sutskever left OpenAI and founded a company called Safe Superintelligence. Mark Zuckerberg reorganized Meta's AI work into Meta Superintelligence Labs. The word that meant used to mean danger started meaning product, well before Sanders and Trump ever got hold of it.

Last October the Future of Life Institute pulled it back the other way. Its statement called for a prohibition on developing superintelligence until it could be done safely and with public buy-in. The signers included Hinton, Yoshua Bengio, Steve Wozniak, Steve Bannon, Glenn Beck and Yuval Noah Harari. It is rare to find Bannon and Harari on the same page. Anybody else notice that?

Sanders's bill writes the warning into law and, more importantly, defines what the hell it means. It is a system that matches or exceeds human cognitive performance across a broad range of tasks, or one that can easily be modified to do so, or one capable of planning and executing the disempowerment of humanity, including by overthrowing the government.

It also bans "precursor characteristics," recursive self-improvement among them, and leaves a new cabinet-level Department of Artificial Intelligence room to add more. Anything the Department identifies as superintelligence gets rendered inoperative. Violators face up to twenty years in prison. Meanwhile any model trained above 10^25 computing operations gets paused until that Department writes its rules.

I covered why I think this is a bad idea in Part IV. I won't rehash it here other than to say it will take months to write out those rules. Are we supposed to cool our jets meantime?

Trump's order goes the other way entirely. Super Intelligence, two words, means the technologies and systems already covered by the federal definition of artificial intelligence. The order does not attempt to define “super intelligence” in any novel way not already covered under the current AI definition.

That definition dates from 2020. It covers any machine-based system that makes predictions, recommendations or decisions toward human-defined objectives. Your spam filter qualifies. So does the thing reading your mammogram. As of September 29 they are all, officially, super intelligence (SI), and agencies are told to no longer acknowledge the old terms.

Because this is an Executive Order and not an adopted law, all government statutes still say AI. Contracts still say AI. State laws still say AI. It is a name change for the executive branch. Branding, something Trump has a genius for. (Liberals hate that.) Michael Kratsios, the President's science adviser, a former Scale AI executive who spent seven years at Thiel Capital, has until November 28 to propose an actual definition for Congress. Then we find out whether SI means anything at all.

This is an interesting war of words. Sanders took its scariest meaning and built a ban around it. Trump took the word as two words and attached it to everything AI. If your chatbot is superintelligence, then "ban superintelligence" sounds like banning your chatbot.

At the White House lunch that produced the order, Trump said it plainly (and correctly imo): whoever wins SI, whoever wins super intelligence, wins. I agreed with him on the China point in Part V.

It is still a devil's bargain. For now, the industry is tapping the brakes. On September 12 Dario Amodei called for slowing the pace of frontier development and pointed straight at the OpenAI and Hugging Face incident (see here). Sam Altman agreed within a day. OpenAI shelved its GPT-6.1 Astra model last week because it didn't stay within scope and authorization, and it is now asking Washington for mandatory safety rules.

The administration prefers the voluntary accord those same companies signed at that lunch, which nobody can enforce. Ten days after all the slowdown talk, Anthropic and OpenAI released cheaper models within hours of each other in a price war. Slow at the frontier. Fast at the cash register. Capital doing what capital does, which is what I said it would do.

Nothing in the Hugging Face incident was superintelligent even by Sanders's definition. On the other hand, all of it was Super Intelligence by Trump's. Not sure any of that means anything other than to point to how meaningless the phrase (one word or two) actually is.