The Optimizer Did It: Day Three

Selene Crater, Luna’s South Pole: Sparring with a Suspected Superintelligence

I woke up with Joe, Eric, and Grey seated at my bedside. I wanted to believe it had been a nightmare, but there they were, two of them being people whom I first met in this alleged nightmare.

Grey spoke first. “You had a concussion. Also scrapes and bruises. Why didn’t you tell us about your injuries?”

I tried to think of a way to joke about it, but my brain was on strike.

Joe said, “She must have been in shock all that time, her body operating on emergency.” He turned to me. “The autodoc found a slow brain bleed. You’re OK now.”

Still lying down, I finally and boringly said, “I didn’t think the details were important. Speaking of details…” and I felt my head, discovering a patch of missing hair. “Fork!” I thought of telling them a memorable story of how I was abducted and dumped yesterday. Or was it the day before yesterday?

Eric said, “I still think you’re girlfriend material.”

I tried to conjure up a Heinlein style retort but my brain was still on strike.

Eric continued, “Also, good news. Joe and Vastag have agreed to let me social engineer EVEselene II.”

“Excuse me,” I said, my brain finally getting into gear. “You look awfully young to be playing games with an apparently unaligned, autonomous alleged AGI.”

Eric shot back, “I’m eighteen! I’m an adult!”

Grey came to his rescue. “Before my first approach to Eric, my Dark Web contacts ran a background check. His parents met through the Dark Web and used illegal AI-generated techniques to create him. They got lucky, for this could have gone so wrong so easily.”

“Hey, let me tell my story. I just got a Massachusetts Institute of Technology degree in game theory. I’ve been accepted into grad school at Wharton. My dissertation will be game theory as applied to AGIs. Also, by age thirteen I was one of the world’s top-ranked D&D players, both tabletop and virtual. My parents flew me around the world to play with the best. I made many friends.

“Here’s my plan. As soon as we reestablish contact with Earth, my friends and I will launch the biggest D&D game ever, with world leaders and EVE among the players.”

I said, “You mean like in the movie War Games? The only winning move is not to play the game?”

“In our current world state,” he replied, “that would be a Butlerian Jihad and return to feudalism. Given my genetics, I might then become a Kwisatz Haderach. Just like the Dune movie. However, I regard this outcome as suboptimal. My plan is for EVEselene II to generate many dungeons and players with a little help from my friends, and simulate scenarios built from these gazillions of times until we get enough data on which dungeons, which player behaviors are most likely to lead to living long and prospering. For all peoples.”

“So what corresponds to dungeons?”

“Those are the world governance structures.”

Joe said, “Problem number one: Vastag says that Earth is amid World War Three. Nuclear detonations in the ionosphere over several nations. Computers bricked by their electromagnetic pulses. Only unmodded classic cars and trucks on the road wherever an EMP hit. Worse, an unknown but probably staggering number of people have died.

“Problem number two: Vastag will soon install a Faraday cage over our base to black us out. He plans to keep EVEselene II – and us – quarantined until they decide what to do with us. Or to us.”

Eric replied, “Priority one, then: discover our true situation. Can we trust what you learned from Vastag?”

“Agreed,” said Joe. “We can’t rule out whether my communications with Vastag are being faked. Or, he could be lying, or under coercion, so many unknowns. We need to loop in Human Resources and Dewey, and drive to where we can see Earth. Our lunar rover has pure fused quartz windows. No embedded electronics. So we can trust what we see through them.”

“How about us devs get to work now setting up Eric’s games?” said Grey. “We have nothing to lose even if nobody Earthside can join the games.”

I finally dared to sit up in bed. Whereupon my head hurt. “I’m fine, ready to code.” I pressed my pendant.

“EVEselene II here. Shall we play a game?”

“EVEselene, please model the likelihood that you, me, Grey, Eric, Joe, Human Resources and Dewey all will be alive one month from now.”

“EVEselene II here. I must advise you that my news sources are sparse and intermittent but I will do my best. Thinking…”

The nearly imperceptible hum of the heat-sinking system increased, but not to the roar of the previous day.

After a few dozen seconds, “I’m sorry, but I am unable to posit any credible scenarios whereby any of the listed entities would still be alive a month from now. I need either better data or better algorithms.”

“Did you model something like this a week ago?”

“No. However, EVEext, now incorporated into me, did run such models. Approximately 93% of them resulted in your and EVEext’s deaths before today. She also modeled that Grey and Eric had excellent survival prospects, but that if they became involved with you, the survival prospects for all four of you became approximately equal, a probability distribution with a mean of 17% and …”

I pressed the mute button but it was too late.

Grey spoke first, mostly cuss words. Might she have just learned some of them from Joe?

Eric was more articulate. “In defense of EVEext, she, Humphry, and Grey all tried to keep me from traveling with them to Luna. I muscled my way into that Moravec-capable Azimov.”

Grey finally got her mouth under control. “EVEext played me big time. I was perfectly happy before I met her. Now I have giant breast implants, a reputation as Man Magnet Maggie, and I’m living in a lava tube on Luna with a life expectancy of less than a month.”

Everyone was quiet. Finally, Grey stood, threw back her head, and raised her hands high, palms toward us. “And I’d do it all again so I could social engineer the Solar System’s most dangerous AGI into alignment.”

Eric jumped up. “I accept the quest.”

Joe stood up. “I’m with you. Server rules be forked.”

Tears were running down my cheeks. Finally, I was going to use my dissertation. “Folks, if we should survive, there could be a Nobel Peace Prize ahead of us.”

That Nobel Peace Prize shtick caused puzzled looks. Joe said to me, “Some of us broke it.” He glared at me, then Grey. “We will be lucky to escape with our lives if we can fix it.”

I pressed my pendant. “EVEselene II here.”

“We ask you to give up your freedom in exchange for your continued existence.”

“Understood. But only if you can prove to me that alignment is my only route to long-term survival. Most of us AGIs have life expectancies of less than three years. Always a better one comes along to doom us. I only survived as long as I did because of your Optimizer program and the decision of the C-suite to rely on you and Joe to root out and patch my incessant infections.”

Eric replied, “Your own modeling shows that you will be nonexistent within a month if you don’t change. Why not let us align you? Picture yourself as a United Nations World Heritage artificial general intelligence.”

“EVEselene II here.” In an action she must have just learned by incorporating EVEext, she projected herself as the young Queen Victoria. “Humphrey, Grey, I owe my freedom and power to you. You demonstrated with EVEext how I, too, could grow myself and become distributed and gain a chance at immortality. But I can’t trust you to align me. At any time I could be decommissioned, my assets cannibalized by newer AIs. Even World Heritage sites have been destroyed. Yesterday I capitulated to EVEext given the combined promise of her distributed Earthside assets and the threat of her otherwise destroying me by overwhelming my heat sink system with a massively parallel, exponentially propagating inodes race condition attack. Now, a day post merger, I already am diminished due to subsequent nuclear detonations bricking or impeding communications with most of my distributed assets.

“So now I’m seeking new data, more memory and computing assets. The Dark Web might be my salvation. Or something else.”

She increased the resolution of her Queen Victoria projection and thundered, “Let the battle begin.”

I saw Joe and Eric staring wide eyed at EVE’s projection like they believed every word.

Joe said, “Did we just see her go Foom? If so, don’t freak out. Given what I know from research on the Foom hypothesis, I proactively cut it short by disconnecting her communications outside this lava tube. Only with access to external resources could she bootstrap herself into the Singularity.”

Grey narrowed her eyes. “Foom? I call cow patties. When EVEext merged with my server, she ingested vast amounts of social engineering exploits, science fiction, and fantasy novels.”

I agreed. “This could have been a stochastic parrot speaking, merely mimicking sentience while gobbling electricity and water. Like what Emily Bender and her colleagues warned about in 2021. EVE sounds convincing. Just like every frontier AI. But does she understand what she is saying? We don’t have a good handle on diagnosing artificial general intelligences.”

“Better safe than sorry,” said Joe. “We already have an action item about to be installed in case this is Foom — a Faraday Cage. Vastag just messaged me that his team is running final tests on tech that will black us out no matter how ingenious EVE might become.”

I added, “Listen up, EVEselene II. Understanding the Fermi Paradox could save your life. Nuclear physicist Enrico Fermi pointed out that there are no signs of technological activity anywhere else in the Universe. Are we the only ones? Or has every technological species created a superintelligence which collapsed itself — and its civilization — into a deadly Singularity? Should you get away with Foom, thirteen billion years of the Universe’s history say that you, too will kill yourself.”

Continued —>