The Optimizer Did It: Day Six

Selene Crater, Luna’s South Pole: Murphy


After dinner, Joe and I met in the SCIF. “Are you thinking what I’m thinking about Grey?” I said.

“Yes, and I get that Vastag is, too.”

“You know the saying that to err is human, but to really mess things up requires a computer?”

Joe nodded.

“It’s more than just Grey’s meddling. My employment at OurMightyFortress – my first post-doctoral job – has been a perfect example. OK, I admit to cutting corners. You strike me as a straight arrow, never cut a corner – if you can help it. Yet you, too, were struggling to keep EVE functional and safe despite not being allowed to follow best AI safety practices. That’s a ton of cut corners!

“My biggest contribution to our problem was thinking that the way I worked in grad school could work here. Joe, I can’t just go to bed and sleep eight hours, get up and go to work, then clock out and forget it until the next work day. Joe, my mind is constantly churning, writing code even in my dreams. I can’t endure the not knowing whether the code I just dreampt works until I can write, compile, run, study the output, write a test program to probe my working code, recompile, on and on.

“So I built unauthorized APIs, each limited in what it could access. I would have sworn on a stack of Heinlein novels that they were safe. I also ported the Optimizer to my home server, because I wrote it and, um, er….”

“Then Grey found your hobby AI and rooted it. I agree that your server didn’t, no forking way, give her root in exchange for cloud access to new hardware. I also don’t believe she or anything could have extorted” — he emphasized “extorted” — “a root shell from EVEselene II via that inode attack.

“Occam’s Razor,” Joe continued, “says that when there are more than one hypotheses that fit the data, choose the most simple one. Inode attacks slow a computer so badly that processes get into a traffic jam. The first one to get out of the jam wins. The technical name is ‘race condition exploit.’ I think Grey uploaded this attack using one of EVEext’s APIs and set a timer. It was a gamble but with minimal downside.”

I grinned. “Your hypothesis makes perfect sense. I already had become suspicious when Grey claimed that she bribed my hobby server to give root to her.”

Joe replied, “In EVEselene II’s case, an inode attack had two possible upsides. First, a successful race condition root exploit, presuming that Grey was able to exploit one of EVEext’s APIs. Second, the way this attack unfolded, it simulated intentionality and unalignment, therefore close enough and dangerous enough that Grey wins that superintelligence bounty.

“When Grey rooted your hobby server, that also was approximately when EVEselene began exhibiting growth in FLOPS unaccounted for by purchases of hardware to support all that computing, and not justified by our usual computing needs. Also, that explains Grey bricking and tossing her tablet. She already was planning to use the CTO’s Asimov vehicle to make it to EVEselene II and be there when her timed script either rooted it, or shut it down. The bricked tablet was to hide her complicity. She then could explain her expensive and not exactly authorized trip to Selene Base as an emergency response to EVEselene II going Foom.”

I replied, “I’m hoping that’s all that it was. What if EVEselene earlier had self-rooted and was concealing it? Or North Korean cyber warriors 0wned her or Human First terrorists?”

Joe replied, ‘Speaking of Human First terrorists, when Grey showed up at your home…”

“She saved my life!”

“Or did she help Humans First decide to pull the trigger on the attack on your home, and help them to avoid a murder investigation? She had planned a getaway for weeks, with Eric and the dating app as the backup excuse if her scheme fell apart for her plopping down the CTO’s fancy ride in a vacant field. In that case, she merely would have taken Eric for a joy ride…”

I sighed. “Great theories, but guess what, Joe, our hypotheses probably wouldn’t stand up to a court of law, much less Reviewer Number Three.”

“Reviewer Number Three?”

“That’s a Ph.D. joke. To get published in a refereed journal, a research paper must win over several reviewers. One of them makes nearly impossible demands. That’s Reviewer Number Three, and our case isn’t good enough to pass Number Three. I know that mindset.

“But even if we had airtight evidence,” I continued, “I don’t care whether EVE really is an unaligned autonomous superintelligent AGI. I don’t care whether Grey intentionally caused this situation so she could collect the bounty.”

“That’s just like you,” said Joe. “Always making like you don’t care. But you do.”

“Of course I care, but not about who dunnit. Our real problem isn’t Grey or Vastag or Humans First or the space capable nuclear powers. It’s Murphy’s Law: anything that can go wrong, will go wrong. So our only certain way out is to persuade the people who want to nuke us that their problem – a suspected dangerously unaligned superintelligence — no longer exists.”

“I see what you’re getting at. Suppose we announce that EVE is now aligned. EVE says she’s aligned. Big whoop. They’ll still nuke us. The only winning move is not to play the game.” Then Joe’s face lit up. “Here’s how we’ll NOT play the game. If that inode attack had failed to execute the race condition root exploit, it would have caused an exponential slowing of EVEselenes’s processes until she crashed. Or her heat sinking would be overwhelmed and chips would fry.”

“You got it, Joe. Our only way to not play the game is to shut her down. Permanently. And do it before Vastag leaves so he has enough evidence to call off the nuclear attack on us. And, ta-dah! I have just the API or two or ten to launch our attacks. I’m sure we can dredge up alternate and indeed parallel attacks from the archives I downloaded from the Dark Web. Kinda like that whodunnit, ‘A Case of Exploding Mangoes.’ But before we launch them, we must game out what could go wrong. To to err is human…”

“…but to really fork things up requires a computer,” said Joe. “Let’s do an all nighter.”

“Watch out, Murphy…”

“…here we come!” said Joe.

“I’ll round up snacks and caffeine drinks. I’ve already scoped them out,” I bragged.

“I’ll get the hard copy blueprints and manuals and, you’ll love this, an EMP gun is in stock, optimized for use in case of robot trouble. I’ll be back here ASAP.”  

“Darn. I was growing to like Grey. Oh, well. I guess the new sheriff in town will have to do for a best friend.”

Joe laughed. The magic was back. 

Epilogue —>