Oppenheimer and AI

I’ve studied Oppenheimer, the Manhattan Project, and scientific ethics for most of my career. The biopic comes out at a crucial time, when the potential of technology to harm society is higher than it’s been in decades. What can we learn from the film?

There was an old joke in the aerospace program at my alma mater, kind of like that game where you add the words “in bed” to the end of any fortune cookie to get what it’s really about, but instead you added “for making bombs” to the end of any engineering course to understand what it was really about. Strength of Materials… for Making Bombs. Thermal & Fluids Engineering I… for Making Bombs. It wasn’t that far off. In my Propulsion Systems Engineering class we learned how to compute the velocity of the pressure wave of a nuclear blast. In Foundations of Applied Mathematics, we learned how to use dimensional analysis and the Buckingham Pi Theorem to compute the yield of a nuclear bomb from a photograph. (I used this method many years later to accurately estimate the yield of the Beirut docks explosion within hours of it taking place.) We were trained to build weapons.

September 11 happened during my sophomore year. I distinctly remembering sitting in Aerospace Fundamentals—10:00 AM, West Hall, Tuesdays and Fridays—when it happened, and I remember when our professor, a former NTSB investigator, walked us through the calculations of the force with which the airplanes hit the Towers. His class gave no partial credit. Airplanes crash, or are crashed, and people die, and you don’t get partial credit from the families of the dead. It wasn’t a balanced education; one of the selling points of Rensselaer at the time was that you can graduate while taking only four humanities courses. But it wasn’t without balance. The school had a great program on the social sciences of technology. Langdon Winner, author of one of the great technology ethics papers of all time, “Do Artifacts Have Politics,” is on the faculty. We had to learn about teamwork and psychology and leadership and ethics. We were trained to be engineers.

One of those classes I took, I can neither remember the course title nor be bothered to look it up, was about the social impact of technology. This was a great class taught by a great professor, and while I was probably an obnoxious little prick in it, it was formative. We read on the “technological sublime” of American culture, we studied Robert Moses, and of course, we spent a great deal of time with Jungk and The Day After Trinity and learning about the Bomb, which of course included Oppenheimer and his cast. Brighter than a Thousand Suns sits on my shelf, as does Surely You’re Joking, Mr. Feynmann and The Man Who Loved Only Numbers and a dozen other books that dive into the history of these men (it was, of course, always men) and what they did in the world and what they did to the world. They lived one shelf down, until recently, from a vast collection of books on the history of aviation, stories about the B-17 and the P-38 and Lockheed’s Skunk Works and the space program and the weapons we built and the deeds they did. I did my undergrad fellowship at Princeton, a fact I am immensely proud of, even if I did mess it all up. Every few years I find an excuse to spend a night there, take my wife to PJ’s Pancakes, and walk the campus trying to recreate the paths that Einstein and Oppenheimer and Nash and Gödel must have taken. This was my education, this was my training.

Oppenheimer jumps directly into scenes with characters intended to be recognized only by their last names, which may alienate the audience who hasn’t spent the last 25 years of their life diligently studying the material. But this is the easier material to engage with, the material that is palatable for the American taste, with its stories of individual heroics and people rising up in patriot duty during a time of need. I had wondered if Nolan would go into the uglier aspects of Oppenheimer’s story, the persecution and character assassination of Oppenheimer as a suspected communist. It does, perhaps for 45 minutes too long. There are no spoilers here: the material has been in the public record for decades. The film is self-aware. Christopher Nolan is hardly subtle in his intention when somewhere near the third hour of the film a character asks, “will there be anyone to tell the truth of what went on here?”

The downfall of J. Robert Oppenheimer is one of those unpleasant elements of American history that we sweep under the rug. We’re more than happy to lift him up as a hero, and as a Jew, who helped America win a technological race against the Nazis. We’re more than happy to celebrate him bringing America into the fold of quantum mechanics when the best research was all happening in Europe. The parts where we denied him due process, where we persecuted him for his vaguely left-wing political beliefs, where we found him (metaphorically) guilty by association, those are the un-American parts of the story that we’d rather ignore. This, more than the bomb, is the true subject of the film.

Oppenheimer, the film, struggles with presenting the moral conundrums surrounding the development of the bomb, though they’re not absent. There is an unresolved and unresolvable question about how long it would have taken the Nazis to develop the bomb if they weren’t being soundly defeated. The movie engages with the fact that many of the scientists involved in the Manhattan project were Jews and certainly includes a scene or two where characters briefly question the ethical nature of their work. There’s no secret that even during the height of World War II, plenty of Americans with power felt we were fighting the wrong enemy, and similarly, Nolan touches on this material but fails to draw it to its conclusion. We could have done with 20 minutes less of quick cuts between hearing scenes so claustrophobic that you could smell the actors' sweat and used the time instead to explore whether America was the second fascist country pursuing the bomb. This is a missed opportunity.

Oppenheimer, the man: sans italics, is remembered inter alia for his comment after setting off the first bomb at Trinity, where he quotes the Bhagavad Gita. “I am become Death, Destroyer of Worlds.” A phrase, starkened partly due to its Germanic and altmodisch use of the “is”-form of the auxiliary helper, serves the dual purpose of being incredibly prescient and also convincing every insufferable nerd who memorized it that they know fuck all about ethics. The famous utterance became the sacrificial anode on the hull of our ship of scientific progress. Oppenheimer felt bad so we don’t have to. Oppenheimer battled the demons for us. I’ve heard many times the quote exit the mouths of people who build weapons.

It’s easy to fixate on this statement because it remains the most poignant part of the story. The ability of scientists, nay, technologists to create tools of immense, world-altering power brings a certain sobriety in discussions of progress, or it should. I had the fortune of studying engineering in a formal context surrounded by experts with first-hand experience, not only in technology and how it can go wrong, but also in war. In high school, my senior project studied World War II aircraft. My advisor flew B-24 Liberators during the war. He understood what terror is. I went to Rensselaer in part because I “only” had to take four humanities courses. To today’s technologists, studying any humanities at all is little more than a time tax on advancement. Today we build AI and release it to the world with nary a second thought. I find it difficult to believe that contemporary techno-utopians have ever wondered if they are become Death, Destroyer of Worlds. I can’t even induce ChatGPT to use the old grammar anymore.

I’m not yet sure if AI will rise to the level of the bomb. I simply think we have to be asking the question. Do not get me wrong. I put no credence in the sci-fi vision of super-intelligent AIs bringing humanity to extinction. Rather, it’s the wholesale death of culture, the mass displacement of jobs, the pollution of the internet, and the impending techno-hegemony I fear. I worry for a future of children who never learn to think critically because ChatGPT can do it for them. I fear for an opaque system beyond anyone’s ability to understand being purely in the control of private hands that open up eagerly for money. I worry that AI foretells the conclusion of the history of the world: equality is irrelevant, fairness is irrelevant, culture is irrelevant. In terms of the scale of devastation, of the gross consolidation of power into the hands of the wealthy few, I worry that AI will rise far past the level of the bomb. The Jewish scientists on the Manhattan project surely knew what they were building and for what reason. There are plenty of similar challenges today: climate change, genocide, pandemics, that today’s AI researchers could be organizing their efforts together with subject matter experts to solve. Instead, they’re motivated by greed. OpenAI raised more money since the beginning of 2023 than the entire US nuclear fusion budget has had for the past ten years. By more than a factor of two.

I saw the film at the Kino International, Berlin’s best and most beautiful building; sitting, as it does, on Karl-Marx-Allee, on one of the grand, evincing boulevards built for the DDR’s communist utopia. It’s a fittingly ironic venue for the film. Nobody breathed in the theater when Oppenheimer gave his rousing speech about how he wished the bomb had been done in time to drop on Germany. We were certainly sitting nearly at ground zero for where that would have been. Germany is a strongly anti-nuke country. Recently, they shut down the last of their nuclear power plants, part of a passionate (if not misguided) Green Party agenda from years ago to de-nuclearize the country. The audience almost surely saw no tension in the ethical question of whether America should have forged ahead with the bomb.

Germany also sits in the European Union, which has acted assiduously to regulate AI, data privacy, and digital platforms. The Union views unchecked abuse of AI as an existential threat, they do not need to wait for a leading scientist to have a crisis of conscience. We are, of course, not without such figures. Plenty of people have blown the whistle on the behaviors coming out of Redmond and Menlo Park and Mountain View. The problem is that they’re on the outside now, and people like Sam Altman, the CEO of OpenAI, have publicly and badly missed the point of the film. Long ago, I decided that talking about tech ethics to technologists was like trying to put out a raging fire with a dixie cup.

The truth, however, is that as guilty as Oppenheimer might have later felt about the bomb, particularly after its use on Hiroshima and Nagasaki, there was then as there is now very little actual resistance to building world-altering technologies, even when the world ends up much worse as a result. Nolan’s latest might be a cautionary tale, but it’s actually very difficult to get people to stop doing things that are beneficial to themselves. The one field where formal ethics has reliably acted as a bulwark against inhumane behavior is in medical ethics, and we only got to that point after a series of horrifying crimes committed in the name of medicine. Two of the worst: the Nazi experimentation by Karl Brandt, and the Tuskeegee Syphilis Study, performed by the United States Public Health Service. Oppenheimer is a terrible image of restraint in the face of progress; he once considered irradiating the German food supply, but only if it could kill half a million people.

On the other hand, the people who are speaking out about the real dangers of AI today are finding themselves characters in a familiar story. People like Timnit Gebru and Frances Haugen, among many others, have blown the whistle on the harms of the technology only to find themselves exited from the companies they criticized. Today, they’re harassed and criticized for pushing “woke agendas”. It’s not too hard to look around and see another Red Scare brewing, or something a little worse. Perhaps this is actually the warning of the film: as soon as you stop being useful, be prepared to be disposed of; or, remember that your bosses never really thought fascism was the true enemy.

Posted: 22.07.2023

Built: 03.10.2023

Updated: 07.08.2023

Hash: ef3ffe4

Words: 2140

Estimated Reading Time: 11 minutes