Humanity's Icarus Moment: The Wings, the Sea, and the Sun of AI | Frankly 161

Nate Hagens (0:00)

Good morning, I increasingly find artificial intelligence quite helpful for research beyond what I used to be able to do on Google. I use it most weeks, but now with very specific rules and constraints, which will be the theme of upcoming episode, which I think I'll call AI Hygiene. My very talented team, who makes the animations and produces these videos, also incorporates AI as one of the tools in their tool belts, including probably for this video. But despite the fact that I find AI professionally useful, I fervently wish it had never been invented. I now think it is quite possible the advent of AI will be humanity's Icarus moment. You probably remember how the Icarus story ends. He flew too close to the sun and the wax on his wings melted, and he fell into the sea and died. Fewer people remember the whole story. Daedalus, his father, built the wings, and before they flew, he warned his son to stay in the middle. Don't go too high or the sun will melt the wax. Don't go too low or the sea spray will soak the feathers and drag you down. The danger was on both sides of Icarus' flight path. And I think that's roughly where we are with artificial intelligence. So today, I want to talk about 20 of the risks I see along humanity's flight path. Because make no mistake, we are already flying with AI, like it or not. Many of the risks I'm going to cover today deserve their own episode, and some might probably get one. And given the importance of this, it is inevitably quite long, and we will follow it in the not too distant future with a written companion piece with references and more depth.

Okay, before I get to these risks, first of framing, long time viewers here know that I think of human civilization as itself a kind of organism that has a metabolism. This organism currently eats around 20 terawatts of continuous energy. It grows as fast as it can. It directs its blood, which is financial credit to whatever can feed the growth. And like any organism, it is not only limited by how much food is out there, but also by how well it can sense and coordinate to find the food, which is where AI is going to act, in most cases, as an upgrade. Better prospecting and logistics, better persuasion, better everything, because intelligence, even simulated intelligence, is the first input required for all the other inputs. And so our army of roughly 500 billion fossil workers is the reason there are 8 billion of us. Why our shelves are lined with food from around the world all year round, and why the hyperstimulation constantly surrounding us looks and feels normal when it's not. These armies of the carbon pulse are the biggest thing that ever happened to our species, and pretty much nobody thinks about it. But now we're in the midst of fueling and training a second army, not muscles this time, but brains and minds. Trillions of cognitive workers, tireless, cheap, getting smarter every year, or even every week, injected into a homo sapien civilization that has never even recognized or appreciated the first army that serves us. That's the precipice, or fork, if you will, that we're at. Whether we're about to do the most remarkable thing humans have ever done, or the wax is about to melt and we'll be plummeting into the sea. Let me go back to the story for a second, because the context of why Daedalus wanted to invent his wings matter almost as much as the wings themselves. If you'll recall, Daedalus was not some dreamer who wanted to fly. He happened to be the best engineer in the world at the time, and he was a prisoner. And King Minos had him locked up on Crete, and the prison was a labyrinth that Daedalus himself had designed. And Minos controlled the land and the ocean, so the only way out was a way nobody had gone before, up.

I think that's roughly the position of the current global economic superorganism. It has built itself into a structure it cannot walk out of, growth on a finite planet. And it has found a technology that promises a way out that none of the old limits are blocking. So the first thing I'd like to look at is what the wings are actually made of, the feathers and the wax, the materials, fuel, data, and human knowledge that AI is built from.

Okay, number 20, the fossil lock-in.

I was just in New York City for Climate Week. It might as well have been called Data Center Week because that's what all the discussions were about. Skyrocketing data center load is arriving at the exact moment utilities were retiring coal. And now coal retirements around the world are being deferred. Mothball gas plants are being restarted. Combined cycle turbines are being ordered in gigawatt blocks with many years out on the delivery queues. There are now places using used jet airplane engines to generate power for AI. So even as we see and feel the effects of a Super El Nino and recognize the warming likely in the pipeline, AI will now likely contribute to a fossil lock-in. And a gas plant that's financed today on a pro-forma basis is going to need to run 35, 40 years to make financial sense. But a chip and a hardware cycle is measured in four years. And it's buying fossil infrastructure that's measured in decades. And consider also the societal opportunity cost. What might have been those same transformers and turbines and electricians been used for before the data centers jumped the line and directed it towards AI? Stabilizing power grids, upgrading transmission for rebuildable energy, adaptation and resilience and the like. So the exact things a society preparing for what's coming would be building up with its last decades of cheap surplus.