The Move Worth Finding
Moved earth and heaven, that which we are, we are;
One equal temper of heroic hearts,
Made weak by time and fate, but strong in will
To strive, to seek, to find, and not to yield. Alfred, Lord Tennyson — Ulysses
In 1997, IBM's Deep Blue beat Garry Kasparov. Chess, in one sense, was over. The question of who could make the best move had been answered, and it was not us. By every reasonable expectation, chess should have ended that year.
Instead, more people play chess than ever before.
We now have faster computers and better engines. Stockfish can fit on your phone and defeat any player in history. But we still choose to play chess. Hundreds of millions still play and memorize openings and sit across from each other, burning hours over positions that can be evaluated in mere seconds. Rather than becoming obsolete, chess has morphed into something else – a thing humans do, knowing full well it can be done better by something else, and choosing to do it anyway.
I thought about this when, to my surprise, my friend's face appeared on a YouTube thumbnail. She was opening for the opposition at the Oxford Union. This House Believes that Artificial General Intelligence Will Be Humanity's Last Great Invention.1
The answer seemed almost too simple. Of course! AGI will recursively self-improve until, one day, it becomes superintelligent, rendering human invention obsolete in the process. Or we will go extinct, and the question answers itself. Done.
But grandiose predictions like that should never be taken as definitive. Fission was supposed to give us energy too cheap to meter. Before that, the automobile was supposed to solve the great manure crisis. Fusion has replaced fission, and greenhouse gases have replaced horseshit.2
A better question is not whether AGI will be the last great invention, but whether we will still invent when something can do it better than us. Chess suggests an answer to this dilemma.
Invention is not merely something humans do. It is how we survived and is core to our identity.
Humans underwent an evolutionary gambit some 2 million years ago. We sacrificed almost every proven specialization in order to build tools, and without them, we can't survive. Our bodies are rather unimpressive. We are not especially strong or fast. We can't hear half as well, nor smell a scent a mile away. We are born helpless with no genetic guide to the world. But in return, we were gifted a brain capable of recursively reinventing the environment.
Invention is not just some byproduct of our intelligence but a necessary component of our survival. Our first tool represented more than a useful object. It was our admission that we cannot survive with our bare hands alone. From that admission came civilization. And so, we named our history after our tools – the Stone Age, the Bronze Age, the Industrial Age, and the Information Age. Humans are the animal that cannot survive as given, and so must build the conditions of our own survival.
AGI represents a fundamentally different tool from all previous technologies.
For the first time, we may build a tool that does not simply extend the human ability but threatens to replace the human role behind it altogether. Every tool we have ever built requires a human to remain as the principal agent in its operation. The printing press requires a writer. The telescope requires an astronomer. The fMRI requires a technician to run the scan and a radiologist to interpret it. AGI threatens to become both the operator and interpreter.
A general intelligence machine feels existential in its risks because it has the potential to take over the very foundation of what it means to be human. Extinction risk is indeed a valid one, but that is the boring version of the Oxford proposition. The debate ends because we do too.
The more interesting version of the question is what happens if AGI succeeds in becoming the very best of us. Suppose AGIs are aligned with our interests, and as a result, humanity gets to live alongside the world they build. AGI becomes what its most optimistic advocates imagine – a benevolent intelligence capable of independent scientific discovery, engineering, strategy, design, and creation at a level no human mind can match.
Discovery will no longer require us. Neither will art nor music. There is no serious reason to assume that AGI will remain permanently inferior in the creative domain. Whether or not it understands beauty, comedy, or tragedy the way we do, it will learn to express them with an uncanny fluency. And perhaps, it may understand the human condition with greater accuracy than humans themselves, for the sole reason that it stands outside of humanity.
This is my real fear with AGI: redundancy. What will we do when we become spectators to creation? Since our beginning, humans have found meaning by participating in the making of the world. We started fires, pressed our hands onto cave walls, wrote laws onto parchment, raised cathedrals toward God, painted frescoes and composed symphonies, split the atom, and walked on the moon. We made the tools, and the tools made the age. AGI is the tool that becomes the toolmaker.
When the tools are no longer made by us, what is left for us to become?
Architects of Purpose
The optimistic answer is that we don't disappear from the process. We ascend within it. Humans move beyond primitive toolmaking and focus on the bigger picture. AGI becomes the executor of human intention. It invents the how, while we preserve the what and why.
We tell it what kind of world we want, and it discovers how to build it. It cures diseases, cheapens energy, designs cities, and extends life. It focuses on solving the impossible problems, and we reap the benefits.
This may be the best possible future. A civilization freed from the burden of execution would not necessarily be empty. The Athenians, freed from manual labor through the injustice of slavery, produced philosophy that still shapes the world. Renaissance artists, supported by wealthy patrons, were freed from resource constraints and produced works that endured for centuries. Leisure, when structured by a higher aim, can become the condition for greatness.
However, the Athenians were freed from manual labor, but not from thought. The Renaissance artist was freed from material constraints, but not from creativity. For both, they had to cultivate in order to create. AGI threatens to take what remains. It does not merely handle the chisel and master the technique. It also knows exactly what to carve.
What if it understands our goals better than we do? What if it can map every future, anticipate every point of failure, every pitfall, and recommend the optimal path that maximizes human flourishing?3 What if it becomes better not only at answering questions, but deciding which to ask?
If the AGI is a better architect of our future, why would humans still invent at all?
A Return to Chess
Since Kasparov's defeat, chess has lived under the shadow of machine superiority. We have invented an oracle that can tell you which move to play and show with brutal indifference why the kingside castle is a mistake in this instance.
And yet people still play chess – and they enjoy it too.
The point of chess is not simply to generate the strongest possible move. If this were true, human chess would be an absurd waste of time and energy. The point is to enter the infinite complexity of the game with a finite mind and see how far one can go.
Winning by Stockfish feels nothing like winning with your own mind. A brilliant sacrifice suggested by an engine is not the same as a sacrifice you found yourself. And that is why we study openings and endgames, risk blunders and mistakes, return after humiliating losses: so we can one day, by our very own intuitions, discover the brilliant move ourselves.
The struggle is not the cost of meaning. It is the source.
Each game in chess is a new invention. The pieces are the same, the starting position is the same, and the rules are the same. But each path is different. Almost every game generates a unique sequence of choices that has likely never existed in the history of the universe. The invention is not necessarily the result. It is the path carved through possibility. This is the first clue for how to think about invention after AGI. Our proposition assumes that invention matters because of the thing produced: the theorem proved, the machine built, and the move found. If this is true, then a superior intelligence really does end the human role. Why search for the answer when the answer can be given? Why make the move when the algorithm already knows?
But chess suggests that this perspective is too narrow. A move can be known and still be worth finding. The arrival of an omniscient machine intelligence does not render obsolete the player's experience of reaching into the statistical unknown and, through sustained effort, discovering the exact same move.
The same can be said of invention in the post-AGI world. AGI cannot make our attempt to reach those things meaningless. It can solve the position, but it cannot play the game for us.
Mathematics already understands this distinction well. A theorem can be settled, but it cannot be exhausted. There are over 350 known proofs of the Pythagorean theorem. Each one takes a different path to the same truth. We would still admire the 400th proof, not because it changes the theorem, but because another mind took a different path to reach it.
Even the greatest discoveries have more than one route. General relativity was independently conceived by both Einstein and Hilbert nearly simultaneously, but by different means. Einstein approached it through physics, and Hilbert through mathematics. The equations are the same, but the path is distinct. This is where the proposition begins to fail. AGI may discover every law of physics that governs our universe and find cures for every conceivable disease, but it cannot exhaust invention by arriving first. There are many paths to the same truth. AGI may reach one in an instant, while it takes humanity over a century. The path we find will be greater. Not because the truth was new to the universe, but because a human mind had found its way to reach it.
A Lesson In AlphaZero
There is a second lesson from chess that took longer to arrive.
The advent of superhuman computer engines confirmed much of what humans already believed about chess. Material mattered. King safety mattered. Piece activity mattered. The strangest move Stockfish made could be understood. The engines calculated further than any person could, but played a more so exacting version of the same game. This familiarity made sense. Traditional engines were built and designed by us. We embedded our own understanding into them. Their evaluation function became based on the game as we knew it – the value of pieces, pawn structure, and certain positional advantages, etc.
AlphaZero was not built the way Stockfish was. It was given only the rules of chess and nothing else. It refined its own evaluation function from self-play without human intervention. Within 4 hours of self-play, AlphaZero surpassed Stockfish's ELO. Within 9 hours, it had decisively defeated Stockfish in a 100-game match: 28 wins, 72 draws, 0 losses.4
AlphaZero did not just play better chess. It played chess entirely differently. It surrendered material that violated principles engines spent decades reinforcing and played moves that seemed obviously wrong. It built a relentless pressure that even the strongest engine could not escape.
What AlphaZero revealed beyond its ability to mesmerize in chess was that the ceiling was not where we thought it was. By defeating Stockfish, AlphaZero proved that there was a much deeper layer to chess than anyone expected. Chess did not shrink as a result of AlphaZero's victory. It expanded beyond what even the best engines foresaw.
A similar fear is that AGI will rapidly approach the frontier and that once superintelligence arrives, there will be nothing left to discover. The AlphaZero lesson suggests the opposite might be true. The arrival of stronger intelligence in chess revealed how much more of that domain remained to be explored. Stronger intelligence will not close the frontier but expand it.
The same may be true beyond the 64 squares of chess. The Athenians and Renaissance artists were not merely idle in their leisure. Freed from labor and material constraint, they did not settle on the earthly comforts but turned toward the meta-physical. They reached toward the truth, toward beauty, and toward God and the transcendent.
Something similar may happen to us. Freed from the burden of solving the immediate, we may finally have the room to confront the distant questions. What is the nature of consciousness? Why do we exist? Are we alone in the universe?
After thousands of years of thinking, perhaps the human search for truth might not be nearing its end but only just beginning. AGI may push the frontier out so much that these unanswerable questions finally begin to look answerable. It might show that we have spent our entire existence mistaking the hills for the mountains.
Sisyphus at the Top
For cheating death, Sisyphus is forever condemned to push a boulder up a hill, for it to only roll back down each time he nears the summit. His relentless struggle is meaningless by every external measure, and nothing is achieved as a result of his effort.
Camus asks us to imagine Sisyphus happy.
This has always seemed difficult, but it is quite simple. Meaning is not derived from the success of outcomes. It was never in the boulder reaching the top, but in his dedication to continue. Zeus has unknowingly granted Sisyphus the ultimate gift of meaning in a world without. The rock is for the gods, but the push is his.
The post-AGI human finds themself in a similar predicament, but our struggle is not exactly comparable. The underlying question for Camus was how a person finds meaning when the work accomplishes nothing. Our question is, how do we find meaning when all work has seemingly already been done?
When every problem can be conceivably solved with time and computation, what remains for us to do in the meantime? What happens when Sisyphus finally reaches the top and finds that the boulder stays?
Much of human history has been necessity disguised as a virtue. We worked because we had to eat and built shelters so as not to freeze. Our resources were constrained; our intelligence limited. We learned to derive meaning from this useful output because it was necessary for our survival.
But in the post-scarcity world, our resources are abundant, and intelligence is scalable. Output will and perhaps should be taken for granted. If AI someday builds a better world without us, then our task will not be to compete with it for production. Our task will be to recover and find the part of the invention that was never reducible to output in the first place.
The path not yet taken. The choice not optimized. The push. Not because we must, but because we can.
Sisyphus gives the rock a long stare.
He has seen this moment before. The brief moment of stillness before the gods send the boulder tumbling down.
But nothing happens. It just sits there.
He thought the Gods were bored. Perhaps they have discovered a new punishment through false hope. Were they that cruel?
But the boulder did not move.
He waits. He shouts at Zeus from the peak. No answer comes.
No god answers his call. Only he can now.
He walks down the mountain, then looks back up. The stone is still there. Sitting there. Fixed against the sky.
Silence.
“What now?” he thinks.
For the first time, nothing is demanded of him.
Then, slowly, he smiles.
He starts his climb back up. His steps feel light. The burden of the rock is not his.
At the summit, he looks out into the world, down toward the place where he began.
Then he rolls the stone down.
It gathers momentum along the old path, passing every familiar turn before thudding into the same hollow it has worn into the earth through eternity.
Sisyphus follows it down.
At the base, he places his hands against the stone once more. He knows the feeling. He has done this a million, a billion, a trillion times. He lost count a long time ago.
He looks up toward the peak. He gives it a push.
This time, the push is different.