How The Enlightenment Ends: Can We Trust Ourselves To Use AI Without Crowning It?

[Celerity]

https://sineadbovell.substack.com/p/how-the-enlightenment-ends



Last week, OpenAI announced that one of its general-purpose reasoning models had disproved a conjecture posed by the Hungarian mathematician Paul Erdős in 1946, a problem that had stumped mathematicians for eighty years. It did so by drawing on a branch of mathematics with no obvious connection to the original problem, which is itself remarkable. External mathematicians reviewed the proof independently and confirmed it correct. OpenAI has not yet released the full 100-page document, and the degree of human involvement in guiding the approach hasn’t been fully accounted for. So more details will hopefully follow.

But if the result is what it has been reported to be, this is a significant moment. It represents exactly what AI tools can be used for in advancing mathematics and science and pushing the frontier of knowledge forward. While there are many competing positions on AI, “slow it down,” “speed it up,” “it’s over/under-hyped,” the technology’s application in science, maths, and medicine is where most camps tend to agree it will be most valuable. Everyone is cheering on AlphaFold! I, too, am most excited about its application in these domains and eagerly await the next AI-assisted Nobel Prize winning moment. But I also wonder whether we can be trusted to use AI to advance our understanding of the world without crowning it something that stands beyond it.

How the Enlightenment Ends

In June 2018, Henry Kissinger (a figure whose legacy remains deeply and rightfully contested) penned an essay in The Atlantic that was widely read in technology and philosophy circles, discussed with interest, and then largely set aside. The title was “How the Enlightenment Ends.” The Enlightenment — the 17th and 18th-century European movement that placed human reason at the center of knowledge and redesigned society around the pursuit of truth and freedom over inherited authority — would be undone, he argued, not by war or sudden collapse, but by a gradual cognitive shift: society abandoning human reason for algorithmic convenience, replacing introspection and empirical analysis with digital outputs, effectively putting human cognition in the passenger seat. “The Enlightenment started with essentially philosophical insights spread by a new technology,” he wrote. “Our period is moving in the opposite direction. It has generated a potentially dominating technology in search of a guiding philosophy.”

In their final collaborative work, Genesis: Artificial Intelligence, Hope, and the Human Spirit (2024), Kissinger, former Google CEO Eric Schmidt, and former Microsoft Chief Research and Strategy Officer Craig Mundie extended the argument into more specific territory. For roughly five centuries, the scientific method has organized human knowledge through transparency: form a hypothesis, design an experiment, publish the methodology, allow others to replicate and verify. The method’s power is not only in the results it produces but in the fact that anyone can, in principle, follow the reasoning. One of the great challenges with our current AI systems is their opacity, sometimes referred to as the black box. We can know what data we have inputted into the system, and we can observe the output, but we cannot fully account for how the system arrived at it. Factually speaking, we do not know why our AI systems work as well as they do. Extend this into the domain of science, and for the first time in five centuries, we could produce real, confirmed results using AI whose reasoning is structurally inaccessible. Not necessarily wrong, just opaque. The authors argue we risk being catalyzed backward into a premodern acceptance of “unexplained authority.” But this time, instead of accepting what could not be questioned because it came from God, humanity might accept what cannot be questioned because it came from a model.

snip

[cachukis]

2. Seriously frightening world on the horizon.