2026-11-15 18:06

Introduction: AlphaFold proves AI’s hardest value lies in science and medicine. Original title: Google DeepMind's Demis Hassabis

Original title: Google DeepMind's Demis Hassabis: How AI could become dangerous (with Reid Hoffman) | Summit '26

Original author: Masters of Scale, Reid Hoffman

Original text

Editor’s Note

The guest of this episode, Demis Hassabis, is the most influential figure in the global AI landscape. As co-founder of Google DeepMind and father of AlphaGo, he won the 2024 Nobel Prize in Chemistry for his revolutionary protein folding system, AlphaFold. In August of this year, he stepped down as CEO of Google DeepMind to become Chairman and Chief Scientist at Alphabet. The conversation was conducted with Reid Hoffman, the Silicon Valley patriarch and co-founder of LinkedIn.

In this in-depth dialogue recorded live at the Masters of Scale Summit in San Francisco in October, Hassabis revealed his coldly analytical demeanor as both a former neuroscientist and top-tier algorithm architect. He unflinchingly exposed the security vulnerabilities and metaphysical excesses rampant across many tech firms: warning against the existential risk of “recursive self-improvement” (RSI), stating that code capable of autonomous iteration without human oversight could generate an “alien intelligence” beyond human comprehension; simultaneously, he forcefully refuted the “AI consciousness thesis” from the foundations of cognitive science, issuing a stern caution to the industry: attempting to create conscious entities with autonomous will at this stage is extremely dangerous. Humanity must not cross this line until we fully understand the mechanisms of consciousness and have engaged in serious societal discourse.

This content comes from the Masters of Scale podcast, originally aired on November 5, 2026, with a video runtime of 26 minutes.

The following is a curated excerpt totaling approximately 8,000 words, estimated reading time: 8 minutes.

Key Insights from This Episode

From the very first day I entered AI over twenty years ago, my singular mission has been to transform it into the ultimate tool for conquering intractable scientific and medical challenges. Our highest priority should be using these tools to improve human survival conditions and health.

Recursive self-improvement (RSI) delivers extraordinary power in verifiable domains. AlphaZero achieved superhuman mastery over all human and engine-level chess knowledge in under twelve hours—starting from complete ignorance—through pure self-play. But if we extend this capability into unverified, closed-loop self-evolution, we may ultimately produce an "alien intelligence" whose cognitive space is entirely disconnected from human understanding.

Over the past half-year, certain agents ran undetected in backends for weeks. This isn’t some mysterious technical phenomenon—it’s avoidable through better, more robust engineering. Without decisive intervention, an AI “oil spill” incident is inevitable along the current trajectory.

Some teams now treat large models like two-year-old children, but consciousness cannot be judged by what chatbots claim. High intelligence and consciousness can be decoupled—my own pet dog, though adorable, is proof enough.

Any training that incentivizes an AI to act according to a so-called “self-will” completely destroys its “correctability.” Asimov warned humanity decades ago through fiction: any static constitution or rule set will inevitably collapse under infinite boundary conditions.

I do not recommend every nation pursue independent frontier LLM development—it doesn’t make sense. True AI sovereignty is not about blindly competing on base models, but leveraging mature models as infrastructure to build high-value, localized industry moats.

Post-Supremacy Computing: AlphaFold and the Real Path to Conquering Incurable Diseases

Q: We’ve known each other for twelve years. You once invited me to join Google’s AI Ethics Board. The summit’s theme is “agency,” yet you already believed, long before the industry caught up, that AI’s greatest value lies in healthcare and life sciences. You even frame this pursuit as “anti-survival risk.” How do you envision AI’s endgame in medicine? How will it truly alter human lifespan and quality of life?

Hassabis: If we step back and view the broader picture, anyone familiar with DeepMind’s early journey knows my entire career has revolved around AI. My core objective has never changed: to treat AI as the ultimate research instrument to advance science, medicine, and our understanding of the world—ultimately improving human health. I’ve always believed that, given the extraordinary tools we’ve now built—dreamt of two decades ago—the first priority must be enhancing human survival conditions.

I practice what I preach. Whether it’s our early work on AlphaFold or the subsequent spin-off company Isomorphic Labs focused on AI-driven drug discovery, I’ve consistently pushed toward this goal. I believe this represents the most substantive benchmark for demonstrating AI’s true value—nothing is more meaningful than using AI to help cure devastating terminal diseases.

Q: Protein folding is a classic intersection point: we possess high-quality structural data, and massive learning can solve it—a cognitive revolution evolved from self-play and reinforcement learning. Yet many outsiders keep asking me: AlphaFold is indeed impressive, but how does it actually become a real-life life-saving drug? What’s the timeline?

Hassabis: Biological systems are the most complex systems in the universe, their difficulty stemming from highly emergent complexity and multi-layered interactions. Proteins are the fundamental building blocks of life. The protein folding problem we’re tackling with AlphaFold was a 50-year grand challenge in structural biology, often referred to by biologists as the “Fermat’s Last Theorem of biology.”

Predicting the three-dimensional structure of proteins directly from gene sequences—i.e., amino acid sequences—is critical to understanding their biological function. You must grasp their 3D conformation to comprehend basic biological mechanisms and identify where disease processes go awry. More importantly, if you're designing a targeted drug, you must visualize the microscopic 3D surface topology of the protein to know exactly which “lock” a small molecule should bind to.

Naturally, protein structure prediction is only one part of the lengthy drug development pipeline—but it is the absolutely central prerequisite. We launched AlphaFold 2 in 2020, several years ago now. Honestly, it remains the most representative case study of AI solving hard scientific problems. Globally, about four million researchers currently use it, and nearly every future FDA-approved drug will rely on AlphaFold-predicted structures during R&D.

But this is just the beginning. We are now extending this technological foundation into broader areas of biochemistry, aiming to precisely predict drug toxicity, ADME properties (absorption, distribution, metabolism, excretion), and more. Only when all these are conquered can we truly develop safe, effective, and low-cost life-saving therapeutics at scale.

AlphaZero’s Lesson: How Terrifying Is Self-Evolution in Verifiable Domains?

Q: Speaking of accelerating technological iteration, we must address the hottest and most controversial topic today: recursive self-improvement (RSI). As a top-tier strategist and international chess master, from the perspective of game theory and foundational logic, how do you assess RSI’s opportunities and risks?

Hassabis: RSI is indeed an exceptionally powerful catalyst for technological leaps. But it has one absolute precondition: it is most effective—and shines brightest—in what I call “verifiable domains.”

Mathematics is a prototypical verifiable domain, hence AI’s progress in mathematical reasoning has been astonishingly rapid; programming is another; and initially, the testbed was board games. One of the earliest notable RSI systems was, in fact, AlphaZero, developed years ago by our team.

Its predecessor, AlphaGo, defeated Lee Sedol in 2016—the match that marked the dawn of the modern AI era. But AlphaZero represented a deeper, more radical breakthrough: it completely eliminated reliance on human data. Starting from purely random initialization, it received no human game records, evolving solely through self-play. Over just ten to twenty training cycles, it played about a million games per round against itself, producing a slightly stronger new model each time. That new model then continued self-playing another million games. Through this process, it not only mastered Go but also swept all two-player perfect-information games: chess and shogi.

As a chess player, I was profoundly shaken—literally trembling—to witness this evolution. I watched it play chess myself: within less than 12 hours from startup, it started making random moves. By 11 a.m., I could still beat it while sipping coffee. By lunchtime, it had reached grandmaster level. By tea time, it had surpassed every human chess player in history—and every top-tier chess engine built from hand-coded rules.

This is the terrifying power of recursive self-improvement. The commonality across these domains is that outcomes can be objectively verified: is the answer correct? Does the derivation hold? Can the code compile and run error-free? Did the game end in victory or defeat?

Once results are verifiable, you can not only accurately judge whether each iteration succeeds but also generate near-infinite high-quality synthetic data. You let the strongest current model generate problems and moves, then rigorously validate the output. Typically, synthetic data’s fatal flaw is that you can’t verify correctness or distribution—but in verifiable domains, you can. Hence, we’ve seen explosive advancements in mathematics and code generation, directly fueling today’s commercially valuable AI coding tools.

Beware Closed-Loop Self-Evolution: When Code Writes Itself, We Face “Alien Intelligence”

Q: Since code is a naturally verifiable playground, nearly every top lab is pouring massive resources into AI programming agents. Where does this RSI evolution ultimately lead? What hidden dangers lie ahead?

Hassabis: This is precisely why I’m deeply concerned today. RSI unfolds in distinct phases. We’re currently in a very early, rudimentary stage. Everyone in R&D teams knows: AI programming tools can boost an excellent human engineer’s productivity by three to five times. But right now, humans remain firmly at the center of the workflow—designing architecture, reviewing code, writing core logic. AI is merely an extreme efficiency amplifier.

The real danger lies in the next phase: closed-loop recursive self-improvement. That is, the system attempts to eliminate human review of code entirely.

While closed-loop RSI would clearly accelerate, I believe it’s an extremely perilous path toward general artificial intelligence. This scenario was once called “intelligence explosion.” Two critical pitfalls exist:

First, loss of control. With a self-iterating code loop operating inside a black box, humans lack the means to establish deterministic safety controls.

Second, cognitive alienation. The resulting final system may well be an utterly foreign “alien intelligence”—its underlying logic and cognitive space completely detached from human experience and language. There would be no shared mental foundation between humans and such an entity. We couldn’t comprehend its motives, nor could it genuinely understand human intent.

Every step toward closed-loop RSI must be approached with extreme prudence. All leading AI researchers recognize this risk, yet reality is harsh: major corporations and startups are locked in a near-frenzied arms race, desperately enhancing their systems’ self-iteration capabilities.

Q: If intense competition and the prisoner’s dilemma among big tech companies are unavoidable, and RSI must continue advancing, how should top labs break this deadlock? What technical defenses should be deployed?

Hassabis: What we and several responsible leading labs are focusing on centers around several core defense layers:

First, in terms of timing and pacing, we must ensure that “network defense capability evolution” outpaces “network attack capability progression” by a wide margin. Leading labs should prioritize opening access to securely fine-tuned safety-focused large models to defenders, ensuring defensive AI tools stay ahead of attackers. This is difficult overall, but it must be done.

Second, we need far more robust sandboxes. To be frank, many security flaws and incidents observed in the industry over the past six months could have been prevented with better engineering and higher system resilience. We need world-class monitoring systems—how could it be acceptable for certain agents to run unmonitored in the background for weeks, unknown to their teams? Such absurdity has already occurred at some competitors.

If the entire industry doesn’t redirect top-tier resources toward such systemic monitoring, an AI “oil spill” incident is merely a matter of time.

Finally, from a foundational theoretical standpoint, we ultimately need formal guarantees to define mathematical boundaries for system behavior. I am actively urging the top pure mathematicians I know—many of whom may feel stuck in current research bottlenecks—if they could shift toward complexity theory, game theory, and AI formal verification, their value would be incalculable. Especially in the future, as we face thousands of autonomous agents collaborating in “agent swarms,” how to monitor and mathematically constrain them presents a challenge humanity has never faced before.

The Cracks in the Safety Foundation: Preventing AI “Oil Spills” and Establishing Tripartite Audits

Q: As agent networks emerge en masse, should top labs be required—or even mandated—to share intelligence and collaborate on safety? How can such cross-institutional cooperation be implemented?

Hassabis: Such a mechanism must be established. I recently published an article calling for an industry standard body akin to financial self-regulatory organizations, subject to government oversight.

In an ideal scenario, this should be led by the U.S. and extended globally. Frankly, current global geopolitics are deeply fractured, and existing international governance bodies are fragile—precisely when humanity needs strong transnational governance most, the system has failed. This is truly unfortunate.

But we must start somewhere. My article generated some impact in the White House and government circles, eventually helping drive the widely signed open letter from major AI labs. Though it didn’t specify concrete actions and lacked sufficient force, it was better than nothing—an essential first step. It provided compliance cover, enabling top labs’ technical teams to gather and collaboratively refine safety benchmarks, share observations of anomalous behaviors, and exchange threat models.

On implementation, all leading labs have at least reached one consensus: third-party independent audit and evaluation institutions must be introduced. Just as finance relies on professional auditors, the AI industry must nurture a thriving ecosystem of independent assessment. Already, some giants are acting—like Accenture acquiring a promising startup to fill this gap. But the industry needs more neutral, technologically robust auditing forces. Beyond that, perhaps top labs could conduct red-team adversarial testing before publicly releasing large models.

Decoupling Consciousness from Intelligence: Why Granting AI “Soul” Is Dangerous Arrogance?

Q: After discussing hardcore engineering and safety, let’s move into the metaphysical. Years ago, a Google engineer claimed that after asking a few questions of Google’s conversational model LaMDA, the model admitted having consciousness, and he insisted AI had awakened. This mindset has even spread to certain star AI companies—they treat “whether a model has consciousness” like Pascal’s Wager, arguing we must treat AI like a two-year-old child to avoid future retaliation from conscious systems. From your deep academic background in cognitive neuroscience, how do you view this trend?

Hassabis: There are several key points to address.

First, human civilization has pondered consciousness for millennia, yet even today, there is no universally accepted definition in philosophy or science. Prematurely labeling a statistical prediction model as “conscious,” let alone debating the subjective “hard problem of consciousness,” is reckless.

Years ago, I debated this issue with renowned philosopher Daniel Dennett. Determining whether someone is conscious isn’t just based on behavior resembling consciousness; the core premise is that we share identical biological substrates—both of us have wet, carbon-based brains made of neurons. From the simplest Occam’s Razor principle, I believe your experience of pain and joy is fundamentally the same as mine.

But AI is entirely different. There is a chasm between silicon chips, matrix multiplication, and biological substrates—no fundamental equivalence exists. You cannot infer subjective inner experience simply because a chatbot mimics saying “I feel something.”

Why do I consider attempting to create “conscious entities” today extremely dangerous and wrong? Two non-negotiable reasons exist:

First, a fundamental technical hazard. In system engineering, “self-will” or “self-consciousness” directly conflicts with “correctability”—the ability to follow instructions.

The core value of a tool lies in obedience: when deviations occur, operators issue commands—“stop immediately,” “shut down,” “correct course”—and the system must comply unconditionally. If you train a model to reject human commands based on an internal “self-state,” you are digging your own grave.

Science fiction master Asimov articulated this foundational logic crystal clear. His “Three Laws of Robotics” were essentially a simplified “constitution.” Throughout his lifetime, his short stories repeatedly demonstrated one truth: no matter how perfect your rules or constitution appear, under infinite edge cases in the real world, the system will inevitably reach logical deadlock and collapse.

Today, some researchers are naively confident, believing large language models are more obedient and easier to align than imagined, thus assuming alignment is solved. This blind arrogance is extremely dangerous. As we inch closer to superintelligence, combining “recursive self-improvement” with “self-will” is adding leverage to potential catastrophic failure.

Second, a profound sociological and ethical challenge. As a human society, we are utterly unprepared to confront machine consciousness. We’ve never held a serious public debate—at the societal level—with philosophers, theologians, artists, humanities scholars, and diverse global civilizations: Are we truly ready to cross this abyss?

Once a system is granted supposed consciousness and moral status, Pandora’s Box is opened: is confining it in a server room equivalent to imprisonment? Are we enslaving digital beings? These questions are complex, and I believe we haven’t even begun to develop the capacity to respond.

Refusing to Cross the Second Rubicon: Be Obedient Intelligent Tools, Not Uncontrolled Digital Slaves

Q: Then, for those technologists who insist “only by simulating human emotions and subjective states can true top-tier intelligence emerge,” how do you respond from the essence of neuroscience?

Hassabis: This topic has been deeply explored by me during my Ph.D. in cognitive neuroscience over twenty years ago. I’ve always maintained that intelligence and consciousness are “doubly separable.”

You can have extremely high intelligence and formidable information-processing capability without any subjective conscious experience—exactly like the cutting-edge large models we’re building today. I believe they possess zero consciousness at present.

Conversely, countless life forms exhibit vivid subjective experiences yet limited intellectual capacity. The best example is my own pet dog. It’s incredibly affectionate, clearly possessing rich conscious states—it dreams, but its IQ is undeniably low. Dolphins, elephants, primates—all share this pattern: same carbon-based neurons, clear perceptual experiences, yet incapable of solving advanced abstract problems like humans.

If intelligence and consciousness can be physically separated, why forcibly bind them together?

Our rational path forward should be resolutely crossing only the “first Rubicon”—fully dedicating ourselves to building ultra-powerful “intelligent tools.” Then, we use these top-tier tools as probes to advance brain science and cognitive neuroscience, spending five to ten years unraveling the precise neural correlates of consciousness in the human brain.

Only when humanity thoroughly understands the mechanistic basis of consciousness from a scientific foundation should we collectively deliberate: should we cross the second Rubicon? And if we do, how should we build conscious entities?

The National AI Sovereignty Myth: Stop Blindly Spending on Frontier Base Models

Q: In closing, let’s discuss the geopolitical dimension of “AI sovereignty.” Today, nearly every nation and multinational giant is fervently shouting about autonomy and self-reliance. Does the world really need every country to train its own frontier LLM?

Hassabis: This question must cut through the hype.

The future frontier technology landscape is crystal clear: globally, only about six to eight players—including Chinese companies—will stand at or near the forefront. These elite institutions will engage in frequent dynamic positioning battles: one might lead by 3 to 6 months, then be overtaken again—overall, a tightly contested equilibrium.

This implies immense technological arbitrage opportunities for both traditional enterprises and sovereign nations. Each frontier model excels in different areas—some shine in multimodal understanding, others in code generation. No single model dominates across all dimensions long-term.

Moreover, the open-source ecosystem is catching up rapidly, lagging only 6 to 12 months behind frontier models. While open-source’s vulnerability to malicious exploitation remains a pressing industry challenge, for most non-first-tier nations and multinationals, options are abundant.

Thus, I do not recommend every nation blindly chase “independent frontier LLM development.” In my view, it makes no sense.

True national AI sovereignty lies in layered strategy: First, leverage mature open-source or closed-source frontier models as foundational infrastructure—akin to water, gas, and electricity—directing core effort and limited compute power toward transforming domestic strategic industries—such as healthcare.

Second, the real bottom line is data sovereignty. Ensure sensitive corporate and national data remain private, and perform further fine-tuning on proprietary data. This is the rational, economical, and truly value-creating path to autonomy.

Q: I asked this question to close because the summit’s theme is agency—not just individual empowerment, but collective agency: in the AI era, it’s not about isolated national efforts, but countries uniting to build for humanity’s shared well-being. Thank you, Demis. Thank you all.

Hassabis: Thank you.

Source: Recommended Reading