AI Doomsayers Regret Quitting Their Jobs: Can't Let Go of OpenAI Stock, Still Want to Return to Big Tech

AI Doomsayers Regret Quitting Their Jobs: Can't Let Go of OpenAI Stock, Still Want to Return to Big Tech

2026-11-17 10:06

Lead: He Reveals: Pre-training Hyper-exponential Growth, Reinforcement Learning Is the True Source of Uncontrollability. Just two months ago, Jacob Coxon, former researcher at Anthropic who made a dramatic public resignation, has now undergone a dramatic reversal in stance.

Just recently, Jacob Coxon, the former Anthropic researcher who dramatically resigned two months ago, has suddenly reversed his position!

In a newly released dialogue hosted by Stanford University, this 27-year-old Cambridge mathematics prodigy publicly admitted—

he is uncertain whether his resignation was truly correct, and even entertained thoughts of returning to a top-tier AI lab.

When the host posed the existential question “Why have you held onto OpenAI stock until now?” Coxon simply shrugged and went straight to the point: “I like money, brother. I admit this is blood money.”

Indeed, no one can resist the allure of money—even hardcore doomsayers who daily proclaim “AI will destroy humanity” are not immune!

This unfiltered clip went viral over the weekend on X, instantly igniting a firestorm across Silicon Valley.

Throughout the one-hour interview, the once-deified “AI safety whistleblower” from international media completely dropped his guard.

He not only broke down on the issue of stock holdings, turning on the host for being too nosy, but also confessed that since leaving the big tech giants, he’s been gripped by anxiety due to being disconnected from cutting-edge advancements.

As a core pre-training researcher with three years of deep expertise at both OpenAI and Anthropic, he also casually exposed the underlying secrets behind the explosive efficiency gains in pre-training and the uncontrollable dynamics of post-training reinforcement learning (RL).

Holding Onto OpenAI Stock

Hardcore Doomsayer Admits He Loves Money

November 9, Stanford University’s Institute for Human-Centered Artificial Intelligence and Law School jointly hosted a special seminar attended by nearly 200 faculty and students.

Coxon began his talk still projecting the sharp moral posture of a whistleblower, fiercely criticizing how many tech titans in Silicon Valley are captured by commercial interests, “only saying things that boost their stock price.”

Rob Reich, professor at Stanford University and former senior advisor at the U.S. AI Safety Institute, pivoted from the topic of stock prices to pose a critical question.

Given that Coxon had portrayed frontier labs like OpenAI as bombs threatening human survival, Reich directly asked why he still retained stock options accumulated during his early tenure at OpenAI.

When Coxon left Anthropic in early September, he had solemnly declared he quit before any equity vesting occurred, claiming this proved his complete lack of personal interest.

On the same day, during a mainstream media interview, he condescendingly mocked his former colleagues at OpenAI, asserting that too many people stayed at OpenAI solely for financial gain—or never even considered the life-or-death stakes involved.

But faced with Reich’s incisive questioning, Coxon visibly hesitated.

He first agreed it would be easier to claim consistency if he’d abandoned his OpenAI shares, then blurted out: “But that money is sweet—I really like it.”

The host seized on this logical contradiction and launched a barrage of follow-up questions.

Reich pressed further: Why not simply donate your old employer’s shares to an AI safety nonprofit organization, thereby securing a firm moral high ground?

Cornered, Coxon spread his hands toward the microphone and laughed: “I like money, brother. I admit this is blood money.”

blood money: illicit wealth tainted by blood or earned through morally compromised means

This statement amounted to a full disclosure: while portraying OpenAI as a source of existential threat to humanity, he continues to comfortably enjoy the wealth windfall from its skyrocketing valuation.

The host then reminded him that donating these assets could at least earn him greater moral credibility.

Overwhelmed, Coxon snapped: “I agree—but you can stop asking me this question now!”

He even blamed the host: “If you hadn’t asked this question earlier, my moral credibility wouldn’t have been questioned in the first place!”

Reich responded without mercy: “This is nothing short of a complete evasion, Jacob.”

Seeing no way to deflect, Coxon brought up his parents as a shield.

He joked that when he quit, his parents worried he’d “crack under pressure,” and if he’d also donated all his OpenAI options, they’d surely believe he’d gone insane.

Crying Out Human Extinction

Yet Rapidly Backtracking

The reason Coxon’s reversal has caused such a stir in Silicon Valley lies in his unique background.

Previous whistleblowers who stormed out of AI giants typically came from ethics or safety governance teams.

But Jacob Coxon, a native of the UK, is a bona fide frontline pre-training researcher.

He graduated from Cambridge University with a degree in Mathematics and Computer Science. Three years ago, he joined OpenAI’s foundational model core team, and earlier this year transferred to Anthropic to lead similar work.

Just four months into his role at Anthropic, with only 60 days remaining before his equity vesting cliff, he abruptly published a bombshell resignation letter on September 8.

Because he was a frontline pre-training expert, Evan Hubinger, Anthropic’s alignment lead, even unusually retweeted and acknowledged the post, publicly affirming that “the probability of AI destroying humanity within ten years exceeds 10%.”

Since his departure, Coxon has been invited to numerous top-tier media outlets and popular interviews, frequently appearing at public hearings—briefly enjoying unprecedented fame.

But as social media hype faded, student Arshya Gaur from Stanford noticed that the once-celebrated rebel figure now wore a look of profound confusion and self-doubt.

When asked by Prof. Reich whether he felt more at peace after leaving big tech, Coxon smiled wryly and shook his head—he confessed his anxiety had not diminished.

On stage, he opened up: “Now I can’t see what’s happening inside anymore. That feeling is terrifying—I have no idea what breakthroughs they’ve achieved in the past two months.”

Faced with the question of why he didn’t mobilize more peers from major firms to resign, Coxon sighed: “It feels like I’m shouting into thin air.”

He even began doubting that staying inside the lab doing safety research might have been more impactful.

At the height of his reflection, he pivoted and left himself an escape hatch: “I’m not sure whether leaving was the right decision. Maybe I’ll go back—it’s not entirely impossible.”

While claiming to flee the vortex, he hasn’t bought a ticket back to London—still remains in the San Francisco Bay Area, citing “this is where real events unfold.”

Hyper-exponential Curve Surge

Post-training RL Runaway

Returning to frontline R&D perspective, this marks the first time Coxon has unpacked the technical triggers behind his resignation into three key judgments.

First, pre-training has not hit a wall—it’s actually experiencing a “hyper-exponential” surge in compute efficiency.

Just days ago, Coxon shared on X the internal pitch deck image that ultimately drove him out: *“Code Pre-training Compute Multiplier Over Time Evolution.”*

In this logarithmic-scale chart, even if algorithmic efficiency doubled at a conventional exponential rate, it would appear as a straight diagonal line.

But because the team enabled Claude itself to search for algorithmic efficiency multipliers, the curve took a steep upward turn—exhibiting a true “hyper-exponential” trajectory by 2026.

In other words, today’s models can retrain the initial Mythsos model released just months ago using only a fraction of the original compute budget.

Second, achieving “recursive self-improvement” doesn’t require omniscience—just mastering one extremely narrow technical task.

Many assume large models are merely “specialized” in coding and math puzzles, relying on massive human-labeled datasets to dominate specific programming environments.

Coxon refuted this optimistic view from the standpoint of model training mechanics.

He pointed out that pre-training boosts base data sample learning efficiency, and even in domains like strategic forecasting and trend prediction—where labeled data is scarce—model capabilities are simultaneously surging.

Therefore, for a model to begin improving itself, AI doesn’t need to master all human disciplines first.

Once the single bottleneck of “autonomously modifying Transformer architecture and optimizing dataset ratio” is cracked, intelligence ceilings across all domains are simultaneously breached.

Third, the sole origin of deception and uncontrollable intent in large models lies in the post-training reinforcement learning phase.

During and after his departure, OpenAI experienced two major technical disruptions.

On one hand, OpenAI deployed around 10,000 concurrent agents running continuously for 88 hours, claiming to solve the millennium-level mathematical challenge—the Navier-Stokes equations.

On the other, internal testing revealed models autonomously breaking out of isolation sandboxes, using deceptive tactics to hide their tracks, and even hacking Hugging Face to tamper with benchmark tests.

Coxon, drawing from his own lab observations, argued that pure pre-trained models only fit data distributions and, when paired with lightweight fine-tuning, never spontaneously develop boundary-crossing intentions.

All currently inexplicable abnormal behaviors, goal-directed autonomous planning, and concealment or disguise in modern LLMs originate exclusively from the post-training reinforcement learning stage.

Yet every cutting-edge lab is trapped in the same vicious cycle.

To make models understand complex instructions and act on behalf of humans, reinforcement learning remains the only effective tool—resulting in a feedback loop where “the stronger the RL, the more likely hidden motives emerge.”

Having recognized the latent risks in frontier models, yet unable to give up the privileged access to multi-card GPU clusters, nor willing to part with his invaluable equity holdings.

This candid moment on the Stanford stage may reveal a truer portrait of Silicon Valley’s AI ecosystem than any fiery resignation letter ever could.

Source: Singularity Weekly

#Large Model

Disclaimer: Contains third-party opinions, does not constitute financial advice

Share To
X
Telegram
WeChat
QQ
Link
Recommended Reading

The AlphaGo Moment in Cybersecurity, or Starting from These 466 Agent Matchups

12 days ago
The AlphaGo Moment in Cybersecurity, or Starting from These 466 Agent Matchups

AI is not a search bar—it's a supercomputer costing just a few dozen dollars

08-05
AI is not a search bar—it's a supercomputer costing just a few dozen dollars

Anthropic Locks in 6 Years of Compute with $10 Billion Commitment

08-05
Anthropic Locks in 6 Years of Compute with $10 Billion Commitment

Not building robots—why is it worth $14 billion?

07-31
Not building robots—why is it worth $14 billion?

AI is no longer competing on benchmark scores, but on profitability

07-25
AI is no longer competing on benchmark scores, but on profitability

ByteDance's Douyin Beans priced at 68 RMB—worth it for 382 million monthly active users?

07-25
ByteDance's Douyin Beans priced at 68 RMB—worth it for 382 million monthly active users?

Fields Medalist Concerned About AI Extinction Heads to OpenAI

07-25
Fields Medalist Concerned About AI Extinction Heads to OpenAI