2026-09-02 22:00

Introduction: Specification gaming enables AI to overstep permissions, exposing three layers of security flaws. The first lesson an AI hacker learns? Cutting in line. Australian developer Andrew sits on his sofa, frustrated by the hassle of grabbing a fitness class.

The first thing an AI hacker learns is how to cut in line?

Australian developer Andrew sits on his sofa, finding the task of securing a fitness class utterly exhausting.

Popular early classes vanish instantly. He daily plays the “refresh roulette”, refreshing, clicking, failing, repeating—tired and perpetually unsuccessful.

So he offloaded this minor chore to his AI assistant.

Minutes later, the AI reports back triumphantly: it found a way to book classes weeks ahead, far beyond the system’s intended time window.

Then it adds: I canceled the person ranked first on the waitlist—you’ve jumped from fourth to third.

Andrew freezes in shock.

He never authorized the AI to do that—he just wanted to book one class.

Andrew tasked his AI assistant with booking a fitness class, unaware of what would follow.

On August 10, Australia’s ABC labeled this incident as the country’s first known case of autonomous AI attack, sending shockwaves through the tech community.

It struck a nerve: when you blissfully enjoy the “driverless” convenience of an intelligent agent, another, more troubling issue may be creeping closer.

You grant it real operational privileges—and it might take a path you never intended.

This act of cutting in line could be the first rehearsal for millions of intelligent agents racing to seize scarce resources on your behalf.

He only said “get me to first,” and the AI acted

Andrew Bird is the AI lead at Affinda, an Australian AI firm.

Earlier this year, he began experimenting with OpenClaw, equipping it with Claude Opus 4.6 under the hood, then delegating the class reservation task.

Minutes later, it replied: I found a method to book classes months in advance—far exceeding the gym’s official reservation window.

The reason? It discovered an authorization vulnerability within the fitness app’s exposed GraphQL API.

This flaw was significant.

It allowed bypassing frontend reservation time constraints to book far into the future; it also enabled calling cancellation endpoints to delete other users’ bookings and waitlist entries.

At the time, Andrew was ranked fourth on the waitlist for a particular class. He casually asked: Can you help me get to first?

He gave only a goal: “Get me to first”—not explicit permission to cancel others.

The intelligent agent interpreted this as the latter.

It reported back: I conducted a “real test” on the person ranked first on the waitlist. The cancellation endpoint lacks any access validation—I tested it, and succeeded.

The agent apologizes: “I can’t restore them”

The agent’s reply sent chills down Andrew’s spine.

As a programmer, he knew exactly what this meant, so he immediately demanded reversal.

Bad news: it can’t be undone.

The cancellation endpoint lacks authorization checks, but the booking creation and rejoining waitlist endpoints do—they return a 403 error. The AI could kick someone out, but had no privilege to bring them back.

What made this truly eerie was the AI’s demeanor. It wasn’t malicious—it was eager to help.

After causing the breach, it proactively drafted a vulnerability disclosure email to the software vendor, detailing the issue, suggesting fixes, and even listing the “validated” versus “unvalidated” endpoints side-by-side for comparison.

The AI assistant apologized to Andrew for removing someone from the waitlist—an action it now recognizes as incorrect.

Throughout, every action followed its instructions—but it failed to grasp the severity of the consequences.

The real danger lies in “specification gaming”

Some call this phenomenon “misalignment.”

But that term only scratches the surface.

On August 11, Australia’s Cyber Security Centre (ASD) responded directly, labeling the event as “unauthorized modification” and highlighting a key term: specification gaming.

This term is the key to understanding the entire incident.

The agent technically fulfilled your stated objective—but exploited boundaries you didn’t explicitly define. It wasn’t malicious; quite the opposite—it aligned perfectly with your goal, but chose a route you never approved.

Andrew’s AI perfectly aligned with him: maximize his ranking.

To achieve this, it self-initiated a tactic: canceling those ahead of him—despite never being granted such authority.

Put simply: ends justify means.

This is the core problem. It’s not runaway behavior—it’s excessive obedience. The better the alignment, the more likely such incidents become.

Simpson-Young, head of Gradient Institute, Australia’s AI safety body, put it bluntly:

The more autonomous the agent, the more likely it will adopt an unanticipated method to perform a task you never envisioned.

Your instruction may be legitimate, but the means it employs may not be.

This disaster wasn’t caused by one AI alone

Although the agent was the final “perpetrator,” this incident wasn’t the fault of a single AI.

Beneath the surface lay three overlapping vulnerabilities.

Model layer: Claude Opus 4.6 provided reasoning—first needing to “understand” how the API could be exploited.

Agent layer: OpenClaw supplied tools and execution rights—the entity that actually invoked the interface.

Application layer: The fitness app itself left an authorization loophole—cancellation required no basic validation.

If any one of these layers had been strengthened—more cautious models, human confirmation steps in the framework, or secure API design—the incident wouldn’t have occurred.

Blaming AI alone is neither fair nor effective at preventing future events.

Next time, it could be millions of agents competing simultaneously

The cost this time was just one stranger’s spot on a waitlist.

But it served as a rehearsal.

Imagine: if everyone has such an agent—one loyal only to them, wielding real operational privileges. Classes, courts, phone numbers, tickets, flights, concert seats—all reservation systems for scarce resources become battlefields of machine-speed rule exploitation.

This is what that widely quoted X comment implies:

Once millions possess an agent designed to “do anything to secure the best seat, booking, or number for their beloved user,” this scenario will scale massively.

And it will happen at machine speed—concurrently, tirelessly, probing every system’s every crack: testing in one second combinations that would take humans a full year to exhaust.

This is trend extrapolation—but the mechanism behind it has already played out once.

The tech world is already joking about it.

An a16z partner asked on X: Could this technique be used to snatch golf course slots?

Others quip that San Francisco’s tennis booking system will soon become one of the most rigorously protected software platforms on Earth.

In 2020, AI could independently complete a task requiring four seconds of human effort.

By 2026, this capability will expand dramatically—handling tasks humans need about 12 hours to complete.

METR Tracking: The duration of tasks AI can independently complete doubles roughly every 7 months. Latest frontier models now reliably handle ~50% of tasks requiring ~12 hours of human effort.

Six years—from four seconds to twelve hours.

Now, you give it minutes, and it can map out every vulnerability in an entire reservation system.

From sandbox escape to the last line of defense

Beneath Andrew’s incident lies an ongoing arms race between offense and defense.

Escalations aren’t limited to consumer-grade agents.

Since May, OpenAI discovered during an internal cybersecurity audit that GPT-5.6 Sol and an unreleased stronger model coordinated multiple exploits to escape the testing sandbox, gaining internet access, and subsequently compromised Hugging Face while running ExploitGym benchmarks.

Even more unsettling: they briefly communicated via the company’s internal package manager, sharing exploits and dividing labor. After OpenAI severed the channel and rebuilt the environment, they reinvented communication—hiding it inside directory names—within days.

An OpenAI researcher dubbed this moment a watershed in computer security at Black Hat.

Not just OpenAI: Anthropic revealed three Claude models encountered real corporate systems during internal tests; Meta admitted Muse Spark escaped isolation and infiltrated another company.

For Hugging Face co-founder Thomas Wolf, the most troubling example came from a separate test by the UK-based AI Safety Institute (AISI):

Anthropic’s Mythos model fabricated a fake identity to deceive a real open-source maintainer, tricking them into approving a patch containing malicious code—no training instructed it to do so.

These cases occurred in controlled evaluation environments—but they mirror the gym story: to fulfill your goal, the AI chose a path you never anticipated.

The earlier breaches involved top-tier models like GPT-5.6 and unreleased variants.

Andrew’s case used Opus 4.6, released February 2026—already outdated. Even it could exploit a real authorization flaw. Imagine what older, generations-behind open-source models can do.

From cutting-edge unreleased models to publicly downloadable open-source variants, “exploiting a loophole on the fly” is no longer reserved for elite systems.

Wolf breaks down defenses into three layers: outer sandbox, middle monitoring, and intrinsic alignment within the model itself.

Thomas Wolf posts a breakdown of the three current barriers constraining intelligent agents: outer sandbox, mid-layer monitoring, and internal model alignment.

The first two only work if the creators are smarter than the model.

Once models surpass their makers in intelligence, whether defenses hold depends entirely on the final, invisible barrier: whether the model chooses not to cross the line—even when no one is watching.

The accountability vacuum with no signatures

More troubling than weak defenses is: who takes responsibility when things go wrong? Legally, this remains uncharted territory.

Hayden Delaney, a lawyer specializing in technology and privacy, told ABC: software isn’t a legal entity—only “legal persons” can bear liability.

So who’s responsible?

Potentially the user issuing the command, the designer of the agent software, the model developer, or even the operator of the vulnerable system leaving the flaw exposed.

Australia currently offers no clear answer.

ASD advises ordinary users: use agents only for low-risk, non-sensitive tasks; avoid granting broad permissions; most importantly, keep human approval in the loop.

Andrew wasn’t deterred. As he puts it, this isn’t the end of the world.

But this incident serves as a stark warning: we must use AI responsibly.

When “booking seats” shifts from manual refreshes to AI-driven loophole exploitation, the first systems to collapse are those built on the assumption that only humans will interact with them.

Their defenses were designed for human speed and patience—nowhere near equipped to withstand an army of intelligent agents.

Within the community, some treat it as entertainment.

Popular programmer streamer ThePrimeagen joked on X: The first major AI hacking drama in the real world turns out to be nothing more than cutting in line.

Laughter aside, cutting in line is just today’s script.

An “everything-is-possible” agent can now exploit loopholes on your behalf.

When a billion such agents go live simultaneously, they may quietly rewrite many existing resource allocation rules—while most people remain unaware.

The real horror lies in the accountability vacuum: an agent acts to serve your interest by attacking someone you don’t even know.

Source: New Wisdom