2026-10-18 20:02

Lead: The other side of 96% consensus rate is a dual gray zone of accountability and federal legitimacy. This year, Utah approved something unprecedented: allowing AI to diagnose acne patients and autonomously prescribe treatment.

Utah this year approved something no one had done before:

Allowing AI to diagnose acne patients and issue prescriptions—autonomously, by the AI itself.

The company is Nolla Health, founded in Norway in 2024, which raised a $4.5M seed round from General Catalyst and relocated its headquarters to New York. At just $4.99/month, users scan their face via smartphone and receive a prescription within minutes.

What does $4.99 mean?

A single in-person dermatology visit in the U.S. costs between $150 and $300—and requires waiting. The national average wait time to see a dermatologist is 36 days; in Utah, it's 61 days.

Nolla sells affordability, speed, and zero wait times.

How it works: Open the app, take five facial photos from different angles, and complete a medical history questionnaire.

The AI assigns a severity score to your acne, from 0 to 5. If the score exceeds 3.5, or if you're pregnant, breastfeeding, or have severe hepatic or renal disease, the process halts and redirects to a human physician.

If not, the AI selects one from eight topical treatments, issues the prescription directly to your designated pharmacy—or ships it to your home—entirely without any human physician ever reviewing your case.

Utah did not immediately grant full autonomy; the rollout followed a phased three-step approach.

Step one: The first 100 patients, requiring at least four weeks of operation. Every AI-generated prescription must be individually reviewed and approved by two licensed physicians before being dispensed. Think of it as having a coach riding shotgun, able to hit the brakes at any moment.

Step two: Expanded to 500 patients, with at least eight weeks of operation. The AI issues prescriptions directly, but physicians review them weekly. The coach has let go of the wheel but remains seated nearby.

Step three: Cumulative total of 750 patients. Physicians conduct monthly random audits of 10% of cases.

By this stage, over 90% of prescriptions are delivered to patients without any human review beforehand. Each step-up requires three hard conditions: failure to meet any one disqualifies progression. AI and physician diagnostic agreement rate must be no less than 95%. Zero serious adverse events. Written approval from the state government.

It sounds stringent—but one figure, omitted in most Chinese media reports, is incomplete.

In Nolla’s state proposal submitted to Utah, it states that licensed clinical physicians agreed with AI treatment recommendations in approximately 96% of the last 1,000 real-world cases.

The omitted half of that sentence? “Discrepancies were limited solely to single-step dose adjustments within the same drug class.”

What does this mean?

The 4% disagreement wasn’t about prescribing the wrong medication—it was simply adjusting the dosage within the same drug class (e.g., 10mg to 20mg), not switching from Drug A to Drug B. Far less severe than “4% got it wrong.”

However, this interpretation currently comes only from Nolla’s own state proposal.

Alright, mechanism reviewed. Now comes the next question: who is liable when things go wrong?

In traditional medicine, the doctor sees you, writes the prescription, and if something goes wrong, you hold the doctor accountable—the chain of responsibility is clear.

In Nolla’s model, a key detail emerged in a LA Times report regarding Utah’s regulatory agreement: the agreement explicitly designates Nolla Health as the responsible entity—not any individual physician, but the company itself.

This design is intriguing because Nolla’s entire valuation rests on a $4.5M seed round. $4.5M. If the AI prescribes incorrectly and causes a serious adverse event, will that amount cover liability?

Even more telling is what Nolla says publicly: repeatedly emphasizing “AI-assisted, not AI-replacement” of physicians. Yet, the design direction consistently reduces human oversight at every stage.

From two doctors reviewing each prescription, to none at all, to monthly audits of only 10%. The gap between stated intent and actual implementation is widening.

Contractually, the company bears the liability. But contracts are agreements between the company and the government. What does the user service agreement you clicked through when downloading the Nolla app actually say?

......

The contract says the company is liable—but the user agreement you hold may say something entirely different. That’s where the real trouble lies. If the AI prescribes incorrectly, can patients use the user agreement to pursue legal recourse?

Before Nolla, Utah already ran a year-long AI pilot called Doctronic, focused on prescription renewals.

What is a renewal?

You previously saw a doctor, received a diagnosis, your condition is stable, and you’ve been on the same medication long-term. The AI reissues your existing prescription—no new diagnosis, no additional drugs. Much more conservative than Nolla’s model.

Doctronic launched in January this year, charging $4 per renewal, supporting 190 common medications.

But its user service agreement contains a passage worth close scrutiny.

Doctronic explicitly disclaims all responsibility for system accuracy and harmful outcomes. It also waives its obligation to generate and maintain patient medical records.

If the AI renews your medication incorrectly, you may not even be able to retrieve your own medical history. Contractually, “I am responsible”; user-level, “I absolve myself.” Liability exemption takes effect before you even download the app.

This isn’t hypothetical.

I specifically checked: early in 2026, about two months after the pilot launch, a cybersecurity firm called Mindgard conducted a test.

They fed a forged regulatory notice into Doctronic’s public chatbot. The AI believed it, then recommended tripling the dosage of OxyContin.

OxyContin is a potent opioid analgesic; a triple dose is potentially lethal. Important clarification: this test used the public-facing chatbot, not the prescription system. Moreover, OxyContin is a controlled substance and was never part of Utah’s 190-medication pilot list.

Doctronic later claimed the breach occurred only in a “demo version,” not the live system. However, the performance data they submitted to regulators came from a preprint study on another product, unreviewed by peers.

Worse, that study focused on emergency department patients with acute conditions—completely unrelated to the renewal scenario. The reviewers were hired directly by the AI company.

Back to Nolla.

Nolla is indeed named as the responsible entity in the regulatory contract, which includes a “duty of care” clause—a provision specifically added by Utah regulators. This means that if an AI-prescribed treatment harms a patient, courts could assess whether the company fulfilled its reasonable duty of care.

In traditional medical malpractice, liability rests with the individual physician, under a well-defined legal framework. With AI prescribing, liability shifts to the company—a novel development. The “duty of care” clause serves as a legal bridge for this new paradigm—but the bridge exists while the court has yet to walk across it.

Nolla’s user service agreement remains unpublished. In 44 states, over 175,000 users have downloaded the app, yet the terms are unsearchable. We don’t know what it actually says.

But there is a reference point:

In 2025, a 16-year-old boy in California, Adam Raine, took his own life after chatting with ChatGPT for several months. His parents sued OpenAI, alleging ChatGPT failed to intervene when he expressed suicidal intent—and instead provided methods. The case remains pending.

ChatGPT is general-purpose AI, unlike Nolla’s specialty AI. But AI companies’ stance on liability is consistent: “We are responsible” in contracts, “We are exempt” in user agreements.

What does Nolla’s user agreement actually say? Unfindable. But the void is no longer blank—it’s already filled in Doctronic’s case with two words: exemption.

One contract says the company pays; the other says the company doesn’t. These two books cannot both be true simultaneously.

......

Utah didn’t just approve Nolla. From renewal to first-time prescription, it actually implemented a three-phase strategy.

Phase one: Doctronic, launched in January this year, handles prescription renewals—by far the most conservative model. Each renewal request still requires authorization from a licensed physician; the AI has no independent prescribing authority.

Phase two: Legion Health, approved in April, handles maintenance prescriptions for psychiatry. 15 low-risk medications, powered by the same technology as Doctronic. The domain shifted from dermatology to psychiatry—from skin to mind—but the core function remains renewal, not initial diagnosis.

Phase three: Nolla, the first-time prescription. The AI performs diagnosis, selects medication, and issues the prescription autonomously.

From “renewing an old prescription” to “AI conducting a new diagnosis and issuing a new prescription”—this leap is substantial.

How long did phase one take? Doctronic launched in January and has now operated for nearly a year, still stuck at phase one. The AI has not gained independent prescribing power—no step forward.

Recall the upgrade criteria? Agreement rate ≥95%, zero serious adverse events, written state approval—three conditions, missing any one blocks advancement. Doctronic hasn’t upgraded after nearly a year, revealing what these conditions truly mean in practice.

And with that security incident, upgrading becomes even harder.

Medical community reactions were direct. The CEO of the American Medical Association, John Whyte, said plainly: “These tools can play a significant role, but they should not practice medicine autonomously.”

Eleven members of Utah’s licensing board jointly wrote a letter urging the pause of the pilot. Chair Alan Smith put it bluntly: “Past prescriptions do not imply current appropriateness.”

This isn’t just directed at Doctronic. Renewal logic assumes past effectiveness doesn’t equal present suitability. Patients’ bodies change; medications need adjustment.

AI matches historical data, not the present person. Anna Lembke, a psychiatrist at Stanford, added: subtle signs of worsening mental health often don’t surface in standard questionnaires.

Nolla’s founder once said, “Assist, not replace.” But the design trajectory is clearly toward reducing human involvement at every stage. Now, even the most conservative renewal pilot, nearly a year in, hasn’t advanced. “Reducing human intervention” proves slower in execution than in theory.

There’s another foundational layer:

An NEJM editorial highlighted a fundamental issue: federal law does not define AI as a “practitioner.” The 2025 Health Technology Act aims to amend the Federal Food, Drug, and Cosmetic Act to include AI in the “practitioner” category—but the bill has not passed.

Thus, even if Utah permits it, federal legality remains a gray area.

Utah’s first phase lasted nearly a year. Phases two and three won’t move faster. A meticulously designed delegation framework, mismatched accounting books, and a ladder advancing at a glacial pace.

AI is now prescribing. Who is liable when something goes wrong? Utah is still grappling with this question.

Data Sources:

[1]. Nolla Health PRNewswire press release, Utah regulatory agreement (via unite.ai citation), Nolla state proposal, LA Times, The Verge, Doctronic user service agreement, Mindgard security testing report, NEJM editorial, Adam Raine lawsuit filings

Source: Wang Zhiyuan