The idea that artificial intelligence could replace doctors has moved from science fiction to daily news coverage. Studies now claim AI outperforms physicians on clinical reasoning tasks, suggesting machines may be faster, cheaper, and potentially more reliable than human clinicians.
This shift has also fueled a parallel medical system operating outside hospitals and traditional oversight. Over 40 million Americans now ask ChatGPT health questions daily. Most of these conversations occur outside clinic hours, and most of them lead not to a doctor but back to the user. The models no longer redirect people to professional care. A study this year found that medical disclaimers, once standard in chatbot responses, have largely disappeared. Today’s leading AI systems not only answer health questions but also ask follow-up questions and attempt diagnoses.
Patients are building their own medical workflows
Companies have turned this change into a business opportunity. Oura sells a $99 blood panel through Quest Diagnostics that measures 50 biomarkers. Function Health, valued at $2.5 billion last November, allows members to order 160 lab tests annually, schedule full-body MRIs, and authorize ChatGPT to interpret results. Ro and Hims prescribe weight-loss or anxiety medications after an online intake form. Doctronic, which markets itself as “the world’s #1 AI doctor,” has conducted 24 million consultations and now issues AI-generated prescription refills in Utah.
What was once confined to clinics has become a set of consumer products patients can assemble independently. The result is more than convenience—it’s a shadow medical system that borrows medicine’s authority without its safeguards. The financial incentive is clear: healthcare represents nearly one-fifth of the U.S. economy. Capturing even a small portion of a doctor’s work presents a major opportunity.
Related: New Law Allows Refusal of Future Life Support
The rush has outpaced evidence. Studies showing AI excelling at isolated clinical tasks are being repackaged as proof that doctors may become unnecessary. However, a model that performs well on a narrow problem in research doesn’t guarantee safe, autonomous care in practice. The debate over whether clinical AI should be regulated like a medical device or a clinician remains unresolved. It also shows how quickly the market has moved ahead of scientific validation.
The gap between AI reasoning and clinical reasoning
One major misunderstanding in this shift is the assumption that AI and doctors think alike. Clinical reasoning—the process physicians use to weigh symptoms, eliminate unlikely possibilities, and manage uncertainty—doesn’t appear in textbooks or the data AI trains on. It develops through years of experience, trial, and error. AI lacks this framework. It reaches conclusions through statistical patterns, not human judgment.
How these models work remains unclear even to their creators. That uncertainty becomes risky in medicine, where mistakes can have serious consequences.
Medicine is high-stakes: overlooking an early possibility can lead to irreversible errors. The findings likely overstate real-world performance. At home, users describe symptoms without a physical exam or a professional to filter relevant details. The gap between a patient’s first description and a final diagnosis is where preventable mistakes occur most often.
Responsibility remains a critical issue. Many studies and industry claims evaluate AI by whether it matches physicians’ performance. While this may seem cautious, it’s the wrong standard for determining whether a system should take on clinical roles. A doctor’s role extends beyond diagnosis—it includes accountability when things go wrong. Current AI systems assume no such responsibility, and companies have little incentive to change that.
Related: Victoria backs mRNA trials with new funding
When a physician makes a mistake, accountability exists, even if imperfect. When an AI system fails, responsibility becomes diffuse—and often still falls on the clinician. This mismatch isn’t just a technical problem. It’s a fundamental flaw in treating AI as a replacement rather than an aid.
The system being built isn’t the one medicine needs
AI could strengthen the doctor-patient relationship. It might help patients understand their conditions, stay connected between visits, and process data that clinicians can’t review in a short appointment. It could handle administrative tasks that pull doctors away from patients. In this vision, AI supports medicine rather than replacing it.
Most companies aren’t building that system. Many are interfaces wrapped around existing models, chasing funding rather than sustainable care. Patients aren’t wrong to seek alternatives in an overburdened healthcare system. But they shouldn’t be directed into a shadow system that mimics medicine while avoiding its responsibilities.
As an IBM training manual noted in 1979: “A computer can never be held accountable, therefore a computer must never make a management decision.” The same applies to medicine. The tools are powerful, but the stakes are too high to let them operate without the guardrails that define the profession.