Utah has quietly become the first US state to let an AI chatbot renew prescriptions without a doctor. The program, run by a company called Doctronic, launched in January and has set off a fierce medical debate. Residents can skip the doctor’s office and refill prescriptions online through the chatbot. It asks about their medication and history, checks a national pharmacy database, and either renews the script or escalates to a human doctor.
The launch was possible only through a “regulatory sandbox” that lets Utah officials waive laws for promising AI. State and federal rules otherwise restrict prescribing to licensed medical professionals. This sandbox was designed to encourage innovation, but critics argue it bypasses essential safeguards that protect patients from harm.
How the AI Prescribing System Works
Doctronic’s system is a chatbot that interacts with patients through a web interface. When a patient requests a refill, the AI asks for their name, date of birth, current medications, and medical history. It then cross-references this information with a national pharmacy database to check for potential drug interactions or contraindications. If everything appears safe, the AI approves the refill and sends it to a pharmacy. If there is any uncertainty or red flag, the case is escalated to a human doctor who makes the final decision.
In the current phase, a human doctor still reviews every refill before it is finalized. However, Doctronic expects to move to fully automated refills soon, meaning no human oversight will be required. This prospect has alarmed many in the medical community, who worry about the risks of relying on a machine to make clinical decisions.
The Medical Board’s Reaction
Utah’s medical licensing board says it only learned of the program when the January launch made the news. In an April letter, 11 members called for the pilot to be halted, citing the risks of auto-renewing drugs with side effects or interactions. “We were essentially told: ‘Yes this is going on. And no, you don’t have a say in it’,” said Dr Alan Smith, a family physician who chairs the board but spoke for himself.
The board’s concerns are not theoretical. Doctronic’s roughly 190 refillable medications include blood thinners, which can become dangerous if a patient develops internal bleeding. Other medications on the list include antidepressants, blood pressure drugs, and diabetes medications, all of which require careful monitoring. The American Medical Association has echoed the concern that “prescription renewals aren’t routine checkboxes”.
Despite the board’s plea, the state declined to suspend the program. The state argues that human doctors still review every refill in this first phase, so the risk is minimal. But critics point out that the oversight board overseeing the program is composed of five AI specialists, none of whom are doctors. This raises questions about whether the program is being evaluated with sufficient medical expertise.
Regulatory Vacuum by Design
The case exposes a jurisdictional tangle, since medical technology is regulated federally while medical professionals are overseen by states. Doctronic frames its AI as part of state-regulated medical practice, though some experts argue it has crossed into FDA territory. The company would not say whether it has sought FDA permission. The agency told the AP that it has authorized no AI chatbots but wants to encourage innovation, a hands-off posture that fits a broader loosening of oversight on AI health tools.
Critics see history rhyming, with Dr. Eric Bressman from the University of Pennsylvania comparing the moment to the haphazard medicine of the early 20th century, before boards and benchmarks existed. “We have crossed a threshold in terms of giving something that is not human a medical license, whether or not we want to call it that,” Bressman said. He and others say they are not opposed to AI prescribing, but want it held to standards as rigorous as those for human doctors.
The template for licensing AI medical services in other states comes from the Cicero Institute, a pro-AI think tank founded by Palantir co-founder Joe Lonsdale. This institute has been advocating for similar regulatory sandboxes nationwide, arguing that current laws stifle innovation and prevent patients from accessing cheaper, faster care. However, patient safety advocates warn that moving too fast could lead to serious harm.
Safety Concerns and Historical Context
The stakes are not abstract. Safety researchers have warned that medical chatbots can sound authoritative while dispensing dangerous advice. For example, some AI systems have recommended incorrect dosages or failed to recognize when a patient’s symptoms indicate a more serious condition. Others caution that removing humans from care can undermine the very outcomes it promises, as patients may lose the benefit of a physician’s intuition and experience.
Historically, medical prescribing has been tightly controlled because of the potential for abuse and error. In the early 20th century, before modern medical boards were established, anyone could prescribe drugs, leading to widespread addiction and death. The current regulatory system was built to prevent such tragedies. By bypassing that system, Utah is effectively returning to an era of unregulated prescribing, albeit with a high-tech twist.
Rivals are scrambling to map those failure modes too. Meta went as far as posing as teenagers to test how competing chatbots handle sensitive topics. These tests revealed that many AI systems are not yet ready to handle the complexities of medical decision-making. Doctronic plans to publish peer-reviewed studies later this year, though its only published paper so far was written by its own scientists and not independently reviewed.
What Other States Are Doing
Utah is not alone in experimenting with AI in healthcare. Several other states have considered similar sandboxes, but none have gone as far as Utah. For instance, Arizona and Florida have allowed AI to assist in diagnosis, but not to prescribe medications. The difference is significant because prescribing involves direct control over what drugs patients take, which can have life-or-death consequences.
The Cicero Institute, which helped design Utah’s sandbox, is now working with lawmakers in Texas, Arizona, and Tennessee to introduce similar bills. If those efforts succeed, AI prescribing could become widespread within a few years. However, the lack of FDA oversight and the absence of clear federal guidelines create a patchwork of rules that could confuse patients and providers alike.
Some experts argue that the FDA should step in and regulate AI medical devices more aggressively. Currently, the FDA classifies many AI health tools as “software as a medical device” (SaMD), but it has not specifically addressed chatbots that prescribe drugs. This regulatory gray area allows companies like Doctronic to operate without a clear legal framework, putting patients at risk.
The Role of Healthcare AI Ethics
The ethical implications of AI prescribing are profound. If an AI makes a mistake, who is liable? The patient might sue the company, but doctors typically have malpractice insurance and are held to professional standards. Companies like Doctronic argue that they are not practicing medicine, but rather providing a tool that assists in medicine. However, critics say that if the AI makes the final decision, it is essentially acting as a doctor.
Another ethical concern is access. Proponents argue that AI prescribing can help underserved communities where there is a shortage of doctors. But if the AI is only available to people with internet access and a credit card, it could exacerbate health disparities. Additionally, the lack of a physical exam means that the AI may miss important signs that a doctor would catch in person.
Doctronic has said that it plans to address these concerns through transparency and rigorous testing. But as one Utah law professor put it, companies risk letting the technology race beyond the evidence, and betraying public trust in the process. The professor noted that once AI is given prescribing authority, it will be difficult to take back, even if problems emerge.
The medical community remains deeply divided. Some see AI as a necessary innovation that can reduce costs and improve access. Others view it as a dangerous experiment that could harm vulnerable patients. Both sides agree that more data is needed, but they disagree on how much risk is acceptable in the meantime.
In conclusion, Utah’s experiment with AI prescribing is a test case for the future of medicine. If it succeeds, other states may follow, and the role of doctors could change dramatically. If it fails, it could set back AI in healthcare for years. Either way, the outcome will have implications far beyond Utah’s borders.