ewicklund@healthleadersmedia.com, Linkedin
Healthcare is in the middle of an evolution. Faced with a continuing decline in doctors and nurses and an increase in clinical care needs, health systems and hospitals are looking to AI to fill in the gaps.
Amid the hype and promise, a growing tension is becoming clear: the reality of what AI can deliver versus what it can’t. While advocates tout AI’s potential to revolutionize healthcare delivery, the technology still faces significant hurdles in most areas.
That may not be what a healthcare industry fraught with waste, burnout and misaligned incentives wants to hear, but that’s what we’ve got.
Healthcare executives are dealing with ethical concerns, governance issues, and the technology's current lack of maturity raise questions about its readiness to assist in decision-making processes. As they grapple with these challenges, many are realizing that the road to effective AI integration is more complex and cautious than some had anticipated.
TAKEAWAYS
-
The healthcare industry is eager to adopt AI for clinical care, but the technology’s current limitations and immaturity raise concerns about its ability to deliver on its promises.
-
As AI tools become more integrated into healthcare, ethical concerns, governance complexities, and the cost of maintaining AI systems are significant hurdles that need to be addressed.
-
While large health systems partner with tech giants to explore AI, smaller and resource-limited healthcare providers struggle to implement and govern AI effectively, potentially widening the gap in care quality.
AI in Clinical Care: Is There an ROI?
Is a complex department ready for a complex “solution”?
Large and expansive health systems like Providence, Intermountain Health, Kaiser Permanente, the Mayo Clinic and Cleveland Clinic are partnering with tech giants to develop and launch AI programs, sometimes several at a time. Smaller health systems and hospitals, however, don’t have the resources to go that route. They have to pick their path carefully and look for immediate results.
Most healthcare organizations aren’t ready to use AI to affect clinical care, in large part because the technology isn’t yet mature enough to trust with decision-making functions. But providers are using AI tools to improve tasks like identifying and messaging certain populations and connecting them with appointment scheduling, reminders and other resources.
Vi-Anne Antrum, DNP, RN, MEA-BC, CENP, FACHE
CNO, Cone Health
At Cone Health, clinicians are using an AI tool developed by Lirio to sort through the reams of data they have on underserved populations and create targeted messages aimed at re-engagement.
“It's not that they were going to our competitors,” says Vi-Anne Antrum, DNP, RN, MEA-BC, CENP, FACHE, the North Carolina-based health system’s CNO. “They just weren't seeking healthcare.”
“They don't have to lose limbs; they don't have to wait until they have a stroke or need dialysis or have a heart attack.”
—Vi-Anne Antrum, DNP, RN, MEA-BC, CENP, FACHE, CNO, Cone Health
In just a few months, the platform has helped them reach more than 2,000 people and generated some 300 primary care appointments that have led to a diagnosis of either hypertension or diabetes, two chronic health concerns that often lead to serious health concerns or even death if untreated.
“They don't have to lose limbs; they don't have to wait until they have a stroke or need dialysis or have a heart attack,” Antrum says. “We're really using that with under-engaged patients with specific targeted messaging to those people.”
Like many smaller health systems, Cone Health serves a large percentage of underserved patients, often rural. The opportunity to use AI to identify how and where to focus outreach and deliver care is crucial.
“You're getting them in for visits that they normally wouldn't do because … they haven't been contacted before,” Antrum points out.
AI is also proving popular as a dictation tool, giving clinicians the opportunity to focus on their patients while an app captures the conversation and extracts important data for the medical record.
Some providers are taking that capability a step further. They’re designing AI tools for specific care pathways, such as cancer or chronic diseases.
Global Generative AI in Healthcare Market (Size, by application, 2020-2032 (USD Billion)
SOURCE: AI in Healthcare Statistics 2024 By Pioneering Health Tech (https://scoop.market.us/ai-in-healthcare-statistics/)
At Texas Oncology, a Dallas-based cancer care health system covering most of the Lone Star State, more than 1,000 clinicians across close to 300 locations have access to an ambient AI tool from DeepScribe that allows them to customize care delivery. That customization is important, because no two patients undergoing cancer treatment are experiencing the same side-effects and services. And an AI tool can drill down into the data faster and more efficiently than any doctor or nurse and fine-tune treatment protocols.
“Cancer care is becoming increasingly complex,” says Gurjyot “Gury” Doshi, MD, a medical director and physician at Texas Oncology using the new tool. “This really does allow us to sit with our patients and focus on the human in front of us.”
Texas Oncology works with DeepScribe to fine-tune the technology to understand words and phrases that are often used in cancer care. This ensures that the transcriptions are more accurate, which in turn leads to more effective care.
James Lindsey
Principal, IT strategy and innovation, Texas Oncology
James Lindsey, Texas Oncology’s principal for IT strategy and innovation, says off-the-shelf AI products—and there are dozens, if not hundreds, out there now—aren’t accurate enough to be of use. They don’t recognize the drugs or terms used in cancer care and would thus drift off into hallucinations when confronted with an unfamiliar word. In addition, he and his team can prompt the AI tool to understand not only English but Spanish, as well as Spanglish.
“This is the future of oncology,” he says, noting the technology enables clinicians to review their notes more quickly and efficiently, reduces errors or misinterpretations, helps patients to get the right care they need sooner, and enables care teams to treat more patients per day.
“Clearly we are still doing the work,” Doshi says, regarding clinician review of all AI-generated notes. “But now we can start reaching for the stars.”
The important thing to remember is that, while automation simply repeats a process over and over and over again, AI is being designed to learn from that process, and to improve what it does. Healthcare executives and the clinicians using AI are banking on that gradual improvement, even expecting the technology to someday crunch the data and suggest all the alternatives and the best care options.
“With actual delivery of care, we may find a future state where they actually want AI to be part of that care delivery,” says Antrum, at Cone Health
An important factor in AI adoption is integration with the electronic health record, which directly affects—good and bad—so much of the clinical workflow these days.
“Off-the-shelf AI products—and there are dozens, if not hundreds, out there now—aren’t accurate enough to be of use.”
—James Lindsey, Texas Oncology’s principal for IT strategy and innovation
AI is “the last-mile challenge,” says Phil Lindemann, vice president of data and research at Epic. “To be able to make is effortless [within the EHR] is really the special sauce.”
Lindemann and Carissa Kathuria, the R&D group lead at Epic who focuses on clinical applications and efficiencies, say many healthcare providers mistakenly assume that AI will be the silver bullet for all of the industry’s woes. The challenge lies in getting them to understand that this is a tool to facilitate better care delivery.
“The healthcare system is not really designed for physician efficiency,” Lindemann points out. “We are working to change that. In order to do that you have to start with incremental experiences—the boring stuff. You start small and build up over time.”
Gurjyot “Gury” Doshi, MD
Medical Director and Physician, Texas Oncology
In time, Kathuria says, clinicians learn how to use AI rather than sitting back and expecting AI to do its own little magic without them. They become better users of the technology, and in turn make the technology better for care delivery.
It’s a familiar refrain in the EHR space. Many healthcare providers thought the EHR would just simply come in and make healthcare better, to the point that the technology often made healthcare delivery worse. Over time, clinicians have learned how to use EHRs (and the technology has improved as well), and they now understand what the technology can and can’t do.
For now, Kathuria says, AI is best used for four functions: Creating messages, summarizing reaching for the stars.”encounters and information, translating, and automating actions.
“You start there, and see what [the technology] can do,” she says.
Addressing The Elephant in the Room: AI Governance
Aside from the promising adoption of AI in healthcare, beneath the excitement lies a concern: AI, if left ungoverned, poses significant risks to patient safety and privacy.
From algorithms that could perpetuate hallucinations to biased data that could further inequities, the potential dangers of AI are not just theoretical—they are imminent.
The healthcare industry must address these risks now, as the unchecked deployment of AI systems could lead to life-threatening errors and undermine public trust. This is a critical concern that affects every patient's right to safe, equitable, and effective care.
As healthcare organizations acclimate to the AI landscape, how have their strategies for governance evolved?
Brian Anderson, MD
Co-founder and CEO, Coalition for Health AI (CHAI)
Brian Anderson, MD, co-founder and CEO of the Coalition for Health AI (CHAI), says health systems and hospitals are standing up their own governance committees to handle everything from procurement, configuration, installation, and training through maintenance and management.
And that process begins with deciding whether to work with a vendor or stay in-house.
They’re asking vendors to “share more details around how these models are trained [and] what kinds of patients were these models trained on,” Anderson says. “The methodology around its training and understanding if that model is going to perform well in the specific workflow on the specific kinds of patients that that health system has are really important questions for an AI governance committee.”
They’re also putting more emphasis on model cards, or documents that provide key information about a machine learning module. First proposed in 2018 and established by the Health and Human Services Department’s Office of the National Coordinator for Health IT (ONC) in the HTI-1 Rule in 2023, model cards must address more than 30 categories, including details on indications, limitations, and training data.
“You’re beginning to see health systems demand that kind of information,” he says.
“One of the challenges that I think health systems are going to start facing is the cost associated with this kind of governance.”
—Dane Hudelson, enterprise director of data and analytics, Sanford Health
At Sanford Health, much of the work is done in-house. Dane Hudelson, enterprise director of data & analytics for the South Dakota-based health system, says AI is baked right into Sanford’s enterprise data & analytics department, which launched in 2015 to help the health system manage its automation efforts.
“Once [leadership] realizes that we can do that same functionality in-house without a vendor, we're relatively inexpensive when compared to having somebody come in and do it from a third party,” he said during a recent HealthLeaders podcast. “We're also able to ensure that it is very Sanford-specific, so nothing we do or build is just something you're going to go grab off the shelf. It's intentionally built for Sanford.”
That also works for governance.
“We came to the realization that we have to be very intentional about bringing people along on these projects and these journeys to ensure [that] they're part of the entire build,” he said. “They're not just coming with an idea, but the devil's in a lot of the details, [so] we need them there from conception to delivery and at every stage to make sure that when we move something into a production status that is doing exactly what we intended it to do.”
All of this work, from the technology and governance to data storage, isn’t cheap. Anderson worries about the cost associated with AI adoption and governance.
“One of the challenges that I think health systems are going to start facing is the cost associated with this kind of governance,” Hudelson says. “You’re standing up committees, these cost FTEs, you have potentially to install technical infrastructure with monitoring dashboards.”
Global Healthcare Artificial Intelligence Market Value 2026, by Application
Forecast value of the artificial intelligence healthcare market by application world wide in 2026
SOURCE: AI in Healthcare Statistics 2024 By Pioneering Health Tech (https://scoop.market.us/ai-in-healthcare-statistics/)
And with the healthcare industry struggling to stay in the black, many smaller, rural, and resource-thin health systems and hospitals won’t be able to keep up.
“One of the things that I'm seeing health systems struggle with, particularly lower resource health systems, is how do I participate in the AI revolution and make use of these AI tools that could improve clinical care,” Hudelson points out. “But then once they make that decision to deploy it, not having the resources to be able to monitor it and govern it well and not having the policies in place to answer some of these basic questions, like [how to monitor] for performance degradation. So there's a potential to cut corners, just to save some money”
As an example, Anderson says a healthcare organization could adopt an AI tool and set up a governance committee, then decide to abolish the committee at some point. If that AI tool degrades later on, the organization would still be liable.
“If health systems don't have AI governance committees to manage and mitigate that kind of risk, they're going to become increasingly at risk as these models’ performance degrades over time,” he says. “So they’re going to have to sustain those committees.”
“How do we enable the health systems that don’t have the resources to stand up these governance committee for an indefinite period of time? How do they participate in the AI revolution in a safe way?”
One solution may be open-source software, which CHAI is looking to develop for federally qualified health centers (FQHCs) and other small providers. But Anderson says this could also lead to a new business line for large health systems or vendors: A hub-and-spoke model in which they would handle AI governance for smaller organizations, from hospitals to federally qualified health centers and clinics.
Anderson says he’s surprised at how many healthcare organizations are embracing AI without a clear path to ROI.
“It concerns me that for a sustainable future in AI you need to have … clear economic returns (or) clinical improvements, and in a lot of cases that isn’t happening,” he says. “So the hype is still an issue.”
“It concerns me that for a sustainable future in AI you need to have … clear economic returns (or) clinical improvements, and in a lot of cases that isn’t happening. So the hype is still an issue.”
—Brian Anderson, MD, Co-founder and CEO, Coalition for Health AI (CHAI)
In the long run, however, Anderson expects the industry to come to the realization that the workforce shortage won’t be going away, and as the nation gets older, it will need more healthcare access and services. And AI will play a critical role in addressing supply and demand.
“We can’t hire our way out of the problem right now,” he points out. “We ultimately need to begin grappling with where is the appropriate place where providers aren't in the loop in this, and I think that's going to be one of the real challenges.”
Eric Wicklund, Associate Content Manager and Senior Editor, Innovation and Technology