Researchers at Mount Sinai have developed a strategy to use artificial intelligence in health care settings more cost-effectively, potentially reducing expenses up to 17-fold while maintaining performance.
A new study conducted by researchers at the Icahn School of Medicine at Mount Sinai has revealed novel strategies for deploying large language models (LLMs), a form of advanced artificial intelligence (AI), in health care systems. The strategies, detailed in a paper published in npj Digital Medicine, aim to maintain cost efficiency and high performance, potentially transforming how hospitals manage operational tasks.
The research offers valuable insights into leveraging AI tools to streamline various health care tasks, saving time and reducing operational costs while preserving reliability even under significant workloads.
“Our findings provide a road map for health care systems to integrate advanced AI tools to automate tasks efficiently, potentially cutting costs for application programming interface (API) calls for LLMs up to 17-fold and ensuring stable performance under heavy workloads,” co-senior author Girish N. Nadkarni, an Irene and Dr. Arthur M. Fishberg Professor of Medicine at Icahn Mount Sinai and the chief of the Division of Data-Driven and Digital Medicine (D3M) at the Mount Sinai Health System, said in a news release.
Every day, hospitals generate vast amounts of data, making efficient management crucial. LLMs, such as OpenAI’s GPT-4, present promising solutions to automate and enhance workflow efficiency. However, these models come with substantial operational costs, posing a financial challenge for widespread adoption.
The study, motivated by the need to make AI use in health systems more practical, focused on balancing cost reduction with performance maintenance.
“Our study was motivated by the need to find practical ways to reduce costs while maintaining performance so health systems can confidently use LLMs at scale. We set out to ‘stress test’ these models, assessing how well they handle multiple tasks simultaneously, and to pinpoint strategies that keep both performance high and costs manageable,” first author Eyal Klang, a director of the Generative AI Research Program in Mount Sinai’s D3M, said in the news release.
The research team evaluated 10 LLMs using real patient data to determine how they responded to various clinical questions. In over 300,000 experiments, they increased the complexity of tasks to assess the models’ performance under pressure while measuring accuracy and adherence to clinical instructions.
Economic analysis showed that grouping tasks could significantly minimize AI-related costs without compromising performance.
The findings suggest that grouping up to 50 clinical tasks, such as patient screening for clinical trials and review of medication safety, together can allow the models to handle them simultaneously without a significant drop in accuracy.
Such an approach could optimize workflows and reduce API costs up to 17-fold, potentially saving large health systems millions of dollars annually.
“Recognizing the point at which these models begin to struggle under heavy cognitive loads is essential for maintaining reliability and operational stability. Our findings highlight a practical path for integrating generative AI in hospitals and open the door for further investigation of LLMs’ capabilities within real-world limitations,” added Nadkarni.
An unexpected discovery was the models’ tendency to experience unpredictable performance drops when pushed to cognitive limits.
“This research has significant implications for how AI can be integrated into health care systems. Grouping tasks for LLMs not only reduces costs but also conserves resources that can be better directed toward patient care,” added co-author David L. Reich, the chief clinical officer of the Mount Sinai Health System.
Future research will further explore these models’ performance in real-time clinical environments and their interaction with health care teams. The aim is to establish a reliable framework for integrating AI in health systems, balancing efficiency, accuracy and cost-effectiveness to enhance patient care.
The groundbreaking study represents a critical step towards smarter and more economical use of AI in health care, poised to drive innovation and improve patient outcomes.