COLUMBUS, Ohio – A proposed artificial intelligence tool to support clinician decision-making about hospital patients at risk for sepsis has an unusual feature: accounting for its lack of certainty and suggesting what demographic data, vital signs and lab test results it needs to improve its predictive performance.
The system, called SepsisLab, was developed based on feedback from doctors and nurses who treat patients in the emergency departments and ICUs where sepsis, the body's overwhelming response to an infection, is most commonly seen. They reported dissatisfaction with an existing AI-assisted tool that generates a patient risk prediction score using only electronic health records, but no input data from clinicians.
Scientists at The Ohio State University designed SepsisLab to be able to predict a patient's sepsis risk within four hours – but while the clock ticks, the system identifies missing patient information, quantifies how essential it is, and gives a visual picture to clinicians of how specific information will affect the final risk prediction. Experiments using a combination of publicly available and proprietary patient data showed that adding 8% of the recommended data improved the system's sepsis prediction accuracy by 11%.
"The existing model represents a more a traditional human-AI competition paradigm, generating numerous annoying false alarms in ICUs and emergency rooms without listening to clinicians," said senior study author Ping Zhang, associate professor of computer science and engineering and biomedical informatics at Ohio State.
"The idea is we need to involve AI in every intermediate step of decision-making by adopting the 'AI-in-the-human-loop' concept. We're not just developing a tool – we also recruited physicians into the project. This is a real collaboration between computer scientists and clinicians to develop a human-centered system that puts the physician in the driver's seat."
The research was published Aug. 24 in KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining and will be presented orally Wednesday (Aug. 28) at SIGKDD 2024 in Barcelona, Spain.
Sepsis is a life-threatening medical emergency – it can rapidly lead to organ failure – but it's not easy to diagnose because its symptoms of fever, low blood pressure, increasing heart rate and breathing problems can look like a lot of other conditions. This work builds upon a previous machine learning model developed by Zhang and colleagues that estimated the optimal time to give antibiotics to patients with a suspected case of sepsis.
SepsisLab is designed to come up with a risk prediction quickly, but produces a new prediction every hour after new patient data has been added to the system.
"When a patient first comes in, there are many missing values, especially for lab tests," said first author Changchang Yin, a computer science and engineering PhD student in Zhang's Artificial Intelligence in Medicine lab.
In most AI models, missing data points are accounted for with a single assigned value – a process called imputation – "but the imputation model could suffer from uncertainty that can be propagated to the downstream prediction model," Yin said.
"If the imputation model cannot accurately impute the missing value and it's a very important value, the variable should be observed. Our active sensing algorithm aims to find such missing values and tell clinicians what additional variables they might need to observe – variables that can make the prediction model more accurate."
Equally important to removing uncertainty from the system over the passage of time is providing clinicians with actionable recommendations. These include lab tests rank-ordered based on their value to the diagnostic process and estimates of how a patient's sepsis risk would change depending on specific clinical treatments.
Experiments showed adding 8% of the new data from lab tests, vital signs and other high-value variables reduced the propagated uncertainty in the model by 70% – contributing to its 11% improvement in sepsis risk accuracy.
"The algorithm can select the most important variables, and the physician's action reduces the uncertainty," said Zhang, also a core faculty member in Ohio State's Translational Data Analytics Institute. "This fundamental mathematics work is the most important technical innovation – the backbone of the research."
Zhang sees human-centered AI as part of the future of medicine – but only if AI interacts with clinicians in a way that makes them trust the system.
"This is not about building an AI system that can conquer the world," he said. "The center of medicine is hypothesis testing and making decisions minute after minute that are not just 'yes' or 'no.' We envision a person at the center of the interaction using AI to help that human feel superhuman."
This research was supported by the National Science Foundation, the National Institutes of Health and an Ohio State President's Research Excellence Accelerator Grant. Zhang has received additional NIH funding to continue collaborating with clinicians on this work.
Additional co-authors include Jeffrey Caterino of The Ohio State University Wexner Medical Center, Bingsheng Yao and Dakuo Wang of Northeastern University, and Pin-Yu Chen of IBM Research.
#
