Machine Learning Prediction of Possible Central Line Associated Blood Stream Infections and Rate of Reduction

Summary

Prospective, multi-center, cluster-randomized trial of a hospital Infection Preventionist (IP)-led quality improvement study to provide clinical teams with just-in-time clinical education and reinforcement of existing best practices recommendations based on the output of a possible Central Line Associated Blood Stream Infection (CLABSI) Machine Learning (ML) prediction model. The objective is to determine whether providing this model to Infection Preventionists will decrease the CLABSI rates versus routine clinical practice.

Detailed Description

Central Line-Associated Bloodstream Infections (CLABSIs) remain a persistent and costly challenge in U.S. hospitals, contributing to increased mortality, prolonged hospital stays, and elevated healthcare costs. In 2022 alone, Providence St. Joseph Health (PSJH) recorded 275 CLABSIs across 430,000 central line days. Despite the implementation of best-practice prevention bundles, these infections continue to occur, prompting the exploration of machine learning (ML) as a tool to predict and mitigate CLABSI risk. While prior studies have demonstrated the predictive potential of ML models-with area under the curve (AUC) values reaching up to 0.87-no randomized trial has yet evaluated the real-world clinical impact of deploying such a model. The primary objective of this trial is to determine whether the deployment of a machine learning model that predicts possible CLABSI risk, when provided to hospital Infection Preventionists (IPs) with a standardized workflow, can reduce CLABSI rates compared to routine practice. Secondary objectives include assessing the intervention's impact on central line removal within 48 hours of an alert, the rate of positive blood cultures, and various process metrics such as the frequency of IP interventions. Safety outcomes, including pneumothorax and hemorrhage, are also being monitored. The study is designed as a prospective, open-label, multi-center, cluster-randomized controlled trial conducted across 20 Providence hospitals with the highest CLABSI burden. These hospitals account for approximately 90% of all CLABSI events within the system during 2023 and 2024. Hospitals were paired using Mahalanobis distance based on the hospital's CLABSI count and NHSN Standardized Infection Ratio (SIR) and then randomized into early and late intervention groups. The early group received access to the ML model for four to five months before the late group. Infection Preventionists at early hospitals used a dashboard to identify high-risk patients and deliver targeted education and interventions focused on central line care. The machine learning model was developed using data from over 62,000 patients and more than 730,000 line-days collected between January 2015 and September 2024. A positive class was defined as a positive blood culture occurring within 24 to 72 hours in a patient with a central line in place for more than 48 hours. From 87 electronic medical record (EMR) data elements, 207 features were engineered for model development. The modeling process employed XGBoost and addressed class imbalance through oversampling, undersampling, and SMOTE techniques. The final model achieved an AUC of 0.93, with a recall of 0.72, precision of 0.66, and an F1 score of 0.68. To ensure fairness, the model underwent a bias analysis using the EEOC's four-fifths rule, confirming consistent performance across race, sex, and ethnicity subgroups. Each day, the model scored all adult inpatients with central lines in place for more than 48 hours. Predictions were published to a PowerBI dashboard accessible to IPs at intervention hospitals. These IPs reviewed flagged patients, ensured adherence to the CLABSI prevention bundle, and recommended line removal when appropriate. The IPs actions were documented in the EMR. The intervention was supported by training, scripting for clinical conversations, and access to infectious disease physicians for consultation. The primary outcome of the trial is the CLABSI rate, defined as events per 1,000 central line-days and adjudicated using NHSN criteria. Secondary outcomes include the proportion of lines removed within 48 hours of a model alert, the rate of positive blood cultures, the rate of possible CLABSIs (defined as a positive culture in a patient with a line in place for more than 48 hours), and the total number of central line days per hospital. Additional metrics include the frequency of IP interventions and before-after comparisons of CLABSI rates. The statistical analysis plan centers on a generalized linear mixed model (GLMM), using either a Poisson or Negative Binomial distribution depending on the presence of overdispersion. The model includes a log of line-days as an offset and incorporates hospital as a random effect to account for clustering. Fixed effects include group assignment and calendar month. Covariates are included to improve precision and control for confounding. Hospital-level covariates include hospital type, medical school affiliation, average length of stay, total bed count, and ICU bed proportion. Patient-level covariates include age, race and ethnicity, primary payer, history of CLABSI, line type, and line location. Sensitivity analyses will explore the additional comorbidities such as immunosuppression, obesity, diabetes, and diarrhea. Inverse Probability of Treatment Weighting (IPTW) will be considered to further adjust for confounding. Sample size calculations were based on a baseline CLABSI rate of 0.004 events per patient per month, with an intra-cluster correlation (ICC) of 0.05 and a targeted 20% relative risk reduction (RR = 0.8). Under these assumptions, approximately 8,920 patients per arm are required to achieve 80% power at a significance level of 0.01. The study duration was set at five months to accrue the necessary 64 CLABSI events. An interim analysis is planned at 2.5 months, using the O'Brien-Fleming group-sequential design to allow for early stopping due to efficacy or harm. The interim analysis will apply a nominal p-value threshold of 0.0088, while the final analysis will use a threshold of 0.0467 to maintain an overall Type I error rate of 5%.

Eligibility Criteria

Interventions