Validity of inpatient electronic health record-based measures of oxygen-related therapy in the United States: Lessons applicable for studying COVID-19 endpoints.

INTRODUCTION: During the COVID-19 pandemic, inpatient electronic health records (EHRs) have been used to conduct public health surveillance and assess treatments and outcomes. Invasive mechanical ventilation (MV) and supplemental oxygen (O2 ) use are markers of severe illness in hospitalized COVID-19 patients. In a large US system (n = 142 hospitals), we assessed documentation of MV and O2 use during COVID-19 hospitalization in administrative data versus nursing documentation.

METHODS: We identified 319 553 adult hospitalizations with a COVID-19 diagnosis, February 2020-October 2022, and extracted coded, administrative data for MV or O2 . Separately, we developed classification rules for MV or O2 supplementation from semi-structured nursing documentation. We assessed MV and O2 supplementation in administrative data versus nursing documentation and calculated ordinal endpoints of decreasing COVID-19 disease severity. Nursing documentation was considered the gold standard in sensitivity and positive predictive value (PPV) analyses.

RESULTS: In nursing documentation, the prevalence of MV and O2 supplementation among COVID-19 hospitalizations was 14% and 75%, respectively. The sensitivity of administrative data was 83% for MV and 41% for O2 , with both PPVs above 91%. Concordance between sources was 97% for MV (κ = 0.85), and 54% for O2 (κ = 0.21). For ordinal endpoints, administrative data accurately identified intensive care and MV but underestimated hospitalizations with O2 requirements (42% vs. 18%).

CONCLUSIONS: In comparison to nursing documentation, administrative data under-ascertained O2 supplementation but accurately estimated severe endpoints such as MV. Nursing documentation improved ascertainment of O2 among COVID-19 hospitalizations and can capture oxygen requirements in adults hospitalized with COVID-19 or other respiratory illnesses.