Methods for Analyzing Quality Measure Performance

Quality of care is compared across accountable entities, including hospitals, provider groups, and insurance plans, using standardized quality measures. However, observed variations in quality measure performance might be the result of chance sampling or measurement errors. Contains functions for estimating the reliability of unadjusted and risk-standardized quality measures.

QualityMeasure: analyzing health care quality measures.

About the package

This R package includes functions for calculating measure performance, both unadjusted and adjusted, and estimating reliability with several different methods.

Several example data sets are also included:

example_df_1: This is a simulated data set generated by simulateData() with 100 accountable entities, an average of 50 observations per entity, a marginal outcome rate of 0.2, a median reliability of 0.7, and a regression coefficient of 0 for the covariate.
example_df_2: This is another simulated data set generated by simulateData() with 100 accountable entities, an average of 50 observations per entity, a marginal outcome rate of 0.2, and a median reliability of 0.7, and a regression coefficient of log(1.5) for the covariate.
colonoscopy: This is a data set aggregated to the entity-level. These data are from the Centers for Medicare and Medicaid Services (CMS). They contain the percentage of patients receiving appropriate recommendation for follow-up screening colonoscopy in calendar year 2023.
psychreadmission: This is a data set aggregated to the entity-level. These data are from the Centers for Medicare and Medicaid Services (CMS) and show the percentage of patients who return to a hospital for an unplanned inpatient stay after discharge. Data are from 07/01/2021 to 06/30/2023.

Please email me at [email protected] with any issues, questions, or feedback! I'd be happy to chat and learn how I can make this package better.

Related publications

Nieser KJ, Harris AHS. Split-sample reliability estimation in health care quality measurement: Once is not enough. Health Serv Res. 2024 Aug;59(4):e14310. doi: 10.1111/1475-6773.14310. https://pubmed.ncbi.nlm.nih.gov/38659301/.
Nieser KJ, Harris AHS. Comparing methods for assessing the reliability of health care quality measures. Stat Med. 2024 Oct 15;43(23):4575-4594. doi: 10.1002/sim.10197. https://pubmed.ncbi.nlm.nih.gov/39145538/.
Nieser KJ, Tancredi DJ, Harris AHS. The unreliability of two publicly reported outcome quality measures for characterizing health care quality within the Veterans Health Administration. Health Serv Res. 2025 Oct 01;e70050. doi: 10.1111/1475-6773.70050. https://pubmed.ncbi.nlm.nih.gov/41035300/.

Installing the package

QualityMeasure version 2.0.2 is now available to download directly from CRAN. The latest version corrects an error that occurs when the estimated variance of the random intercepts is 0 and modifies some of the aesthetic features of the plots.

install.packages('QualityMeasure')
library(QualityMeasure)

Or, download the latest development version of the package from this repo:

library(devtools)
devtools::install_github('knieser/quality_measure_reliability')
library(QualityMeasure)

Vignettes

Check out the tutorial vignette which walks through some examples how this package can be used to estimate entity-level reliability.

There are also two vignettes with examples using publicly available aggregated data sets. Note that because these publicly available data are already aggregated to the entity-level, not all reliability methods are appropriate to use: