Original Article
CMS program participation and policy evaluation without administrative data: a case study on Bundled Payments for Care Improvement (BPCI) initiative
Abstract
Background: The US spends significantly more per capita on healthcare than other developed countries. The Centers for Medicare and Medicaid Services (CMS) Innovation Center (CMMI) has created various alternative payment models (APMs) that use financial incentives to reward providers for delivering higher value care, including bundled payments. In 2013 and 2018, CMS scaled up its bundled payment APM nationwide through its Bundled Payments for Care Improvement (BPCI) Initiative and BPCI Advanced Initiative, respectively. Studies of the effects of physician group practice (PGPs) participation have been delayed in part due to a lack lists of participating physicians available via CMS.
Methods: To assess whether health policy researchers could adequately evaluate the impact of BPCI without CMS administrative data, we investigated the accuracy of using non-CMS sources to identify BPCI physicians. Our researcher-created database (“Other Data Source List” or “ODSL”) of individual physicians participating in BPCI through a PGP was compared to a novel data set—a list of physicians in PGPs participating in BPCI directly from CMS (“CMS List”). We performed chi-squared tests to determine whether ODSL-identified physicians differed meaningfully from CMS List-identified physicians.
Results: Sixty-two percent of ODSL physicians were found in the CMS List of participating BPCI physicians, and ODSL contained 46% of BPCI physicians identified in the CMS List. ODSL was statistically different from the CMS List and had significant limitations in identifying participating BPCI physicians.
Conclusions: Policy evaluations that rely on identifying physicians using non-CMS sources may have a large degree of inaccuracy. If these challenges extend to other APMs, policy evaluations of such programs using non-CMS sources may also be inaccurate.
Methods: To assess whether health policy researchers could adequately evaluate the impact of BPCI without CMS administrative data, we investigated the accuracy of using non-CMS sources to identify BPCI physicians. Our researcher-created database (“Other Data Source List” or “ODSL”) of individual physicians participating in BPCI through a PGP was compared to a novel data set—a list of physicians in PGPs participating in BPCI directly from CMS (“CMS List”). We performed chi-squared tests to determine whether ODSL-identified physicians differed meaningfully from CMS List-identified physicians.
Results: Sixty-two percent of ODSL physicians were found in the CMS List of participating BPCI physicians, and ODSL contained 46% of BPCI physicians identified in the CMS List. ODSL was statistically different from the CMS List and had significant limitations in identifying participating BPCI physicians.
Conclusions: Policy evaluations that rely on identifying physicians using non-CMS sources may have a large degree of inaccuracy. If these challenges extend to other APMs, policy evaluations of such programs using non-CMS sources may also be inaccurate.
