Factors associated with participation in the Leapfrog Groups annual survey among acute care hospitals in the U.S. Midwest
Highlight box
Key findings
• If a hospital does not respond to the organizational self-assessment portion of the Leapfrog Group hospital safety survey it is not directly associated with quality-of-care metrics. Responding was found to be associated with larger hospitals most likely because they have the necessary resources and corporate support. Also, hospitals that participate in Medicare Shared Savings Program are more likely to respond to the survey request. This could be due of their effort to adopt reporting practices which in turn may be increasing the likelihood to participate in voluntary surveys.
What is known and what is new?
• It is known that there is no difference in inpatient mortality rates and Leapfrog safety scores. Organizations use Leapfrog scores to market their hospital quality and patient experience as it is associated with higher safety and quality. Responding to the Leapfrog self-assessment may skew the scores in a favorable manner for some hospitals.
• This research adds that whether a hospital completes the organizational self-assessment there is not a significant difference in most quality outcomes. The survey responses are dictated on the size, capabilities, and motivations of each hospital.
What is the implication, and what should change now?
• The push for transparency in hospital processes and best practices is not indicative of the care provided. Though it does provide information for potential patients, it is not associated with care provided. It should be evaluated further to determine if the emphasis on this portion of the scoring should hold the weighting that it currently does.
Introduction
Background
Reducing harm and keeping patients safe from medical errors has been one of the significant initiatives of healthcare leaders over the last few decades. This, along with improving outcomes, decreasing costs, and improving patient satisfaction, are constantly on the minds of those working in healthcare today. The focus on quality and safety began to take shape with the landmark study of “To Err is Human”, asserting that 44,000 to 98,000 people die each year, attributing to massive hospital costs due to errors, inadequate processes, and a lack of organizational culture (1). Hospital practices and processes are evaluated through accreditation and other bodies that review workflows, adherence to standards, and the ability to improve patient outcomes continually (2). Businesses and consumers are becoming more attuned to the fact that safety and outcomes are not in alignment with the costs of healthcare in the United States (3). As the United States continues to slide down the scale of value in healthcare spending and outcomes, more external organizations will be involved in providing perspective on the care that is provided (4).
Congress finally acted on the healthcare industry through the Affordable Care Act (ACA) in 2010, which promotes incentives through multiple quality programs for high-performing hospitals based on various quality and safety measures (5). Before the ACA passed Congress, groups of leaders in multiple industries started to evaluate the costs being shifted to organizations that were not translating to significant savings or human capital advantages. To get ahead of the healthcare spending and outcome disparities, a group of business leaders in 1998 felt that costs, outcomes, and safety were not meeting their needs and outlined a process to assess hospitals to ensure they provide safe and effective care (6). This led to the development of the Leapfrog Hospital Survey, launched in 2001, which asks hospitals to voluntarily submit data on care processes, administrative policies, and their organization’s safety culture (7). The composite score approach was developed to make overall rankings easy for the public to understand and provide a singular overall assessment (8). Today, multiple organizations rate hospitals based on various publicly reported outcomes and anecdotal input from industry insiders. A recent Mayo Clinic editorial found that of the top-rated ranking organizations, only the Leapfrog group included a hospital’s self-assessment as part of the scoring methodology (9). As hospitals start to differentiate by marketing and promoting quality, safety, and patient satisfaction, they utilize these external organizational assessments to create an unbiased opinion of the care they provide.
Rationale & knowledge gap
Several studies have looked at the associations between Leapfrog Group safety scores and other designations held by hospitals. Research has also evaluated outcomes, safety scores, and self-reported assessments that were deemed to be favorable. Kernisan et al. found no significant difference in inpatient mortality rates and the Leapfrog safety score (7). Pakyz et al. evaluated the comparisons of Leapfrog composite scores and Magnet-designated hospitals, those deemed exemplary by nursing standards, and found these two attributes were associated with better quality and safety, the evidence was inconsistent (10). Lastly, Smith et al. found that Leapfrog self-assessments were positively skewed compared to the publicly reported hospital evaluation outcomes (11). This leads one to believe that the overall quality of care provided by the highest-performing hospitals may not depend on a hospital’s self-assessment compared to publicly reported measures by Center for Medicare & Medicaid Services (CMS) or other entities. The hospital self-assessment is of value as it forces hospital leadership to accurately account for the implementation and adherence to best practices.
Completing the Leapfrog survey requires resources including personnel, expertise, software, and, of course, time. Hospitals involve in the process, quality and safety professionals, who are expected to work with information technology (IT) staff to extract data. Hospitals may also need to have in place data analytics expertise and software to streamline the data collection and analysis. The main data sources include electronic medical record (EMR) data, infection control and other patient safety reports, computerized physician order entry (CPOE) data, surgical and ICU staffing metrics, billing records. The process can take over 40 hours, because it involves data collection, validation, and submission through Leapfrog’s platform. In some cases, follow-ups or audits could be expected. Completing the survey allows hospitals to compare their safety practices and initiatives to other organizations as well as it reinforces that not only are they consistently working to improve but have maintained a base level of interventions to improve safety and quality.
Objective
While there have been several research studies about Leapfrog survey scores and quality of care, there has been no published studies so far, to the authors’ knowledge, that examined whether responding to the survey alone, or not, holds implications about the quality of care and patient safety. This has been the motivation for the present study. The purpose of this research is, therefore, to determine if there is a difference in the publicly reported quality and safety outcomes of hospitals that responded to the Leapfrog hospital survey, compared to those that did not respond. As hospital leaders continually deal with shrinking margins and questions on how to best allocate time, the study was also designed to examine if responding to this comprehensive survey offer any benefit outside of publicly available data. We present this article in accordance with the STROBE reporting checklist (available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-25-14/rc).
Methods
This is cross-sectional research which was conducted with the use of four databases that were combined for the purpose of this study. First off, hospital structure information was extracted from the American Hospital Association (AHA) database, for the year 2018. This database includes hospital structure and hospital characteristics information such as the hospital size and staffing, availability of services, teaching status, accreditation status and participation in CMS programs, such as in Medicare Shared Savings Programs (MSSPs). AHA collects demographic and other information annually from hospitals and healthcare organizations throughout the United States through a voluntary survey process (12). In our study specifically, AHA data were utilized to identify the hospital’s accrediting agency, number of beds, medical and surgical volumes, full-time equivalent positions (FTEs), critical care capacity, non-acute care services, hospital system membership, and various other demographics.
The second data source was CMS’s Hospital Compare database. More specifically, the study leveraged publicly reported measures through the Hospital Compare portal (www.medicare.gov/care-compare). Hospital performance is measured by the CMS quality-based payment programs since the implementation of the ACA: Hospital Value Based Purchasing (HVBP), Hospital Readmission Reduction Program (HRRP), and Hospital Acquired Conditions Reduction Program (HACRP) which utilizes Hospital Acquired Infections (HAI) measures from the Agency on Research and Quality (AHRQ). The mortality data utilizes a three-year time frame that ended in June of 2018 and is taken from the Complications and Deaths dataset on Hospital Compare. The data set was further filtered down to only include the six mortality measures that are aggregated by CMS along with hospital demographic data. Mortality data is compiled by CMS using Medicare claims data to calculate a hospital’s mortality rate (13). The rate is the risk adjusted to account for a patient’s age and prior medical history based on diagnosis coding contained in their claims data. The patient safety measures in the HAI data set were then filtered to only include the HAI Standardized Infection Rate (SIR) data only. Healthcare infection data is captured by the Center for Disease Control (CDC) through the National Healthcare Safety Network (NHSN) data collection protocols (14). The HAI SIR rate is also a risk adjusted measure that factors in hospital information and patient demographics.
The third data source is a MedPar Limited dataset which contains all Medicare patient admissions that occurred during the calendar year of 2017 (https://www.cms.gov/data-research/files-for-order/limited-data-set-lds-files/medpar-limited-data-set-lds-hospital-national). The dataset’s original unit of analysis was the hospital admission, and the dataset was aggregated at the hospital level to be used for the purpose of the study. The following hospital outcomes were extracted from this dataset after aggregation: percentage of patients who developed septicemia, overall hospital mortality rate, mean length of stay (LOS), age and sex distribution, admission type (emergency/urgent elective) distribution.
Finally, the fourth dataset included information on whether each hospital responded or did not respond to the Leapfrog Hospital Survey of fall 2024. The hospitals were identified utilizing the Leapfrog website’s hospital reporting indicator (15). A list of all hospitals in the states of Michigan, Ohio, Indiana, Illinois, and Wisconsin were downloaded from the Medicare Inpatient Hospital Lookup database (16). A manual review was then conducted utilizing the hospital name and address to determine per Leapfrog if the hospital had not responded to the Leapfrog survey request. For each hospital the Medicare ID was added to allow for the merging with the other databases that the present study utilized. These four databases all included the unique Medicare payment identification number. The Medicare ID was used to match hospital outcome data in the Hospital Compare data files to with the hospital structure information in the AHA dataset, the Leapfrog survey response dataset and the CMS Limited Data Set (LDS) aggregated dataset. The datasets were merged utilizing the Medicare ID as the primary key to create a comprehensive dataset for analysis. After pairwise eliminations a total of 436 hospitals were included in the final target dataset (Figure 1).
Inclusion-exclusion criteria
The study examined hospitals from the following five Midwestern states located in the (US): Michigan, Ohio, Indiana, Illinois, and Wisconsin. All other US hospitals were excluded from the analysis. The US Census Bureau as well as the Center for Disease Control and Prevention (CDC), organize these states into the Northeast Central Region1, and this classification criterion was used for the purpose of this study. Hospitals with missing information on whether they completed the Leapfrog survey or not, were also excluded from the analysis. The Leapfrog Survey does not contain Critical Access Hospitals (CAHs), and therefore CAHs were pairwise excluded from all other files that were used in the present study. Only inpatient acute care medical settings were included in the analysis. Leapfrog distributes the survey to medical and surgical settings; all surgical settings were excluded. After filtering out the non-included States, the hospitals with missing response/non-response information, and surgical hospitals, a total of 446 hospitals formed the final target dataset.
Statistical analysis
Chi-square statistics were used to examine associations between the categorical variables and the response/no-response dichotomous variable. Independent samples t-test was used to examine if continuous variables have a different mean value between the responding and non-responding hospitals. The dichotomous response/non-response variable was then inserted into five multiple linear regression models, alongside with control variables such as hospital size, teaching status, MSSP status, partnership with insurer status. Each model was created to examine the association with the response/non-response variable with each of the five following outcomes: LOS, overall hospital rating, 30-day acute myocardial infarction (AMI) mortality, 30-day stroke mortality, % developed in-patient septicemias. For the multivariate analysis, multicollinearity between the independent variables was assessed using correlation matrices and calculating the variance inflation factor (VIF) for each variable. No value of VIF >5 was detected; and no pairs of variables were found to be correlated: An r<0.4 was observed between all pairs of the independent variables.
All statistical analyses were conducted using SPSS version 29. A 95% confidence level (α=0.05) was used to determine statistical significance, and all tests were two-tailed.
Results
A total of 441 hospitals were included in the analysis. Of these, 61.9% (N=273) completed and returned the Leapfrog survey, while 38.1% (N=168) did not respond to the survey request. Independent samples t-tests were conducted to compare continuous hospital characteristics between responding and non-responding hospitals. Results indicated that responding hospitals had a significantly higher number of total facility personnel (t=2.75, P=0.03), registered nurses (t=3.39, P<0.001), patient admissions (t=4.59, P<0.001), average inpatient days (t=4.29, P<0.001), and hospital beds (t=4.70, P<0.001). These findings suggest that larger hospitals, characterized by more beds, staff, and patient admissions, were more likely to respond to the Leapfrog survey.
Chi-squared analyses were conducted to examine associations between categorical hospital characteristics and survey response status. The analyses revealed positive and statistically significant associations between responding to the survey and the following characteristics: teaching hospital status (χ2=4.86, P=0.03), Catholic-operated status (χ2=3.68, P=0.06), enrollment in the MSSP Track 3 (χ2=9.15, P=0.002) or Track 1+ (χ2=6.08, P=0.01), provision of physical and rehabilitation services (χ2=5.76, P=0.02), and membership in a hospital-established clinically integrated network (CLIN) (χ2=7.86, P=0.005). Conversely, hospitals with Medicare NextGen status were less likely to respond to the survey (χ2=12.32, P<0.001). These results highlight that specific hospital characteristics, particularly size, operational focus, and participation in certain Medicare programs, influence the likelihood of responding to the Leapfrog survey. Tables 1,2 summarize differences to hospital characteristics between responder vs non-responder hospitals.
Table 1
| Hospital characteristic | Responded | N | Mean | Median | SD | t-score | P value | 95% CI Diff. |
|---|---|---|---|---|---|---|---|---|
| Total facility personnel FTE | Yes | 219 | 2,098.56 | 1,231.00 | 2,950.44 | 2.75 | 0.003 | 235.73–1,407.64 |
| No | 116 | 1,276.87 | 787.00 | 1,726.85 | ||||
| Total personnel FTEC | Yes | 272 | 1,920.52 | 1,159.00 | 2,588.57 | 3.50 | <0.001 | 337.91–1,200.10 |
| No | 164 | 1,151.51 | 791.50 | 1,401.73 | ||||
| Registered nurses FTE | Yes | 219 | 621.26 | 377.00 | 775.76 | 3.39 | <0.001 | 112.69–422.91 |
| No | 112 | 353.46 | 203.00 | 428.46 | ||||
| Registered nurses FTEC | Yes | 272 | 598.74 | 364.50 | 735.32 | 4.07 | <0.001 | 131.78–376.85 |
| No | 164 | 344.43 | 226.50 | 399.82 | ||||
| Admissions | Yes | 272 | 12,168.35 | 8,750.50 | 11,032.22 | 4.59 | <0.001 | 2,584.11–6,443.40 |
| No | 164 | 7,654.59 | 5,202.50 | 7,760.62 | ||||
| In-patient days | Yes | 272 | 58,565.25 | 38,486.00 | 61,072.97 | 4.29 | <0.001 | 12,596.77–33,849.46 |
| No | 164 | 35,342.13 | 24,671.00 | 41,972.66 | ||||
| Total hospital beds | Yes | 272 | 260.63 | 199.00 | 220.50 | 4.70 | <0.001 | 53.88–131.12 |
| No | 164 | 168.13 | 128.50 | 156.11 |
CI, confidence interval; Diff., difference; FTE, full time equivalents; FTEC, full time equivalents clinical; SD, standard deviation.
Table 2
| Hospital characteristic | Responded | Responders | Non-responders | Chi-squared | P value |
|---|---|---|---|---|---|
| Joint Commission accredited | Yes | 215 (62.3) | 130 (37.7) | <0.01 | 0.96 |
| No | 57 (62.6) | 34 (37.4) | |||
| GME recognized | Yes | 157 (66.2) | 80 (33.8) | 3.29 | 0.07 |
| No | 115 (57.8) | 84 (42.2) | |||
| CARF accredited | Yes | 45 (70.3) | 19 (29.7) | 2.01 | 0.16 |
| No | 227 (61.0) | 145 (39.0) | |||
| DNV accredited | Yes | 16 (66.7) | 8 (33.3) | 0.19 | 0.67 |
| No | 256 (62.1) | 156 (37.9) | |||
| Council of Teaching Hospital Member | Yes | 30 (78.9) | 8 (21.1) | 4.86 | 0.03 |
| No | 242 (60.8) | 156 (39.2) | |||
| Catholic church operated | Yes | 79 (69.9) | 34 (30.1) | 3.68 | 0.06 |
| No | 193 (59.8) | 130 (40.2) | |||
| Traditional Medicare MSSP Track 3 | Yes | 21 (95.5) | 1 (4.5) | 9.15 | 0.002 |
| No | 61 (62.2) | 37 (37.8) | |||
| Traditional Medicare MSSP Track 1+ | Yes | 16 (94.1) | 1 (5.9) | 6.08 | 0.01 |
| No | 66 (64.1) | 37 (35.9) | |||
| Rural referral center | Yes | 42 (68.9) | 19 (31.1) | 1.26 | 0.26 |
| No | 230 (61.3) | 145 (38.7) | |||
| Sole community provider | Yes | 16 (50.0) | 16 (50.0) | 2.25 | 0.13 |
| No | 256 (63.4) | 148 (36.6) | |||
| Partnership with insurer | Yes | 53 (74.6) | 18 (25.4) | 2.96 | 0.09 |
| No | 178 (63.8) | 101 (36.2) | |||
| Traditional Medicare NextGen | Yes | 7 (35.0) | 13 (65.0) | 12.32 | <0.001 |
| No | 75 (75.0) | 25 (25.0) | |||
| Physical/rehabilitation care available | Yes | 107 (69.9) | 46 (30.1) | 5.76 | 0.02 |
| No | 136 (57.9) | 99 (42.1) | |||
| Hospital-established clinical integrated network | Yes | 109 (75.2) | 36 (24.8) | 7.86 | 0.005 |
| No | 119 (60.7) | 77 (39.3) |
Data are presented as n (%). CARF, Commission on Accreditation of Rehabilitation Facilities; DNV, Det Norske Veritas; GME, Graduate Medical Education; MSSP, Medicare Shared Savings Program.
Patient characteristics and care outcomes were also analyzed. The age and sex distribution did not differ significantly between hospitals that responded to the survey and those that did not. Regarding patient outcomes, hospitals that responded to the survey had a mean LOS that was 0.35 days longer (t=4.16, P<0.001). Additionally, responding hospitals reported higher rates of in-patient septicemia (t=3.31, P=0.001), as well as higher 30-day mortality rates for AMI (t=3.82, P<0.001) and stroke (t=2.68, P=0.007). Mean hospital charges were also significantly higher in responding hospitals, with an average increase of $3,673.84 per patient (t=2.15, P=0.03). However, no statistically significant differences were identified in patient experience survey dimensions between responding and non-responding hospitals (Table 3). These findings suggest that while certain clinical outcomes and financial metrics differ, patient experiences remain comparable between hospitals based on survey response status.
Table 3
| Variable name | Responded | Mean | SD | t-score | P value | 95% CI Diff. |
|---|---|---|---|---|---|---|
| Patient demographics & admission characteristics | ||||||
| Age group mean | Yes | 3.404 | 3.6 | −0.33 | 0.74 | −0.092 to 0.065 |
| No | 3.417 | 4.3 | ||||
| % of female patients | Yes | 55.1 | 3.7 | −1.41 | 0.16 | −0.014 to 0.002 |
| No | 55.7 | 4.4 | ||||
| % transferred from another hospital | Yes | 4.4 | 7.9 | 0.30 | 0.76 | −0.011 to 0.016 |
| No | 4.1 | 6.8 | ||||
| % of elective admissions | Yes | 17.3 | 12.4 | −1.43 | 0.15 | −0.048 to 0.007 |
| No | 19.3 | 15.6 | ||||
| Hospital charges and payments | ||||||
| Mean total charges ($) | Yes | 39,906.09 | 16,881.84 | 2.15 | 0.03 | 317.21 to 7,030.47 |
| No | 36,232.25 | 17,484.22 | ||||
| Mean claim payment ($) | Yes | 9,467.84 | 2,511.62 | 1.95 | 0.05 | −3.11 to 910.54 |
| No | 9,014.12 | 2,246.83 | ||||
| Outcomes of care | ||||||
| Mean length of stay (days) | Yes | 4.326 | 0.832 | 4.16 | <0.001 | 0.182 to 0.508 |
| No | 3.981 | 0.841 | ||||
| % of patients discharged dead | Yes | 2.5 | 0.9 | −0.21 | 0.83 | −0.002 to 0.001 |
| No | 2.5 | 1.0 | ||||
| % developed hospital septicemia | Yes | 0.7 | 0.4 | 3.31 | 0.001 | 0.000 to 0.002 |
| No | 0.5 | 0.3 | ||||
| 30-day mortality: AMI | Yes | 10.563 | 4.699 | 3.82 | <0.001 | 0.9675 to 3.0126 |
| No | 8.573 | 6.083 | ||||
| 30-day mortality: CABG | Yes | 1.284 | 1.537 | 1.61 | 0.11 | −0.0547 to 0.5528 |
| No | 1.035 | 1.605 | ||||
| 30-day mortality: COPD | Yes | 8.173 | 1.547 | 0.47 | 0.64 | −0.2672 to 0.4347 |
| No | 8.089 | 1.941 | ||||
| 30-day mortality: heart failure | Yes | 11.086 | 2.300 | −1.73 | 0.08 | −0.9347 to 0.0588 |
| No | 11.524 | 2.694 | ||||
| 30-day mortality: pneumonia | Yes | 15.189 | 2.825 | −0.59 | 0.55 | −0.7790 to 0.4174 |
| No | 15.370 | 3.216 | ||||
| 30-day mortality: stroke | Yes | 12.365 | 4.258 | 2.68 | 0.007 | 0.3441 to 2.2141 |
| No | 11.086 | 5.612 | ||||
| Patient experiences | ||||||
| Care transition | Yes | 81.00 | 8.911 | 1.42 | 0.15 | −0.632 to 3.990 |
| No | 79.32 | 15.642 | ||||
| Cleanliness | Yes | 86.65 | 9.778 | 1.08 | 0.28 | −1.125 to 3.914 |
| No | 85.26 | 16.988 | ||||
| Communication about medications | Yes | 77.39 | 8.941 | 0.58 | 0.56 | −1.600 to 2.970 |
| No | 76.70 | 15.340 | ||||
| Discharge information | Yes | 86.68 | 9.789 | 1.14 | 0.25 | −1.046 to 3.967 |
| No | 85.22 | 16.846 | ||||
| Communication with doctors | Yes | 90.22 | 9.725 | 1.39 | 0.16 | −0.739 to 4.363 |
| No | 88.41 | 17.371 | ||||
| Communication with nurses | Yes | 90.74 | 9.846 | 1.40 | 0.16 | −0.738 to 4.406 |
| No | 88.91 | 17.476 | ||||
| Overall hospital rating | Yes | 87.48 | 9.761 | 1.49 | 0.14 | −0.606 to 4.422 |
| No | 85.57 | 16.950 | ||||
| Quietness | Yes | 80.60 | 9.357 | 0.46 | 0.65 | −1.832 to 2.952 |
| No | 80.04 | 16.058 | ||||
| Would recommend hospital | Yes | 86.83 | 10.160 | 1.47 | 0.14 | −0.645 to 4.473 |
| No | 84.92 | 17.033 | ||||
| Staff responsiveness | Yes | 84.63 | 9.647 | 0.95 | 0.34 | −1.267 to 3.678 |
| No | 83.43 | 16.627 | ||||
AMI, acute myocardial infraction; CABG, coronary artery bypass graft; CI, confidence interval; COPD, chronic obstructive pulmonary disease; Diff., difference; SD, standard deviation.
Multiple linear regression analyses were conducted to examine the association between survey response status and five selected outcomes, controlling for hospital size, services, and other characteristics. The results indicated no significant association between survey response status and LOS, 30-day AMI mortality, 30-day stroke mortality, or the percentage of patients who developed in-patient septicemia. Key predictors for the selected outcomes were as follows:
- LOS: the most significant predictor was hospital bed size (β=0.002, P<0.001): for every additional hospital bed, the LOS increases by 0.002 days. This means that hospitals with 100 more beds have inpatient stays that are approximately 0.2 days longer.
- Overall hospital rating: physical/rehabilitation services (β=−1.10, P=0.02) and partnership with an insurer (β=1.17, P=0.02) were significant predictors: hospitals that offer physical or rehabilitation services tend to have overall rating scores that are 1.10 units lower, compared to hospitals that do not offer these services. In contrast, hospitals that have a partnership with an insurer report overall rating scores that are 1.17 units higher, compared to hospitals without a partnership with an insurer.
- 30-day AMI mortality: Medicare MSSP Track 3 status (β=3.22, P=0.006) Hospitals of MSSP Track 3 status have higher 30-Day AMI mortality rates than hospitals that do not have MSSP Track 3 status.
- Percentage of in-patient septicemia: graduate medical education (GME) recognized hospitals were found to be associated with increase rates of inpatient septicemias (β=5.58, P<0.001), see Table 4.
Table 4
| Variable name | Multiple linear regression models | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1: LOS (r2=0.646) | 2: overall hospital rating (r2=0.235) | 3: 30-day AMI mortality† (r2=0.222) | 4: 30-day stroke mortality† (r2=0.178) | 5: % in-patient septicemias† (r2=0.332) | ||||||||||
| β | P | β | P | β | P | β | P | β | P | |||||
| Did not respond to the Leapfrog survey | −0.041 | 0.76 | −0.85 | 0.12 | 0.04 | 0.18 | 0.051 | 0.24 | 0.064 | 0.79 | ||||
| Total hospital beds | <0.05 | <0.001 | <0.01 | 0.41 | <0.01 | 0.12 | <0.01 | 0.16 | <0.01 | 0.06 | ||||
| GME recognized | 0.18 | 0.12 | −1.27 | 0.01 | 0.02 | 0.41 | −0.02 | 0.59 | 0.55 | <0.001 | ||||
| Teaching hospital | −0.16 | 0.44 | 0.22 | 0.80 | <−0.01 | 0.95 | 0.08 | 0.22 | 0.34 | 0.35 | ||||
| Catholic operated | −0.13 | 0.29 | 0.33 | 0.53 | 0.04 | 0.19 | 0.07 | 0.10 | 0.07 | 0.74 | ||||
| Medicare MSSP Track 3 | 0.01 | 0.95 | 0.57 | 0.41 | 0.11 | 0.006 | 0.07 | 0.16 | −0.11 | 0.70 | ||||
| Medicare MSSP Track 1+ | 0.07 | 0.67 | −0.02 | 0.97 | −0.05 | 0.19 | −0.02 | 0.63 | 0.10 | 0.71 | ||||
| Medicare NextGen | 0.04 | 0.78 | 0.65 | 0.30 | 0.02 | 0.60 | 0.09 | 0.054 | <−0.01 | 0.99 | ||||
| Physical rehab. care | 0.03 | 0.78 | −1.10 | 0.02 | −0.01 | 0.79 | 0.06 | 0.08 | 0.07 | 0.70 | ||||
| Hospital integrated network | 0.06 | 0.59 | 0.25 | 0.61 | 0.04 | 0.10 | 0.04 | 0.26 | 0.10 | 0.62 | ||||
| Partnership with insurer | −0.09 | 0.45 | 1.17 | 0.02 | <−0.01 | 0.95 | 0.04 | 0.51 | −0.27 | 0.21 | ||||
| Constant | 3.52 | <0.001 | 89.31 | <0.001 | 3.58 | <0.001 | 3.70 | <0.001 | −8.08 | <0.001 | ||||
†, logit transformed. AMI, acute myocardial infraction; GME, Graduate Medical Education; LOS, length of stay; MSSP, Medicare Shared Savings Program.
Discussion
The findings indicate that it is the hospital structural characteristics that influence the likelihood for responding to the Leapfrog Hospital Survey. While responding was found to be associated with diagnosis-specific mortality rates, hospital LOS, and inpatient septicemia rates, during bivariate analysis, these outcomes were not significantly associated anymore with the “response” variable, after controlling for hospital-specific structure characteristics such as size, staffing, teaching status and more.
Larger inpatient hospitals may have a more suitable infrastructure (databases, and planning and data access) to respond to the Leapfrog Group’s voluntary survey because they have the necessary resources and corporate support. Typically, large hospitals are part of a health system with more readily available resources. Apparently, for a hospital to participate in the Leapfrog assessment it involves not only commitment in terms of manpower and time, but also the expertise and infrastructure to extract data and merge it from multiple databases. The process, in general, requires multiple leaders, health analysts, and IT professionals to commit to reviewing and respond to the survey. Characteristically, a study conducted by the Oklahoma State University Center for Health Systems Innovation found that completing just the one section of the self-reported assessment took 12 FTEs more than 100 hours of work. Another assumption is, that, for hospitals to be incentivized to complete the survey, their leaders would want to be confident that their hospital would receive a good grade from Leapfrog. In reality, and according to the findings of the Oklahoma State University study, achieving and maintaining an “A” grade would require substantial investment: an additional $325,000 in FTE costs would be needed just to monitor and adhere to the standards (17). Maintaining quality care is one thing, but having the capacity to produce and maintain databases and analytics for the hospital performance and quality involves additional commitment, resources, and expertise.
Investing resources and energy in resource allocation, engaging hospital administrators, managers, and health providers themselves, is often not sustainable for smaller hospitals, particularly those operating on tight financial margins or limited staff. The labor-intensive nature of completing such a comprehensive survey and implementing resources for long-term adherence can be a significant barrier for smaller or resource-constrained healthcare organizations. Since larger hospitals often possess the requisite personnel resources, receive more executive support, and have available data infrastructure, they can invest in survey engagement. This finding is crucial for the Leapfrog Group to explore strategies for enhancing the survey’s accessibility, reducing its burden on smaller hospitals in the US. The authors believe that to reduce its burden on smaller hospitals, the Leapfrog Group could consider eliminating redundant questions (or sections) and offer a tiered participation model. Financial support, such as grant programs, and fee waivers, as well as partnerships with state hospital associations, could encourage participation. Providing incentives—such as special recognition for smaller hospitals or benchmarking opportunities could make participation more appealing. In addition, to make the survey more accessible for smaller hospitals, the process could be streamlined by reducing mandatory measures and focusing on foundational quality indicators relevant to hospital size and resources. In addition, providing technical assistance and guidance would help hospitals navigate the survey, and, whenever possible automating data extraction from EMRs or integrating existing CMS quality reporting data could reduce data entry burden.
According to the results of the present study, hospitals participating in MSSP are more likely to engage with the Leapfrog survey. MSSP is part of Accountable Care Organizations (ACOs). ACOs were designed to improve care delivery and in specific improve patient care quality, reduce costs, and reduce medical errors. While the study did not find statistically significant associations between MSSP status and any of the clinical and quality outcomes, it did find that MSSP hospitals are more likely to respond to the survey request. While there is literature evidence that MSSP participating hospitals are likely to be more invested in prioritizing improving their quality and patient safety metrics to realize higher payments from participating health plans (18), this was not found to directly translate to improved outcomes; it would be safer to conclude, though, that their effort to adopt quality reporting practices may in turn be increasing the likelihood to participate in voluntary surveys. In addition, ACOs are frequently under pressure to demonstrate high levels of quality. Therefore, participating in requests from external parties like Leapfrog’s safety survey is a logical choice to reinforce their commitment to patient care.
CLINs are networks of healthcare providers who come together to negotiate more favorable reimbursement rates with third party payors, are also more likely to respond to the Leapfrog survey. CLINs typically collaborate to improve their bargaining power and streamline operations, including reducing supply and contract costs. For these networks, responding to the Leapfrog survey is their way to make known their commitment to safe care and operational efficiencies. These are essential to securing more favorable, for the hospital, reimbursement rates from third party payors. Moreover, many ACOs operate as CLINs or have joined together to form one, further aligning the incentives to participate in voluntary assessments like those offered by Leapfrog.
The authors of this study believe that hospitals ignoring the Leapfrog survey may hold implications for hospitals themselves. As already discussed, non-responding hospitals miss an opportunity to produce evidence of their quality care and operational efficiencies, which is turn would help them in negotiating more favorable reimbursement rates. Participation in the Leapfrog survey and receiving from Leapfrog an “A” grade, can also be used as a marketing tool. Hospitals that excel can disseminate with stakeholders, patient groups and the community; evidence-based information that proved their commitment to high quality care. Another dimension is, that, achieving an “A” grade is itself a recognition for the efficiency, professionalism and technical excellence of their health providers, therefore rewarding them for their contributions to patient safety and care quality. This recognition can be a motivating factor for health providers and in line with a systems thinking approach about quality (19).
For external quality survey groups themselves, such as, in our case, Leapfrog, not receiving responses from hospitals reduces the amount of valuable data and the completeness of data, that could help understand factors influencing quality of care and patient safety. Without this data, Leapfrog cannot always provide to its full capacity information to the public about quality and patient safety, to making informed decisions about where to seek care. This, in turn, affects the organization’s ability to provide meaningful insights to the public, undermining its role as a key resource for healthcare decision-making.
The authors recognize a few data and design limitations. First, there are differences to the data collection methodology across the four data sources that the study utilized. Also, the year data were collected in each of our four data sources was not the same. The authors believe that the temporal difference across the four files that the study used, though, is not substantial enough to warrant significant changes to the clinical quality and services offered across hospitals. The study did not account for the patient case-mix and different distributions of comorbidities across the patient population, which could, in some instances have further explained differences to the outcomes that were examined in the regression analysis. This was beyond the scope of the study though. Finally, for several hospitals, information on whether they responded to the survey was not available and these hospitals were apparently excluded from the analysis.
Conclusions
If a hospital does not respond to the Leapfrog assessment survey it is not directly associated with their quality-of-care metrics, but it is likely to be associated with infrastructure capacity (personnel resources and data infrastructure) which makes it more likely to invest in survey engagement. These hospitals prioritize the impact receiving a favorable Leapfrog grade has on promoting their quality and patient safety performance and organizational commitments to improve patient outcomes in the future.
Acknowledgments
None.
Footnote
Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-25-14/rc
Data Sharing Statement: Available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-25-14/dss
Peer Review File: Available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-25-14/prf
Funding: None.
Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-25-14/coif). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
1https://www.cdc.gov/nchs/hus/sources-definitions/geographic-region.htm.
References
- Kohn LT, Corrigan J, Donaldson MS. To err is human: Building a safer health system. Washington: National Academy Press; 1999. Available online:
https://www.ncbi.nlm.nih.gov/books/NBK225182/ - Centers for Medicare and Medicaid Services (CMS). Conditions for Coverage (CfCs) & Conditions of Participations (CoPs) | CMS. www.cms.gov. 2020. Available online: https://www.cms.gov/Regulations-and-Guidance/Legislation/CFCsAndCoPs
- Hussey PS, Wertheimer S, Mehrotra A. The association between health care quality and cost: a systematic review. Ann Intern Med 2013;158:27-34. [Crossref] [PubMed]
- Papanicolas I, Woskie LR, Jha AK. Health Care Spending in the United States and Other High-Income Countries. JAMA 2018;319:1024-39. [Crossref] [PubMed]
- U.S. Department of Health and Human Services. About the Affordable Care Act (ACA). U.S. Department of Health & Human Services. 2022. Available online: https://www.hhs.gov/healthcare/about-the-aca/index.html
- History. Leapfrog. 2015. Available online: https://www.leapfroggroup.org/about/history
- Kernisan LP, Lee SJ, Boscardin WJ, et al. Association between hospital-reported Leapfrog Safe Practices Scores and inpatient mortality. JAMA 2009;301:1341-8. [Crossref] [PubMed]
- Austin JM, D'Andrea G, Birkmeyer JD, et al. Safety in numbers: the development of Leapfrog's composite patient safety score for U.S. hospitals. J Patient Saf 2014;10:64-71. [Crossref] [PubMed]
- 5 tips for wading through hospital quality rankings, ratings. Mayo Clinic Health System. 2022 [cited 2025 Jan 22]. Available online: https://www.mayoclinichealthsystem.org/hometown-health/our-thoughts/wading-through-hospital-quality-rankings
- Pakyz AL, Wang H, Ozcan YA, et al. Leapfrog Hospital Safety Score, Magnet Designation, and Healthcare-Associated Infections in United States Hospitals. J Patient Saf 2021;17:445-50. [Crossref] [PubMed]
- Smith SN, Reichert HA, Ameling JM, et al. Dissecting Leapfrog: How Well Do Leapfrog Safe Practices Scores Correlate With Hospital Compare Ratings and Penalties, and How Much Do They Matter? Med Care 2017;55:606-14. [Crossref] [PubMed]
- American Hospital Association (AHA) Why AHA Data: AHA Data. Available online: https://www.ahadata.com/why-aha-data
- Hospital Compare, 30-day death (mortality) rates. Available online: https://www.medicare.gov/hospitalcompare/Data/Overview.html
- Hospital Compare, Infections. Available online: https://www.medicare.gov/hospitalcompare/Data/Overview.html
- Hospital and Surgery Center Ratings. Leapfrog Group. 2024 [cited 2025 Jan 22]. Available online: https://ratings.leapfroggroup.org/
- Centers for Medicare & Medicaid Services Data. Cms.gov. 2025 [cited 2025 Jan 22]. Available online: https://data.cms.gov/tools/medicare-inpatient-hospital-look-up-tool
- Erdmann M, Marshall CM, Lay M. Transparency in Hospital Safety: A case study of ‘what it takes’ for hospitals to complete the Leapfrog Group Hospital Survey. Oklahoma State Medical Proceedings 2024;8:216.
- Shared Savings Program. CMS. www.cms.gov. 2024. Available online: https://www.cms.gov/medicare/payment/fee-for-service-providers/shared-savings-program-ssp-acos
- Tesser V. Reporting to the Leapfrog ASC Survey: 4 competitive advantages - NJHCQI. NJHCQI. 2021 [cited 2025 Jan 22]. Available online: https://www.njhcqi.org/reporting-to-the-leapfrog-asc-survey-4-competitive-advantages/
Cite this article as: Zikos D, Kato MT. Factors associated with participation in the Leapfrog Groups annual survey among acute care hospitals in the U.S. Midwest. J Hosp Manag Health Policy 2025;9:27.
