Cost-effectiveness of results-based financing of maternal and child health services in Zimbabwe: a controlled pre-post study

Introduction

Once known for its high achievement of health outcomes in Sub-Saharan Africa between 1990 and 1995, Zimbabwe’s health sector declined in performance and coverage of health services during the decade between 2000 and 2010. Fluctuations in Zimbabwe’s economy since 2000 similarly affected the health status of its population. The maternal mortality ratio (MMR) declined from 695 deaths per 100,000 live births in 1999 to 555 in 2005, but then peaked at 960 in 2010–2011 (1). Although maternal and child health (MCH) indicators showed some subsequent signs of improvement, MMR still remained far short of the Millennium Development Goal (MDG) target of 174 deaths per 100,000 live births. The high MMR in 2010-11 was also accompanied by the high under-five [84] and infant [57] mortality rates per 1,000 live births in the country (1).

These high pre-2012 rates were likely due to low coverage and quality of key MCH services. According to the Demographic and Health Survey (DHS) in 2010–2011, skilled birth attendance was only 66.2% of deliveries and only 64.8% of pregnant women received four or more antenatal visits. Following delivery, only 27.1% of women received postnatal care in 2010 (2). Mothers with no formal education were further disadvantaged, with only 38% births with skilled attendants, compared to 95% for mothers with more than secondary education. Similar disparities existed across economic strata, with skilled birth attendance of 47.5% in the poorest wealth quintile versus 90.6% in the wealthiest quintile. Financial barriers were regarded as one of the major reasons for not being able to access health care, followed by the distance to health facilities (1).

To some extent, low government health expenditures contributed to these problems. Zimbabwe devoted only 8% of central government expenditure on health in 2011 and 10% in 2012 (2,3). The country faced financial constraints, which forced many poor households to forego necessary health care. The quality of Zimbabwe’s health services also presented challenges (1,4). Problems included understaffing and lack of up-to-date clinical knowledge and skills for providers. To address these problems, the Government of Zimbabwe pre-piloted a results-based financing (RBF) program in 2011 and initiated a full pilot with evaluation in 2012 through the World Bank’s Health Sector Development Support Project (5).

Since 2006, the number of countries or organizations piloting or scaling up RBF approaches has increased steadily. As of June 2020, the Zimbabwe RBF program was one of 27 implemented or piloted in low- and middle-income countries (LMICs) supported by US$1.6 billion from the Health Results Innovation Trust Fund (managed by the World Bank) and International Development Agency. RBF impact evaluations were publicly reported for 24 countries (6). Beginning with a seminal study from Rwanda (7), several country studies (8-10) and reviews (11-16) of RBF in LMICs have been reported in the scientific literature. While results vary by indicator and setting, the plurality are favorable, several neutral, and only one adverse (17).

Reviewers have pointed out, however, that even if RBF works, the resources spent measuring and rewarding performance detract from those available for service provision (7). Cost-effectiveness analyses of RBF in other countries (e.g., Haiti and Zambia) found RBF generally cost-effective (10,18). A further question concerns separating the value of additional resources provided to health facilities from their intended conditionality on performance. The Zimbabwe program, like previous RBF programs in Rwanda and Zambia, had a comparison arm with “input-based” financing. Beginning in month 6 and continuing through the remainder of the 27-month study period, these facilities received funding through the UNICEF-operated Health Transitions Fund (HTF) on behalf of several donors. HTF payments, which were not tied to performance, were equivalent to the average for RBF facilities. User fees were removed in both RBF and control facilities.

Thus, this Zimbabwe cost-effectiveness evaluation mainly examines the pure incentive effect of RBF, separate from the additional resources that accompany it. Additionally, this study also helps examine the sustainability of RBF programs in a resource-limited setting. Its designers chose the 27-month period as long enough to separate start up from ongoing activities and impact maternal-child health outcomes, but short enough to provide timely results.

Finally, both the public and international partners had seen waning confidence in institutions due to Zimbabwe’s previous hyperinflation, defaults on international obligations, economic decline, and massive unemployment. This context thus tests RBF in an extremely challenging environment. This study seeks to inform decisions about RBF not only for Zimbabwe, but in applications worldwide. We present the following article in accordance with the CHEERS reporting checklist (available at http://dx.doi.org/10.21037/jhmhp-20-84).

Methods

Research design

The cost-effectiveness analysis followed a published protocol building on the impact evaluation of the RBF program (19) combining multiple sources of data (see Figure 1). The impact evaluation of Zimbabwe’s RBF program used a quasi-experimental research design to evaluate the program’s impact on utilization of key health services (20). The impact evaluation involved 32 districts in 16 representative pairs. To create these 16 pairs, first a principal components analysis was performed on all districts in the eligible provinces based on geographical accessibility, type of health facility, size of the catchment population, proportion of health facility staff in position, presence of key staff (e.g., District Medical Officer or District Health Officer), and historical utilization rates of key maternal child health services (antenatal and postnatal care, vaccinations, and institutional delivery).

Figure 1 Schema for this study. DALY, disability-adjusted life year; QALY, quality-adjusted life year.

Next, two districts from the upper quintile and two from the lower quintile in each province were randomly chosen. The Zimbabwe Ministry of Health and Child Care (MOHCC) then purposively selected one district in each pair to receive the RBF program, with the other district in each pair serving as a control district. The selection may have been an attempt to assign districts to RBF that the MOHCC felt needed more resources, as the RBF districts had 30% of households in Zimbabwe’s lowest wealth quintile compared to 23% in control districts (20). The baseline was built on the DHS survey, and thus it had benefit of decades of previous surveys with similar methodology across dozens of countries.

Consistent with impact evaluations in other RBF programs, both baseline and endline surveys interviewed substantial numbers of households (1,610 households in baseline and 1,830 in endline). The difference-in-differences (DID) analysis, described below, controls for any differences in baseline levels between intervention and control districts. Altogether, the RBF and control districts contained 3.46 and 2.23 million people, respectively. Health facilities in the RBF group received incentives tied to performance on MCH care (quantity and quality of service delivery). Health facilities in the control districts received fixed subsidies unrelated to performance.

Quantity incentives

The quantity incentives were calculated as the sum of the products of the verified numbers of 17 target MCH services times the unit cost of each service set by the RBF program. The unit prices at rural health centers, which approximated the relative importance of the services, ranged from $0.05 for a curative visit to $12.50 for an institutional delivery. Table 1 shows the incentivized indicators and the incentive payment per visit at the health-center level. These payments were increased by a remoteness bonus, which paid remote facilities (e.g., having limited or no public transportation or located at a substantial distance from the nearest district hospital) up to 30% more per service. Some payment rates were reduced in 2013 to ensure that aggregate incentive payments remained within the project’s budget. Supplement I (Table S1) shows the corresponding payment schedule for district hospitals in the incentives arm.

Quality incentives

To assess the incentive payments for quality of care, the RBF program designed a “balanced score card” covering about 100 structural and process quality measures, as well as organization and management performance. Staff of Cordaid, the implementation agency of Zimbabwe’s RBF pilot program, scored each RBF health facility in an RBF district from 0% to 100%. This score could then add up to 25% the payments to the facility based on its earnings for quantity and remoteness bonus. The structure and process measures broadly paralleled those of RBF programs in other countries (21).

Household survey and health facility survey

To assess the impact of RBF on services delivery, two rounds of household surveys were used. For the first round, the impact evaluation team did not collect original data. Instead, the baseline assessment used the 2010–2011 Zimbabwe DHS, conducted from Sep. 2010 to Mar. 2011, as its timing preceded the roll-out of the RBF pilot study in April 2012 (1). The baseline 1,610 households with a pregnancy-related outcome, i.e., live birth, stillbirth, abortion or miscarriage within the 2 years prior to the survey in the study area, were included in the analysis. The follow-up household survey, conducted from May through August 2014, surveyed 1,836 households using a full community listing that identified households with a pregnancy-related outcome in the 2 years prior to survey. The follow-up survey covered socio-economic measures, general health-seeking behavior, use of MCH services and perceived quality. The household survey estimated coverage of antenatal care (ANC), post-partum and postnatal care (PNC), institutional delivery, immunizations, and intermittent preventive treatment in pregnancy (IPTp). Although coverage of family planning could have also been calculated from the household survey, results may not have been representative due to the sample selection of recently pregnant women.

In addition, two rounds of health facility surveys were conducted in the 32 districts covering both primary health centers and district hospitals. At baseline (Nov. 2011 to Feb. 2012), 197 health facilities were surveyed while at follow up (May to Aug. 2014), 222 were surveyed. The health facility survey included a comprehensive review of health data, health worker interviews, health knowledge, patient exit interviews, and service delivery. The health facility survey provided information on utilization of family planning and HIV/AIDS services, including testing, counselling, and antiretroviral (ARV) treatment of pregnant women with HIV. The health facilities also provided information on general quality of care and service-specific quality measures, which we used to construct a quality index. Our effectiveness measures combined the two rounds of quality indexes and the two household surveys.

Cost components

Given that the cost-effectiveness analysis of RBF is primarily designed to inform the Ministry of Finance, the MOHCC, donors, and development partners about the continued implementation and potential extension of RBF, this cost-effectiveness analysis used a healthcare sector perspective (22). This perspective considers aspects that are the most relevant to decision makers: the costs to these funders in delivering services. This perspective excludes non-medical direct costs, such as household travel expenses. This perspective does not explicitly count indirect costs (e.g., the value of income or time lost due to illness or premature death), but does count these benefits through lives saved.

Similarly, to provide operational recommendations for decision making by key stakeholders, this study examined financial costs rather than economic costs, as donors and implementers are most interested in return from direct financial investments. The financial approach recognizes that many decisions must be made within a time-limited period. Within the relevant period, for government facilities most personnel, equipment, and building expenditures are fixed costs, while many consumables are variable costs. Therefore, for the cost analysis, we included all RBF program costs and costs of additional consumables (drugs and supplies) due to the increase of services resulting from the RBF program. Program costs, the most important component, comprised net results-based incentive payments to health facilities, in-country operational and administrative costs for running the RBF program, and costs incurred by the funder (World Bank) for supporting the designing, implementing and monitoring the RBF program.

Data sources for costs

We obtained program costs from Cordaid, the Ministry of Finance, the MOHCC, and the World Bank Zimbabwe office. World Bank staff allocated their time between impact evaluation activities (which served both RBF and control districts) and implementation oversight (dedicated entirely to RBF activities). We asked World Bank staff to estimate the cost for each type of activity and allocated the cost accordingly. For the cost incurred by the World Bank on the impact evaluation, we further allocated it to RBF and control districts based on the share of the operational costs between the two groups.

To capture the induced cost from the increased services due to the RBF program, we derived costs of all consumables, such as drugs and supplies, from a data set compiled by the National Pharmaceutical Company (NatPharm), the national pharmaceutical distribution center. We extracted all quantities and prices of all NatPharm products supplied to study districts from January 2011 through March 2012 (the five quarters preceding the RBF pilot study) and from April 2012 through June 2014 (the nine quarters during the RBF pilot study). We then calculated NatPharm costs per person per quarter in intervention and control districts and employed a DID analysis to determine the incremental costs of consumables. We collected data on disbursements of the HTF to control districts from the Crown Agents, the HTF’s implementing agency. We tallied costs over the RBF’s implementation from 2012 through 2014 and expressed them in 2012 US dollars (the starting year for the intervention).

Quantity component of effectiveness

In view of the objectives of Zimbabwe’s RBF program, the measurement of effectiveness focused on assessing hypothesized improvements in both the utilization and quality of MCH services. To assess impact on utilization, we obtained the statistical results from both the household and health facility surveys on key MCH services (i.e., ANC, PNC, institutional delivery, and child immunizations). Our impact assessment built on the only evaluation of the Zimbabwe RBF program (20).

Using the DID approach, we first calculated quantity-based change in each relevant indicator as the change in quantity in the RBF group less change in quantity in the control group. We added this change to the baseline coverage to derive the quantity-based endline coverage for each indicator.

Quality component of effectiveness

We assessed hypothesized improvements in quality of care from the two rounds of the health-facility survey, which measured general quality, clinical process, availability of drugs and supplies, availability of equipment, and availability of qualified human resources. We used the weights from a Delphi panel of 15 public health experts from a parallel program in Zambia to determine the relative importance of each of the quality components and generated a quality index (ranging from 0 to 1) for each service (18). We conducted two rounds of the Delphi survey and used the results from the second round (after experts had exchanged information with one another) for the analysis. The Delphi questionnaire used to estimate the relative importance of each quality component is provided in Supplements II and III.

We are unaware of any production function that directly assessed the impact of the quality indicators used in Zimbabwe on health outcomes. We therefore used the same expert panel to estimate the health impact of the quality of care to generate an impact index using a quadratic function (see Figure 2). The family of quadratic functions was used because of its flexibility to accommodate concave up, concave down, and linear relationships, including threshold and ceiling effects. Supplement IV (Figure S1) provides the Delphi questionnaire on the impact of quality of care on health and Supplement V (Figure S2) shows the relative importance of each component.

Figure 2 Family of functions showing alternative impacts of quality (label corresponds to effect of a quality index of 50%).

To incorporate quality of care in the analysis, we generated the quality adjusted coverage of each key indicator by multiplying its health-effect index times the coverage of corresponding service from the household or health facility survey. Finally, our quality-adjusted scenario used the Lives Saved Tool (LiST) to convert the increase in quality-adjusted coverage of these relevant indicators into the number of lives saved (quality-adjusted scenario) (23). In our quantity only scenario, we used the coverage information from the impact evaluation directly in the LiST modelling, and estimated the number of lives saved under this scenario (quality unadjusted scenario). The LiST model estimated the efficacy of interventions from the international public health literature. Our application of this model to Zimbabwe implicitly assumes that the intervention would work comparably well in that country. As the country’s health workers are generally well trained, the assumption is plausible.

Cost-effectiveness analysis

We used key parameters from the Zimbabwe data preloaded in the LiST tool (e.g., the age structure of the population), and adjusted the population size to the size of the catchment population in the RBF group. With the estimated changes of the coverage of key MCH services (e.g., institutional delivery, ANC, PNC) from the DID analysis, the LiST tool produced the number of lives saved from each net change in each indicator under two scenarios: one without quality improvement adjustment and one with. As the lives saved were those of children and women of childbearing age, we assumed that the individuals were then healthy. We converted this into discounted quality-adjusted life years (QALYs) based on Zimbabwe’s 2012 age-specific life expectancy (24). We discounted both life years and costs at 3% per year, as reaffirmed by recent guidelines (22).

To standardize the population size, we converted both incremental costs and effectiveness (measured in QALYs gained) to a per capita basis. We then calculated the incremental cost-effectiveness ratio (ICER) as follows:

ICER=IncrementalcostsofRBFprogram/capitaQALYsgained/capita[1]

Sensitivity analyses

To construct sensitivity analyses for quantity and quality outcomes, we calculated the mean and standard deviation, fit a normal distribution, and calculated 95% confidence intervals (CIs) for key results. We selected institutional delivery as the bellwether outcome based on highest unit reimbursement rate, its importance to lives saved (23) and its significant outcomes in the impact evaluation (20). For this outcome, we calculated the ratios of its lower and upper CIs to the central estimate. We then applied these ratios to obtain the 95% CIs of impacts and ICERs on lives saved and QALYs gained both with and without quality adjustment.

This study did not collect nor access any human studies data (all human data were existing aggregate statistics) so that ethical approval was waived.

Results

Net costs

Table 2 shows costs of the RBF program in intervention and control districts, excluding research and evaluation costs. Aggregate costs managed by the purchasing agent (Cordaid), a Netherlands-based NGO, were $24.09 million from April 2012 through June 2014 (2.25 years) or $3.09 per person per year (i.e., $2.04 plus $1.05). The distribution of annual program costs per capita found that $2.04 (66%) went to RBF incentive payments to health facilities while $1.05 (34%) was the cost for the purchasing agent (Cordaid) to operate the RBF program. These annual per capita operating costs comprised staff ($0.41), general administration ($0.20), capacity building for health center personnel ($0.12), headquarters support costs ($0.11), transport ($0.09), Cordaid capital items ($0.09), and supplies and equipment for health facilities ($0.04).

The costs of consumables from January 2011 to March 2012 from NatPharm before the RBF pilot study were $6.1 million in the intervention group and $6.8 million in the control group. During the pilot study (April 2012 to June 2014) the costs of consumables increased to $29.5 million in the intervention group and $33.5 million in the control group (i.e., $13.1 million/year and $14.9 million/year, respectively). Due to the substantial increase in the cost of consumables in the post-period, we calculated predicted costs in the intervention group based on the ratio of pre-period to post-period in the control group. We then computed the difference between actual costs and predicted costs in the intervention group, which was –$0.06/capita.

The estimated program costs at the World Bank headquarters was $752,821 over the 2.25 years, equivalent to $0.10 per capita per year. In addition, the intervention group facilities received no payments from the HTF, whereas the control group did. We computed the net cost of RBF for HTF payments per capita per year as –$0.81 (intervention vs. control group). When considering all costs, the net costs of the intervention compared to the control group was $2.32 per capita per year.

Net impact on service utilization

Based on the impact evaluation, the major improvements in health services lay in more institutional deliveries, increased postpartum care, and post-partum tetanus vaccinations specifically. The net improvements were 13.4, 13.3, and 20.0 percentage points, respectively. The impacts on key variables are shown in Table 3, and the detailed methods and full list of impacts were reported elsewhere (20).

Net impact on quality of care

Table 4 shows that RBF was associated with net improvements in quality of care for all major MCH services. It is noteworthy that these net improvements were the result of maintaining the baseline quality levels in the intervention group compared to declines for all services in the control group. The results from DIDs were all statistically significant. For example, compared to the control group, the quality of care for post-natal care improved 25% more in the RBF group.

Impacts on lives saved and QALYs gained

Table 5 shows the number of deaths and lives saved from the improvements of these two major services. Without quality adjustment, the improvement in institutional delivery and postpartum care translated into 71 lives saved of pregnant women and 587 lives saved of children under five over the 2.25 years in the RBF area. In total, 15,498 QALYs were gained. After we combined quality improvements for all services, such as ANC, vaccination, institutional delivery and PNC, and the utilization of services, the impact of RBF was even greater. The RBF program saved 147 lives of pregnant women and 1,060 lives of children under 5 over the 2.25 years, saving 28,388 QALYs, suggesting that improving quality of care accounted for 45.4% of the QALY gains for institutional deliveries and increased postpartum care (i.e., 100% – 15,498/28,388).

Cost-effectiveness of RBF

Table 6 shows that the ICER was $1,166/QALY gained (or $27,457/life saved) without quality adjustment, with 95% CIs of $471–$1,861 and $11,103–$43,811, respectively. Adding the consideration of quality-of-care improved (lowered) the ICER to $636/QALYs gained (or $14,968/life saved), with 95% CIs of $257–$1,015 and $5,943–$23,453, respectively. The gross domestic product (GDP)/capita in Zimbabwe in 2012 (25) was $956. As the ICER was less than the GDP per capita, the RBF program was highly cost-effective when quality improvements are included. Even the upper bounds of the CIs, being 1.96 and 1.06 times the per capita GNI, make the program at least cost-effective (26).

Discussion

In this pilot RBF program in Zimbabwe, RBF districts achieved significant increases in the coverage of institutional deliveries and postpartum care and in the quality of care compared to matched control districts. With a cost of $2.32 per capita per year, RBF proved to be very cost-effective (i.e., $636/QALY) when projected to a mature program. The program’s design authorized the health facility’s committee to distribute some of the quarterly earnings as bonuses to the staff, thereby helping with critical alignments between institutional and individual incentives (27).

Ideally, the ICER of RBF in Zimbabwe should be judged against that of alternative programs within the country, but we could not identify any such examples. However, as another criterion for evaluating the cost-effectiveness of an intervention, the World Health Organization Commission on Macroeconomics and Health suggested that an intervention with a cost per QALY ratio to the country’s per capita GDP of one or less is very cost-effective (26,28,29). If the ratio is one to three times per capita GDP, then the intervention is cost-effective. As Zimbabwe’s 2012 per capita GDP (25) was $956, these central cost-effectiveness ratios ($1,166 and $637) represent 1.21 and 0.67 times the per capita GDP, respectively. Thus, RBF proved cost-effective based on coverage improvements alone and very cost-effective combining quality and quantity effects.

Following this 2-year pilot study, Zimbabwe extended RBF to the rest of the country’s 62 rural districts. In an attempt to examine the long-term impact or cost-effectiveness of RBF, we compared pre-RBF (2010-11) DHS with its post RBF (2015) counterpart (30). The comparison documents improvements in both incentivized indicators, such as facility-based deliveries (up from 65% to 77%) and non-incentivized indicators, such as use of oral rehydration solution (up from 21% to 41%). As the DHS is not reported below the provincial level, however, it cannot be used to evaluate RBF.

As the initial program’s capital costs would not typically be included as an expense in the national program, national costs per capita would thus be 9% lower, or $2.11 per capita. Not being saddled with a start-up period, the mature national RBF program would probably have greater average annual benefits than the pilot program. The average benefit over the roughly 2-year pilot period was 3,650 QALYs per million population per year. As this benefit represents the net difference between the baseline and follow up surveys about 2 years later, a mature program (not needing a phase-in program) would be expected to have 33% greater benefits, or 4,850 QALYs per 100,000 population. This adjustment would occur because program impacts are phased in uniformly over the first 12 months. Thus, only half of a mature program’s annual benefits would be obtained in the first year, while the second year would realize all of the mature program’s annual benefits. The combined effects make the ICER of a mature program $479 per QALY (0.49 times the per capita GDP)—even more cost-effective.

The pilot’s favorable ICER suggests that the RBF program could be further strengthened by targeting resources to services with high clinical impact, but with low utilization. Incentives to services that have had high coverage should be minimized or should be targeted to populations with low coverage. One example is the coverage of ANC. It accounted for 7% of RBF’s incentive payments at health centers and ranked fifth among all incentivized indicators, and had high coverage in Zimbabwe prior to the implementation of the RBF program, with an average of 90% in combined RBF and control areas (16), leaving little room for potential improvement from interventions.

Our inclusion of multi-dimensional measures of quality of care represents an advance over RBF evaluations based only on quantity or coverage. Our analysis of quality was guided by the principle that the delivery of each service with excellent quality would deliver health benefits equivalent to those in other carefully implemented settings—the effectiveness embodied in LiST. Extremely poor quality, on the other hand, was considered equivalent to not delivering the service at all. Adverse outcomes in such circumstances, if they occurred, were estimated to be balanced by occasional lucky good results. These principles meant that all the impact curves in Figure 2 were constrained to pass through the lower left and upper right corners.

A limitation of our approach was our need to rely on an expert Delphi panel, rather than objective evidence, to select the specific curve in Figure 2 to relate quality levels into health outcomes. Nevertheless, the effects on outcomes of possible misjudgment by our expert panelists were likely limited. For example, if the quality index were very low (i.e., under 10%) all of the impact curves are close to their lower left corners. Thus, the impact can be no more than 20% regardless of which impact curve were chosen. Similarly, if the quality index were very high (i.e., over 90%) all of the impact curves are close to their upper right corners and the impact is at least 80%, regardless of which impact curve were chosen.

The effect of moderate changes in quality is governed by the slope of the curve chosen in Figure 2. However, these curves all have an average slope of 45 degrees. If the misjudgments were random, a possible misjudgment by some experts resulting in their reliance on curve segments that were too flat being counterbalanced by reliance by other expert’s reliance on segments that were too steep. An alternate scenario, which excluded quality adjustments altogether, found that the program was cost-effective based on quantities alone.

Another potential limitation was fact that our healthcare perspective did not explicitly allocate additional personnel costs if volume increased. This perspective follows the actual budgeting of the MOHCC, which did not adjust personnel slots by service volume. However, the distribution of part of the facility’s earnings to staff did, in fact, pay existing staff for greater effort.

Even though Zimbabwe’s RBF program proved very cost-effective, a further policy question concerns RBF’s comparison against other reproductive and maternal child health (RMCH) interventions. Economic evaluations of many RMCH interventions in LMICs have found that most of these are also highly cost-effective or cost-beneficial. The ICERs for the RBF program in Zambia ($837 or 0.48 times the per capita GDP) (18) and for reproductive health vouchers and modeled maternal community-based health insurance in Uganda, were $302 and $298, or 0.59 and 0.58 times the per capita GDP, respectively (31). Systematic reviews have found many economically advantageous RMCH programs (32,33). As with RBF, the setting and implementation can dramatically affect results. For example, an economic evaluation found that the management of childbirth-related complications was 100 times more advantageous in Zimbabwe compared to Gabon (33). Thus, RBF should be not a competitor, but a complement to other RMCH interventions, working with them to strengthen health systems.

As with other RBF programs, context and implementation matter (11). Site visits and process evaluations documented inevitable shortcomings in program implementation, such as delays of up to 3 months in sending incentive payments to facilities due to national budget problems. Open communication from the purchasing agent to facility staff, however, minimized the adverse impact (34).

Building on the experience in this study with systematically measuring quality, Zimbabwe instituted initiatives with continuous quality improvements (35). Then, in 2018, reflecting the institutionalization of RBF, the Government of Zimbabwe took over program management from the World Bank (36). As Zimbabwe and other countries continue refining RBF programs, we suggest several potential improvements. First, for some services, such as vaccinations and prenatal care, coverage was virtually saturated so major improvements in quantity were not possible. Future incentive programs could drop incentivizing coverage of those services, shifting the incentive to quality or other indicators for under-utilized services. Second, it would be helpful to impose penalties for repeated declining performance to complement incentives for good performance, so that participants will work hard to avoid losses. Studies conducted in the US show that the combination of reward and penalties improves the efficiency of service delivery (37,38). Third, to motivate participants more intensively, the program may wish to set thresholds for various services and institute higher rewards only for incremental performance above those thresholds. Fourth, to maximize the impact of RBF, it would be helpful for the MOHCC to consider integrating RBF with demand-side interventions (e.g., voucher schemes or community RBF). Research in the US suggested that incentives shared between health workers and patients (also termed conditional cash transfers) would be more effective than rewards for providers alone (39).

Acknowledgments

The authors are grateful to Shingai Mutetwa (World Bank) for assistance with data collection, to Brig. Gen. Dr. Gwinji (Permanent Secretary MOHCC), Dr. Dhlakama (former Principal Director for Policy and Planning), Dr. T. Goverwa, Dr. P. Mafaune, Dr. Wencelas Nyamayaro, Dr. Bernard Madzima (National Results Based Financing Technical Working Group for Impact Evaluation, MOHCC) and Dr. Mudyiradima (Principal Director Policy and Planning), Priscilla Madzinga, Arjanne Rietsema, Frank van der Looij, and Meindert van der Werf (Cordaid), Ha Nguyen (World Bank) for information and insights, and Clare L. Hurley (Brandeis) for editorial assistance.

Funding: This study was funded by a contract from the World Bank to Brandeis University supported by the World Bank Health Results Innovation Trust Fund.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Journal of Hospital Management and Health Policy for the series “Incentives and health system efficiency in low- and middle-income countries”. The article has undergone external peer review.

Reporting Checklist: The authors have completed the CHEERS reporting checklist. Available at http://dx.doi.org/10.21037/jhmhp-20-84

Data Sharing Statement: Available at http://dx.doi.org/10.21037/jhmhp-20-84

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jhmhp-20-84). The series “Incentives and health system efficiency in low- and middle-income countries” was commissioned by the editorial office without any funding or sponsorship. WZ served as the unpaid Guest Editor of the series and serves as an unpaid editorial board member of Journal of Hospital Management and Health Policy from August 2019 to July 2021, and Dr. WZ reports non-financial support from World Bank, during the conduct of the study. Dr. DSS reports grants from World Bank, during the conduct of the study; grants from Sanofi Pasteur, Inc, grants from Takeda Vaccines, Inc., grants from Abbott, Inc., outside the submitted work. Dr. RM reports grants from World Bank, during the conduct of the study. Dr. AKD reports non-financial support from World Bank, during the conduct of the study. Dr. CNS reports non-financial support from World Bank, during the conduct of the study. Dr. SS reports non-financial support from World Bank, during the conduct of the study. Dr. SB reports non-financial support from World Bank, during the conduct of the study. Dr. JF reports grants from The World Bank, during the conduct of the study. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study did not collect nor access any human studies data (all human data were existing aggregate statistics) so that ethical approval was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Zimbabwe National Statistics Agency, ICF International. Zimbabwe Demographic and Health Survey 2010-2011. Calverton: ZIMSTAT and ICF International Inc.; 2012.
2. World Health Organization. Global Health Expenditure Database. [Cited 2016 Aug 8]. Available online: http://apps.who.int/nha/database
3. World Bank. Zimbabwe - Health public expenditure review. Washington DC: World Bank, 2015. [Cited 2017 Nov 7]. Available online: http://documents.worldbank.org/curated/en/304101468188950998/Zimbabwe-Health-public-expenditure-review
4. Zimbabwe National Statistics Agency. Zimbabwe Multiple Indicator Cluster Survey 2014, Final Report. Harare: ZIMSTAT and UNICEF, 2015.
5. World Bank. Health Sector Development Support Project. Washington DC: World Bank, 2017. [Cited 2017 Nov 17]. Available online: http://projects.worldbank.org/P125229/zimbabwe-health-results-based-financing?lang=en
6. RBFHEALTH. Projects: Country Pilot Grants and their Impact Evaluations. 2018. [Cited 2020 Jun 8]. Available online: https://www.rbfhealth.org/projects
7. Basinga P, Gertler PJ, Binagwaho A, et al. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet 2011;377:1421-8. [Crossref] [PubMed]
8. Bonfrer I, Soeters R, Van de Poel E, et al. Introduction of performance-based financing in burundi was associated with improvements in care and quality. Health Aff (Millwood) 2014;33:2179-87. [Crossref] [PubMed]
9. Van de Poel E, Flores G, Ir P, et al. Impact of performance-based financing in a low-resource setting: a decade of experience in Cambodia. Health Econ 2016;25:688-705. [Crossref] [PubMed]
10. Zeng W, Cros M, Wright KD, et al. Impact of performance-based financing on primary health care services in Haiti. Health Policy Plan 2013;28:596-605. [Crossref] [PubMed]
11. Bertone MP, Falisse JB, Russo G, et al. Context matters (but how and why?) A hypothesis-led literature review of performance based financing in fragile and conflict-affected health systems. PLoS One 2018;13:e0195301 [Crossref] [PubMed]
12. Blacklock C, MacPepple E, Kunutsor S, et al. Paying for performance to improve the delivery and uptake of family planning in low- and middle-income countries: A systematic review. Stud Fam Plann 2016;47:309-24. [Crossref] [PubMed]
13. Friedman J, Scheffler R. Pay for performance in health systems: theory, evidence, and case studies. In: Scheffler R. editor. The World Scientific Handbook of Global Health Economics. London: World Scientific Press, 2016:295-332.
14. Scott A, Liu M, Yong J. Financial incentives to encourage value-based health care. Med Care Res Rev 2018;75:3-32. [Crossref] [PubMed]
15. Witter S, Fretheim A, Kessy FL, et al. Paying for performance to improve the delivery of health interventions in low- and middle-income countries. Cochrane Database Syst Rev 2012;CD007899 [Crossref] [PubMed]
16. RBFHEALTH. Rewarding provider performance to improve quality and coverage of maternal and child health outcomes. Zimbabwe results-based financing pilot program: evidence to inform policy and management decisions. 2016. [Cited 2020 Aug 30]. Available online: https://www.rbfhealth.org/resource/rewarding-provider-performance-improve-quality-and-coverage-maternal-and-child-health
17. Huillery E, Seban J. Pay-for-performance, motivation and final output in the health sector: experimental evidence from the Democratic Republic of Congo. 2014. [Cited 2020 Oct 28]. Available online: https://spire.sciencespo.fr/hdl:/2441/4pmvo3bm7m9claao2gl0337ip4/resources/2014-12.pdf
18. Zeng W, Shepard DS, Nguyen HTH, et al. Cost-effectiveness analysis of results-based financing in Zambia: Results from a cluster-randomized trial. Bull World Health Organ 2018;96:760-71. [Crossref] [PubMed]
19. Shepard DS, Zeng W, Nguyen HTH. Cost-effectiveness analysis of results-based financing programs: a toolkit. Health, Nutrition, and Population (HNP) Discussion Paper. Washington DC: World Bank, 2015. [Cited 2020 Aug 30]. Available online: http://documents.worldbank.org/curated/en/2015/09/25069701/cost-effectiveness-analysis-results-based-financing-programs-toolkit
20. World Bank, International Bank for Reconstruction and Development. Rewarding Provider Performance to Improve Quality and Coverage of Maternal and Child Health Outcomes: Zimbabwe Results-Based Financing Pilot Program Evidence to Inform Policy and Management Decisions. 2016. [Cited 2018 May 29]. Available online: http://www.rbfhealth.org/sites/rbf/files/documents/Zimbabwe-Impact-Evaluation-Overview-Report.pdf
21. Josephson E, Gergen J, Coe M, et al. How do performance-based financing programmes measure quality of care? A descriptive analysis of 68 quality checklists from 28 low- and middle-income countries. Health Policy Plan 2017;32:1120-6. [Crossref] [PubMed]
22. Neumann P, Sanders G, Russell L, et al. editors. Cost-Effectivess in Health and Medicine. 2nd ed. New York: Oxford University Press, 2017.
23. U.S. Agency for International Development. Health Policy Project. Software and Models: Spectrum Policy Modeling System. [Cited 2020 Aug 30]. Available online: http://www.healthpolicyproject.com/index.cfm?id=software&get=Spectrum
24. World Health Organization. Global Health Observatory data repository. 2016. [Cited 2016 Oct 26]. Available online: http://apps.who.int/gho/data/?theme=main&vid=61860
25. World Bank. DataBank: World Development Indicators 2017. [Cited 2017 Nov 28]. Available online: http://databank.worldbank.org/data/reports.aspx?source=world-development-indicators
26. WHO Commission on Macroeconomics and Health. Macroeconomics and Health: Investing in Health for Economic Development. Report of the Commission on Macroeconomics and Health: Executive Summary. Geneva: World Health Organization, 2001. [Cited 2020 Aug 30]. Available online: https://apps.who.int/iris/handle/10665/42463
27. Paul E, Renmans D. Performance-based financing in the heath sector in low- and middle-income countries: Is there anything whereof it may be said, see, this is new? Int J Health Plann Manage 2018;33:51-66. [Crossref] [PubMed]
28. Hutubessy R, Chisholm D, Edejer TT. Generalized cost-effectiveness analysis for national-level priority-setting in the health sector. Cost Eff Resour Alloc 2003;1:8. [Crossref] [PubMed]
29. Robinson LA, Hammitt JK, Chang AY, et al. Understanding and improving the one and three times GDP per capita cost-effectiveness thresholds. Health Policy Plan 2017;32:141-5. [Crossref] [PubMed]
30. U.S. Agency for International Development. Demographic and Health Surveys: Zimbabwe: DHS, 2015 - Final Report. [Cited 2018 May 29]. Available online: https://dhsprogram.com/publications/publication-FR322-DHS-Final-Reports.cfm
31. African Strategies for Health. Cost-effectiveness of reproductive health vouchers and community-based health insurance in Uganda. Arlington: African Strategies for Health, 2015.
32. Horton S, Levin C. Cost-effectiveness of interventions for reproductive, maternal, neonatal, and child health. In: Black RE, Laxminarayan R, Temmerman M, et al. editors. Reproductive, maternal, newborn, and child health: disease control priorities. 3rd ed. Washington DC: The International Bank for Reconstruction and Development/The World Bank, 2016.
33. Maitra C, Hodge A, Jimenez Soto E. A scoping review of cost benefit analysis in reproductive, maternal, newborn and child health: what we know and what are the gaps? Health Policy Plan 2016;31:1530-47. [Crossref] [PubMed]
34. World Bank. RBF process monitoring and evaluation: round two key preliminary findings. Washington DC: World Bank, 2015.
35. World Bank, Cordaid, Ministry of Health and Child Care of Zimbabwe. editors. CQI (continuous quality improvement) initatives in the context of RBF. Harare: Annual Results and Impact Evaluation Workshop for Results-Based Financing, 2016.
36. Cordaid. The Magic of Results-Based Financing. 2017. [Cited 2018 May 29]. Available online: https://www.cordaid.org/en/news/magic-results-based-financing
37. Sibley RA, Charubhumi V, Hutzler LH, et al. Joint replacement volume positively correlates with improved hospital performance on centers for Medicare and Medicaid Services quality metrics. J Arthroplasty 2017;32:1409-13. [Crossref] [PubMed]
38. Stanowski AC, Simpson K, White A. Pay for performance: Are hospitals becoming more efficient in improving their patient experience? J Healthc Manag 2015;60:268-85. [Crossref] [PubMed]
39. Asch DA, Troxel AB, Stewart WF, et al. Effect of financial incentives to physicians, patients, or both on lipid levels: a randomized clinical trial. JAMA 2015;314:1926-35. [Crossref] [PubMed]