Cost-effectiveness analysis of the decentralized facility financing and performance-based financing program in Nigeria

Introduction

Over the last decade, Nigeria has made limited improvement in maternal and child health despite impressive economic growth. The key maternal and child health indicators have stalled since 2003, with some recent progress seen in immunization coverage. Taking skilled birth attendance (SBA) as an example, the national coverage was 41.8% in 2003. In 2013, the coverage slightly declined to 38.1% (1). A similar pattern was observed for other indicators such as contraceptive use, antenatal care, and postnatal care. Partially due to the underuse of essential maternal and child health services (MCH), infant and maternal mortality rates are relatively high in Nigeria when compared to countries with a similar economic status as Nigeria. In 2013, the infant mortality rate was estimated at 73.3 per 1,000 live births, while the maternal mortality rate was 576 per 100,000 live births (2).

The underuse of essential MCH services reflects the inefficient health service delivery system in Nigeria. Shortage of financial resources and lack of adequate and motivated staff are among the top reasons for the lower performance of the health system (3). It was reported that the majority of government budget for health in Nigeria was used for salary, leaving little money for drugs and supplies. Low quality of care (QoC) is another major complaint at health facilities (4). Many health facilities do not have basic drugs in stock, and a substantial proportion of patients were dissatisfied with the services in public health facilities (5).

Recognizing potential barriers for efficient delivery of health services, Nigeria has developed various strategies to improve MCH services. Strengthening the primary health care delivery system, particularly at the local government level (LGA), was regarded as the major intervention for improving primary service delivery (6). In Nigeria, two packages have been implemented in various levels. The Minimum Package of Activities (MPA), which contains 27 preventive and curative health services, are for the community and health center level, and the Complementary Package of Activities (CPA) with 22 services, have been implemented at the first level hospitals. At the global level, performance-based financing (PBF), where financial incentives are tied to performance, has been carried out in many low resource settings to address and promote use and quality of MCH services (7). Although results are quite mixed (8), they generally point to a positive impact on utilization and quality of key MCH services, including antenatal care, institutional delivery, postnatal care, and use of modern contraceptives. A recent review of QoC under PBF also shows favorable results, particularly on structural quality (9). PBF can not only help alleviate financial constraints at health facilities, but more importantly, it stimulates providers’ behavior changes and strengthens accountability and transparency, and catalyzes system-wide changes (10). Studies also show positive impact of programs that provide financial resources to health providers without being tied to performance on improvement of use of MCH services—this would be the decentralized facility financing (DFF) approach (11). In Nigeria, the DFF program also shares many similarities as the PBF program. For instance, the LGA level internal performance contracting was the same between the two programs; this provided, in the DFF arm, regular and high-quality coaching and the application of the same quantified quality checklist as in the PBF arm. Additionally, the DFF facilities also got contracts with certain autonomy of using decentralized funding for locally determined needs, such as drug procurement.

In recognition of potential benefits of PBF and DFF programs on MCH services, the Government of Nigeria, with funding from the World Bank, launched a PBF and a DFF program under the Nigeria State Health Investment Project (NSHIP) (12). The program was first pre-piloted in three LGAs starting in December 2011. In 2013, LGAs in three states from different geopolitical zones, Ondo in the Southwest, Adamawa in the Northeast, and Nasarawa in the Northcentral, were randomized into two groups: PBF and DFF arms. During 2014–2015, a total of 27 LGAs implemented the PBF program, 9 in Ondo, 11 in Adamawa, and 7 in Nasarawa. The health facilities in the remaining LGAs in the same state received 50% of the amount received by PBF facilities, adjusted for population coverage, under the DFF group.

As of October 2017, the completion of the endline household survey, the DFF and PBF has been implemented for about 3 years (estimated as 2.8 years). Both donors and the government are keen on understanding the cost and effectiveness of such a big investment. To understand the cost-effectiveness of the PBF program, as well as DFF, this study combines administrative data on costs and results from the impact evaluation, providing empirical evidence on the cost-effectiveness of both programs.

We present the following article in accordance with the CHEERS reporting checklist (available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-20-82/rc).

Methods

Research design

The NSHIP project was implemented using a quasi-experimental research design in three states in Nigeria: Adamawa, Nasarawa, and Ondo. LGAs within each state were randomized to be included in the PBF or DFF groups. Public health facilities under the PBF received payment based on quality and quantity of health services provided. Those under DFF received a fixed amount of financial payment tied to eligible expenditures, but without any conditions. Besides additional funding, both DFF and PBF facilities received external supervision of service delivery, and enjoyed autonomy and decentralized financing that provided health facilities certain authorities to use the received funding, with community engagement in how those funds are spent. Three states were also selected as controls: Taraba, Benue, Ogun. This selection was based on their similarity of observable characteristics to the three NSHIP states. Health facilities in these three control states received no financial support from the NSHIP project.

Incremental cost assessment

This cost-effectiveness analysis (CEA) was conducted from a health system perspective in order to provide practical recommendations for key stakeholders. For the similar reasons, this analysis examined financial costs rather than economic costs. In this CEA, we focused on the incremental cost incurred in PBF and DFF groups, compared to the control group. Thus, for the cost analysis, the additional costs that we included were: (I) implementation costs, and (II) World Bank headquarters’ cost for designing, implementing and monitoring the program. Costs of consumables (e.g., drugs and supplies) due to increased services were partially subsidized through PBF or DFF. Health facilities that incurred high volume of services due to PBF or DFF used funding received from the programs to purchase drugs and supplies. To avoid double counting, we excluded costs of consumables from the analysis.

Program costs, which encapsulated the cost of administration of the project (e.g., costs of operations, capacity building, verification, and monitoring and evaluation) and incentive payments, were obtained from the World Bank Nigeria office. The World Bank headquarters’ costs were obtained from the World Bank headquarters. As it was hard to break down the headquarters’ costs by program, they were allocated to the PBF and DFF groups in proportion to the program costs. To account for the difference in population size among the three PBF groups, the program costs and World Bank headquarters’ costs were rescaled by population size and calculated as costs per capita. All costs were measured in US 2015 dollars, and a discount rate of 3% was applied.

Incremental effectiveness assessment

Estimate improvement of utilization of health services

To assess the impact of DFF and PBF on key maternal and child health indicators, household surveys were conducted prior to and after the implementation of DFF and PBF in February-April of 2014 and August-October of 2017, respectively. About 9,000 households with women who had had at least one birth within two years prior to the survey were selected. The indicators included in the surveys were coverage of services for antenatal care (ANC), iron supplementation, postnatal care (PNC), SBA, immunization, modern contraceptive use (FP), and children using insecticide-treated bed nets (ITN).

An impact evaluation was conducted by the World Bank using a difference-in-differences (DID) approach, which compares the changes from baseline to endline in NSHIP states to the same change in the control states, to examine the effect of DFF and PBF on coverage of key MCH services and their quality, based on household and health facility surveys (12). The DID analysis adjusted for the baseline differences among the three groups such that NSHIP states would be compared to the control states at the baseline (13). From the household survey, the following services were included: ANC, PNC, SBA, children using an ITN, FP, iron supplementation, and immunizations. However, data on curative services and HIV/AIDS services were not disaggregated in a way that could be converted to coverage at the household level for LiST (See the description in the section of modelling outcomes) to model their impact. Thus, these two services were excluded from the analysis.

Estimate improvement of quality of health services

Two health-facility surveys covering 786 representative health facilities were conducted at the same time as the household surveys. For this study, the health-facility survey provided information on service-specific quality measures for the following services: immunization, FP, SBA, and ANC, and curative care for under age 5.

The health-facility survey included many questions regarding the QoC for maternal and health services (e.g., ANC). There were 17, 36, 22, 19, and 20 items used to measure the QoC for immunization, ANC, maternity services, FP, and curative care for children under age 5, respectively. Among these, ANC and curative care were measured using the direct observation approach, which examined the appropriateness of following dimensions: (I) medical history collection; (II) physical examination, (III) lab tests; (IV) notice of danger signs, and so on. The QoC of immunization was focused on the appropriateness of cold chain maintenance, storage of vaccine, and vaccination delivery. The QoC of maternal services concentrated on the availability of essential equipment, training of personnel, and presence of follow-up or referral systems for patients. Each question had answers of “Yes” and “No” that was coded as 1 and 0 respectively. We generated raw scores on QoC for each service by summing the score of individual items, and then divided the raw score by the number of items that each service has to rescale the score to between 0 and 1. To quantify the impact of QoC on potential health benefits from that care, we consulted with an expert panel in September 2018. The expert panel consisted of 10 experts, who had least 3 years of experience on public health or clinical medicine and were familiar with health systems in Nigeria at the state or federal levels. The experts were asked on what share of the potential impact that each service could achieve if the QoC was 50%. We then fitted a quadratic curve and generated a health-effect index, using the same approach as the one applied in Zambia and Zimbabwe (11,14).

Modelling outcomes by combining utilization and quality improvement

We generated an effective coverage for each service by multiplying the health-effect index by the coverage of corresponding services, and it fed into the Lives Saved Tool (LiST), which converts the coverage of health services to the number of lives saved (15-18). LiST has been widely used to estimate MCH outcomes (mortality) with good validity (19). However, the shortcomings of LiST are that not all MCH could be modelled, nor can it implement probabilistic sensitivity analyses and model the impact on morbidity.

To model the impact of DFF and PBF, we used key parameters preloaded into LiST for Nigeria and adjusted the population size to that covered by PBF. With the change of effective coverage of key maternal and child services, LiST estimated the number of lives saved from improved services. We then converted this into quality-adjusted life years (QALYs) applying the formula for fatal cases (20) considering Nigeria’s life expectancy and disease burden (21,22). The life expectancy for pregnant women and children under age 5 in Nigeria was estimated to be 45.50 and 58.65 years old (21). Using a discount rate of 3%, QALYs lost per maternal death was estimated to be 21.49. Similarly, the QALYs lost per death for a child under age 5 was estimated to be 23.89. We estimated total QALYs gained by multiplying the number of cases saved by QALYs gained per case. As LiST estimates the impact by years, we first quantified lives saved and QALYs gained for two years, and then extrapolated to 2.8 years, which is the duration of DFF and PBF implementation before the mid-term evaluation.

To estimate the aggregate impact of NSHIP, we combined the QALYs gained from DFF and PBF groups together and adjusted for the population difference. Similarly, both costs and QALYs gained were rescaled by the population size when quantifying costs and health benefits of NSHIP.

Cost-effectiveness analysis and sensitivity analysis

After estimating incremental costs and effectiveness per capita, we then generated the incremental cost-effectiveness ratio (ICER) for DFF, PBF, and NSHIP as a whole under two scenarios: one without quality improvement and the other with it.

The sensitivity analysis was focused on the relative importance of QoC over that of the coverage of services. As mentioned in the section on measuring QoC of health services, we generated scores on QoC for each of the key MCH services, used the expert panel to quantify the impact of QoC on potential benefits of the services, and created the index of quality-adjusted coverage. This approach was applied in a previous study. In this study, when we applied the index of quality-adjusted coverage, the underlying assumption was that the impact from an additional 1% increase in QoC was equivalent to that from a 1% increase in coverage of the health service. In the sensitivity analysis, we generated a scenario where the impact from a 2% increase in QoC was equivalent to that from a 1% increase in coverage.

While there has been much debate over thresholds of cost-effectiveness analysis (23-25), a study examining returns on investment specific to MCH valued a healthy life year as 1.5 times GDP per capita (16). In this study, we used this number as the threshold to interpret the ICERs. In 2015, GDP/capita was $2,655 in Nigeria (26), and thus the threshold was estimated at $3,983. Interventions with ICERs smaller than the threshold were regarded as cost-effective.

Ethical statement

This study used the secondary data to perform the analysis. Ethical approval is not applicable.

Results

Over the 2.8 years of the project, a total of USD 132.87 million (129.3 million in 2015 USD) costs were used for operating the PBF and DFF programs. USD 55.01 million (53.5 million in 2015 USD) were spent as incentive payments to PBF facilities, while USD 27.50 million (26.8 million in 2015 USD) were for DFF facilities (50% of incentive payment for PBF), and the rest of USD 50.36 million (49.0 million in 2015 USD) were for the incentives to the governments and operation costs of the program, including technical assistance, payments to LGAs, state disbursement linked indicators (DLIs), monitoring and evaluation, and so on (Figure 1).

Figure 1 NSHIP disbursements of funding by component. (A) Composition of the total spending on the PBF and DFF programs from 2014–2017; (B) composition of payments to PBF and DFF programs, LGA PHCs and HMBs. The disbursement shown in the figure applied to both PBF and DFF arms. PBF, Performance-based financing; DFF, Decentralized facility financing; FMOH, Federal Ministry of Health; PPF, project preparation facility; DSF, debt sustainability framework; NSHIP, nigeria state health investment project; SPHCDA, state primary care development agency; NPHCDA, national primary care development agency; LGA, local government area; DLIs, disbursement linked indicators; HFs, health facilities; PHCs, primary health centers; HMBs, health management boards.

Among the incentive payments to PBF health facilities, family planning, institutional delivery, curative consultation, HIV/AIDS services, household visits, and vaccination shared the largest portion of incentives, representing 19%, 17%, 12%, 10%, 9%, and 9%, respectively (Figure 2). With the large amount of funding spent on family planning and institutional delivery, these two services are expected to be improved in the PBF facilities.

Figure 2 Distribution of incentive payments.

The World Bank headquarters costs for supervising and planning the program was estimated to be USD 3.03 million (USD 3.06 million in 2015 US dollars).

Compared to the control group, the incremental costs of DFF and PBF were estimated at US$45.2 million (US$2.85 per capita per year) and US$87.3 million (USD$5.49 per capita per year) in 2015 US dollars, respectively, after adjusting for population. These costs did not include the cost of the consumables because most of the cost of consumables was covered by the incentive payments under PBF or additional payments under DFF received by the health facilities.

Table 1 shows coverage of the key maternal health services at baseline and endline from the household surveys. A statistically significant difference was found in Bacillus Calmette–Guérin (BCG) vaccination when comparing NSHIP to control facilities (9.41%). PBF improved SBA, and modern contraception use by 9.1% and 5.7%, respectively, when compared to control group, and the effect was statistically significant. DFF had a major impact on vaccinations, compared to control facilities. The coverage of BCG and DPT improved by 9.73%. Compared to DFF, PBF facilities improved the use of iron supplementations (7.31%), skilled birth attendance (11.0%), but had a worse impact on coverage of BCGs, DPT, and Haemophilus influenzae type B (HiB) vaccines.

Compared to the coverage of health services, the impact of PBF, DFF, and NSHIP on QoC was more noticeable. Compared to control facilities, PBF had a positive impact on the QoC in vaccinations (24.6%), family planning (36.1%), and SBA (34.6%), while DFF improved the quality of all services. Combining PBF and DFF, it showed positive impact of NSHIP on the QoC for all five services. However, there was no statistically significant impact on QoC between the DFF and PBF groups, which was somewhat expected because both programs used similar instruments for measuring and validating QoC in health facilities, and providing coaching and feedback to them based on these metrics.

Without including QoC in the analysis, the DFF and PBF program resulted in saving 2,115 and 4,700 lives compared to the control group from the 5.67 million population in the PBF group (Table 2). The majority of lives saved were children under the age of 5. After the quality adjustment, the number of lives saved increased by four times (8,537) in DFF, and almost tripled (12,488) in PBF compared to the control group (Table 2).

When converting lives saved to QALYs gained, it shows that DFF and PBF, compared to control facilities, saved 50,059 QALYs and 110,896 QALYs, respectively (Table 3) without quality adjustment. These were equivalent to gaining 0.0088 QALYs and 0.0195 QALYs per capita. The number of QALYs gained was larger when the improvement of QoC was included in the analysis.

Table 4 shows ICERs of the DFF and PBF program in comparison with the control group. The ICERs of PBF were $904 and $787/QALY gained, when compared with the control group, respectively, without QoC adjustment. These ratios fell to $224 and $296/QALY gained if the QoC was added. The ICER for NSHIP was estimated to be $822 and $268/QALY gained, without and with quality adjustment, respectively.

The sensitivity analysis of QoC showed that if the effect of QoC measured only half of the impact of the coverage of care, then the ICER of DFF and PBF increased to $430 and $589/QALY gained, compared to the control group, respectively. This shows that the ICER of DFF and PBF was sensitive to the change of quality adjustment. The results should be interpreted with caution.

Discussion

This study shows that NSHIP, consisting of PBF and DFF, improved the utilization of MCH services over the 2.8 years. However, PBF and DFF, compared to the control, impacted MCH services differently. PBF’s impact concentrated on SBA and family planning, while DFF had a positive impact on immunizations with some impact on family planning as well. Despite the heterogenous impact of PBF and DFF, both programs, compared to the control group, are cost-effective, whether the improvement of QoC was included in the analysis or not.

The findings of this study are also consistent with those from Zambia, Zimbabwe, Haiti, and Afghanistan (11,27,28). Institutional delivery is often heavily subsidized under PBF programs. Nigeria’s PBF program paid US$10 per assisted delivery, and thus health facilities devoted more effort to it to earn more financial incentives. The higher attention given to deliveries is demonstrated by the relative higher share of PBF incentives of 19% on this service. Additionally, with Nigeria’s relatively low baseline of SBA usage, there is greater room for improvement. In the control arm, the baseline SBA rate was 64% and was even lower in PBF and DFF arms. With additional resources from PBF, the improvement of usage of SBA is feasible.

PBF and DFF both improved family planning uptake in Nigeria. In addition to the systemic improvements such as robust supervision and training, the improved stock of contraceptives may have contributed to the effect. Indeed, data from the impact evaluation shows that NSHIP improved the stock of at least three modern contraceptives in NSHIP facilities by 28 percentage points as compared to control facilities. The 2015 fertility rate in Nigeria was 5.5 children per woman in her lifetime (2). A study shows that there is a substantial unmet need of contraceptive use in the county, particularly among women in rural areas (29). One of the key reasons for the unmet needs was the shortage of key contraceptive products. A substantial improvement in family planning not only avoids unintended pregnancies, but also reduces health risks for both pregnant women and infants. Provision of high-quality family planning services is regarded as one of cost-effective interventions addressing MCH (16,30,31). Additionally, PBF and DFF health facilities were given the autonomy to procure pharmaceutical products including contraceptives at certified pharmaceutical outlets, which contributed to enhanced availability of these products in the facilities. It should be noted that family planning is not only able to reduce the mortality by increasing birth spacing and maternal age, and reducing abortions, but also lower the total number of live births, which does not necessarily reduce mortality. The combination of such effects reduces the total number of deaths among pregnant women and children under 5 and thus yields savings of QALYs.

Unlike PBF, with its key impact on SBA, DFF has a larger positive impact on the coverage of immunization (i.e., BCG and DPT). Not all immunizations have a major impact on the mortality of children under age five. For example, BCG immunization, instead of having the immediate impact of alleviating mortality among children under 5, it has a long-term impact for adults by avoiding TB infections. Meanwhile, DPT’s effect on under-five mortality is mixed (32,33). In fact, the immunization that offers the greatest impact on under-five mortality is the one for measles. The 4% increase of measles immunization under DFF is one of the key contributors to the cost-effectiveness of the DFF group, along with the improvement of family planning. As the incentive received in DFF facilities could only be used for quality improvement, such as purchasing drugs and commodities, health facilities are most likely to improve those services that do not demand substantial resources (i.e., immunization), in comparison with complex procedures.

However, it is surprising that the coverage of immunization (e.g., DPT and HiB) did not increase in the PBF group as quickly as that in the control group, and the impact estimate was not statistically significant. This may possibly be due to the diversion of the PBF program. In this resource-limited setting (e.g., shortage of human resources), delivering services may compete with each other. Given the substantial improvement in deliveries in the PBF facilities, staff in those same facilities may not be able to devote adequate effort on immunization. In addition, the incentive for immunization is relatively small and needs a long time to retrieve as incentives are given when the full schedule of immunization is completed (24 months). Given the incentive differentials in PBF facilities, if a health facility diverts their resources more on delivery services, it is unlikely that they can spend a similar amount of effort as before on other services (i.e., immunization). Despite the negative impact of PBF on most immunizations, its impact on the number of deaths as estimated by LiST is dwarfed by the improvement of SBA and family planning, also because of limitations of the LiST methodology discussed below.

Our estimation indicates that with the quality adjustment, the DFF program is more efficient in gaining QALYs than the PBF program, as ICER of DFF ($224/QALY gained) is lower than that of PBF ($296/QALY gained) when compared to the control group. This finding is consistent to that in Zambia (11). In health facilities where resources are very constrained, simply providing more financial support would improve the coverage and QoC of health care, and thus save lives of mothers and children. As financial resources are gradually improved, additional actions, such as regular monitoring and evaluation and providing staff bonus for select services that constitute the essential components of PBF, would continue to save lives. The additional interventions remain cost-effective and justifiable, albeit with a lower efficiency.

There is a significant improvement in QoC in PBF facilities on vaccination, family planning and deliveries. For DFF, in addition to these three services, QoC of antenatal care and curative care are also improved. In Nigeria, the QoC was assessed using a balanced score card for each service (i.e., antenatal care, family planning), and the quality was assessed by examining the availability of trained staff, commodities, and the facility’s business plan. The assessment remains mostly on structural quality rather than process quality, except for antenatal care where the QoC was assessed through observing the process of the care. The main quality concern in Nigeria is stock outs of essential medications (4), such as antibiotics. This issue can be addressed in PBF and DFF facilities, where they are able to exercise their autonomy to use their own funds to purchase drugs for QoC improvement. Compared to the impact of PBF on QoC in Zimbabwe (14), Nigeria’s PBF program has a much larger impact, with a more than 20% increase in immunization, FP and SBA. The much lower baseline QoC in Nigeria means that there is much more room for improvement. A relatively small investment in QoC in Nigeria may yield significant improvement.

This study also shows that improving QoC is as critical as improving the coverage of MCH services in saving lives. As the coverage of health services saturates, improving QoC is the only way to improve health outcomes, and in fact, making parallel improvements both to coverage and quality is a surer and more efficient pathway to improving health outcomes. Given the importance of QoC in improving health outcomes, it has been placed in an urgent position in the global health agenda (34). In 2017, the Lancet Global Health Commission on High Quality Health Systems (35) was launched, which aims to promote QoC in low- and middle-income countries (LMICs), and makes a strong case for improving both quality and coverage as countries embark on universal health coverage. Most PBF programs have a QoC component: health facilities are paid based on improvement of both coverage and quality of services. Das et al. synthesized literature on the impact of PBF on QoC and concluded that PBF had a positive effect on quality of antenatal care, but not on other services (9). This study also called for more robust assessments and interventions to address prevalent QoC concerns in LMICs.

It should be noted that even after nearly three years following implementation of NSHIP, key MCH service coverage remains low in Nigeria. For example, coverage of SBA in the three arms remains at about 70%, suggesting that further effort is needed for improvement. A study in Cambodia showed that the combination of demand- and supply-side incentives could substantially improve service delivery (36). Nigeria needs to further investigate causes of low use of essential MCH services, noting that some barriers may have to be addressed from the demand side. Developing appropriate demand-side interventions could enhance the impact of PBF and DFF.

A few limitations of this study should be acknowledged. First, we are not able to quantify the impact of curative care, although curative consultation under PBF increased substantially. Thus, the cost-effectiveness of PBF is likely to be underestimated in this results pathway. A cost-effectiveness analysis of the PBF program in the Republic of Congo examined the impact on more comprehensive services and was more favorable for PBF (37). Second, we are also not able to quantify the lifetime mortality impact of vaccinations as LiST only estimates the impact of vaccinations on under-five mortality. Hence, cost-effectiveness of DFF is likely to be underestimated in this results pathway as DFF had a higher impact on immunization. Third, we found that ICERs were sensitive to the measurement of QoC. A change in impact of QoC on health outcomes would significantly affect the ICERs. Unfortunately, there is little evidence that directly measures the impact of quality care on health outcomes (mortalities and morbidities), and it is unclear based on the evidence whether PBF or DFF had a better impact on improving quality. Additionally, our estimation of QoC for some services such as HIV and maternity services was mostly focused on structural quality indicators (e.g., availability of essential equipment or facility). A more direct measure of QoC on process and outcomes should be developed and its impact on health outcomes should be assessed for future evaluation of such programs. Some attempts to improve the measurement of QoC have been piloted and implemented (38,39). Finally, some baseline indicators among the three comparison groups differed substantially, which suggests a potential selection bias of the LGAs, and thus the results could be biased. However, we used a difference-in-differences approach to adjust the baseline differences, which could, to some degree, mitigate the bias.

Both PBF and DFF are highly cost-effective in improving MCH in Nigeria. The NSHIP project is likely successful due to a package of comprehensive interventions under each program, such as decentralization of funds to health facilities, decision-making autonomy in health facilities supported with improved supervision with community engagement, performance-based approaches, and investments in health systems management. All these interventions reinforce each other and constitute important and indispensable components of an inclusive and integrated health service enhancement program. With the improvement of the coverage of MCH services, improving QoC becomes increasingly critical in future incentive programs aiming to improve MCH outcomes. Developing more valid approaches to assess and integrate the QoC’s contribution to overall return on investment in incentive programs is needed. The lessons learnt from NHSHIP have been translated to the national initiative—Basic Health Care Provision Fund (BHCPF), and both PBF and DFF are included in the BHCPF.

Acknowledgments

We are grateful to Eeshani Kandpal, Damien de Walque, David Evans, Mickey Chopra and Ellen Van De Poel for their helpful review and comments on earlier versions of the work.

Funding: This study was funded by the World Bank.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Journal of Hospital Management and Health Policy for the series “Incentives and Health System Efficiency in Low- and Middle-income Countries”. The article has undergone external peer review.

Reporting Checklist: The authors have completed the CHEERS reporting checklist. Available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-20-82/rc

Data Sharing Statement: available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-20-82/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jhmhp.amegroups.com/article/view/10.21037/jhmhp-20-82/coif). The series “Incentives and Health System Efficiency in Low- and Middle-income Countries” was commissioned by the editorial office without any funding or sponsorship. WZ served as the unpaid Guest Editor of the series and serves as an unpaid editorial board member of Journal of Hospital Management and Health Policy from August 2019 to July 2021. All authors report that this study was funded by the World Bank. The findings, interpretations and conclusions expressed in this paper are entirely those of the authors, and do not necessarily represent the views of the World Bank, its Executive Directors, or the countries they represent. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study used the secondary group level data to perform the analysis. Ethical approval is not applicable.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. National Population Commission, ICF International. Nigeria Demographic and Health Survey 2013. Abuja, Nigeria, and Rockville, Maryland, USA: NPC and ICF International; 2014.
2. World Development Indicators [Internet]. World Bank. 2018 [cited Jan 2019]. Available from: https://databank.worldbank.org/data/source/world-development-indicators.
3. Hafez R. Nigeria health financing system assessment. Washington, DC: World Bank; 2018.
4. Ugo O, Ezinne EA, Modupe O, et al. Improving Quality of Care in Primary Health-Care Facilities in Rural Nigeria: Successes and Challenges. Health Serv Res Manag Epidemiol 2016;3:2333392816662581. [Crossref] [PubMed]
5. Ferderal Ministry of Health, World Bank. Impact Evaluation of the Nigeria State Health Program Investment Credit. Washington, DC: World Bank; 2011.
6. The World Bank. Nigeria state health investment project (NSHIP): Qualitative study on key differentiating factors for performance under performance based financing (PBF) approach. Washington, DC: World Bank; 2015.
7. Fritsche G, Soeters R, Bruno M. Performance-based financing toolkit. Washington, DC: World Bank; 2014.
8. Turcotte-Tremblay AM, Spagnolo J, De Allegri M, et al. Does performance-based financing increase value for money in low- and middle- income countries? A systematic review. Health Econ Rev 2016;6:30. [Crossref] [PubMed]
9. Das A, Gopalan SS, Chandramohan D. Effect of pay for performance to improve quality of maternal and child care in low- and middle-income countries: a systematic review. BMC Public Health 2016;16:321. [Crossref] [PubMed]
10. Witter S, Toonen J, Meessen B, et al. Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation. BMC Health Serv Res 2013;13:367. [Crossref] [PubMed]
11. Zeng W, Shepard DS, Nguyen H, et al. Cost-effectiveness of results-based financing, Zambia: a cluster randomized trial. Bull World Health Organ 2018;96:760-71. [Crossref] [PubMed]
12. Kandpal E, Loevinsohn B, Vermeersch CM, Pradhan E, Khanna M, Conlon M, et al. Impact evaluation of Nigeria State Health Investment Project. Washington, DC: World Bank; 2018.
13. Zhou H, Taber C, Arcona S, et al. Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups. Appl Health Econ Health Policy 2016;14:419-29. [Crossref] [PubMed]
14. Shepard DS, Zeng W, Mutasa R, et al. Cost-effectiveness of results-based financing of maternal and child health services in Zimbabwe: a controlled pre-post study. J Hosp Manag Health Policy 2020;4:32. [Crossref]
15. Avenir Health. Spectrum manual: spectrum system of policy models. Glastonbury, CT: Avenir Health; 2015.
16. Stenberg K, Axelson H, Sheehan P, et al. Advancing social and economic development by investing in women's and children's health: a new Global Investment Framework. Lancet 2014;383:1333-54. [Crossref] [PubMed]
17. Boschi-Pinto C, Young M, Black RE. The Child Health Epidemiology Reference Group reviews of the effectiveness of interventions to reduce maternal, neonatal and child mortality. Int J Epidemiol 2010;39:i3-6. [Crossref] [PubMed]
18. Singh S, Darroch JE, Ashford LS. Adding it up: the costs and benefits of investing in sexual and reproductive health 2014. New York, NY: Guttmacher Institute; 2014.
19. Keats EC, Ngugi A, Macharia W, et al. Progress and priorities for reproductive, maternal, newborn, and child health in Kenya: a Countdown to 2015 country case study. Lancet Glob Health 2017;5:e782-95. [Crossref] [PubMed]
20. Sassi F. Calculating QALYs, comparing QALY and DALY calculations. Health Policy Plan 2006;21:402-8. [Crossref] [PubMed]
21. World Health Organization. Global health observatory data repository. Geneva, Switzerland: WHO; 2020. Available online: http://apps.who.int/gho/data/view.main.61850?lang=en
22. GBD 2016 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life-years (DALYs) for 333 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 2017;390:1260-344. [Crossref] [PubMed]
23. Marseille E, Larson B, Kazi DS, et al. Thresholds for the cost-effectiveness of interventions: alternative approaches. Bull World Health Organ 2015;93:118-24. [Crossref] [PubMed]
24. Bertram MY, Lauer JA, De Joncheere K, et al. Cost-effectiveness thresholds: pros and cons. Bull World Health Organ 2016;94:925-30. [Crossref] [PubMed]
25. Neumann PJ, Cohen JT, Weinstein MC. Updating cost-effectiveness--the curious resilience of the $50,000-per-QALY threshold. N Engl J Med 2014;371:796-7. [Crossref] [PubMed]
26. World Bank. GDP per capita (current US$). Washington, DC: The World Bank; 2015. Available online: http://data.worldbank.org/indicator/NY.GDP.PCAP.CD
27. Zeng W, Cros M, Wright KD, et al. Impact of performance-based financing on primary health care services in Haiti. Health Policy Plan 2013;28:596-605. [Crossref] [PubMed]
28. Engineer CY, Dale E, Agarwal A, et al. Effectiveness of a pay-for-performance intervention to improve maternal and child health services in Afghanistan: a cluster-randomized trial. Int J Epidemiol 2016;45:451-9. [Crossref] [PubMed]
29. Austin A. Unmet contraceptive need among married Nigerian women: an examination of trends and drivers. Contraception 2015;91:31-8. [Crossref] [PubMed]
30. Carvalho N, Salehi AS, Goldie SJ. National and sub-national analysis of the health benefits and cost-effectiveness of strategies to reduce maternal mortality in Afghanistan. Health Policy Plan 2013;28:62-74. [Crossref] [PubMed]
31. Goldie SJ, Sweet S, Carvalho N, et al. Alternative strategies to reduce maternal mortality in India: a cost-effectiveness analysis. PLoS Med 2010;7:e1000264. [Crossref] [PubMed]
32. Aaby P, Mogensen SW, Rodrigues A, et al. Evidence of Increase in Mortality After the Introduction of Diphtheria-Tetanus-Pertussis Vaccine to Children Aged 6-35Months in Guinea-Bissau: A Time for Reflection? Front Public Health 2018;6:79. [Crossref] [PubMed]
33. Higgins JP, Soares-Weiser K, López-López JA, et al. Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review. BMJ 2016;355:i5170. [Crossref] [PubMed]
34. Scott KW, Jha AK. Putting quality on the global health agenda. N Engl J Med 2014;371:3-5. [Crossref] [PubMed]
35. Kruk ME, Pate M, Mullan Z. Introducing The Lancet Global Health Commission on High-Quality Health Systems in the SDG Era. Lancet Glob Health 2017;5:e480-1. [Crossref] [PubMed]
36. Van de Poel E, Flores G, Ir P, et al. Impact of Performance-Based Financing in a Low-Resource Setting: A Decade of Experience in Cambodia. Health Econ 2016;25:688-705. [Crossref] [PubMed]
37. Bitran R, Munoz R, Fritsche G, et al. Cost-effectiveness analysis of performance-based financing for the delivery of a health benefits package in the Republic of Congo—HRBF impact evaluation. Washington, DC: World Bank; 2020.
38. Fritsche G, Peabody J. Methods to improve quality performance at scale in lower- and middle-income countries. J Glob Health 2018;8:021002. [Crossref] [PubMed]
39. Peabody JW, Oskombaeva K, Shimarova M, et al. A nationwide program to improve clinical care quality in the Kyrgyz Republic. J Glob Health 2020;10:020418. [Crossref] [PubMed]