The Good, the Bad, and the Disruptive of Performance- Based Financing on the Mozambican Health System: Results from a process evaluation

A significant debate is unfolding around whether performancebased financing (PBF) is a mechanism of achieving broader health systems transformation. This study aims to contribute to this dialogue by assessing how PBF fostered positive, perverse and disruptive effects on the health system in two provinces of Mozambique. The study used qualitative methods to collect data in 24 PBF health facilities from 60+ health workers and facility administrators. PBF improved the facility’s work environment through improved local financial capacity and autonomy, resulting in greater planning. Health workers perceived incentives as a source of motivation, however the allocation of incentives among staff as unfair and lacking transparency. Major improvements in data quality and completeness was observed in facility registers, verified quarterly. However, a heavier workload and enhanced focus on information systems were disruptive to time spent on clinical care. PBF remains a health systems strengthening intervention that results in targeted and program-driven changes that are not yet institutionalized or uniformly applied. Sustaining positive effects will requires greater focus on institutionalizing changes to governance, management structures, and financial autonomy, while enhancing inclusiveness of the demand-side.

A significant debate is unfolding around whether performancebased financing (PBF) is a mechanism of achieving broader health systems transformation. This study aims to contribute to this dialogue by assessing how PBF fostered positive, perverse and disruptive effects on the health system in two provinces of Mozambique. The study used qualitative methods to collect data in 24 PBF health facilities from 60+ health workers and facility administrators. PBF improved the facility's work environment through improved local financial capacity and autonomy, resulting in greater planning. Health workers perceived incentives as a source of motivation, however the allocation of incentives among staff as unfair and lacking transparency. Major improvements in data quality and completeness was observed in facility registers, verified quarterly. However, a heavier workload and enhanced focus on information systems were disruptive to time spent on clinical care. PBF remains a health systems strengthening intervention that results in targeted and program-driven changes that are not yet institutionalized or uniformly applied. Sustaining positive effects will requires greater focus on institutionalizing changes to governance, management structures, and financial autonomy, while enhancing inclusiveness of the demand-side.

Description of PBF Pilot Scheme
In Mozambique, the PBF implementer (Elizabeth Glazer Pediatric AIDS Foundation (EGPAF)), financed by the US President's Emergency Plan for AIDS Relief (PEPFAR) through the Centers for Disease Control and Prevention (CDC), initiated a PBF scheme to accelerate the achievement of maternal and child health (MCH) and HIV-focuses health outcomes in two provinces, Nampula and Gaza. The program began with 19 health facilities in 2011 and was incrementally phased up to 138 health facilities as of mid-2015.
The program established separation of functions. The implementer served as the purchasing agent and fund-holder, while verification of quantity and quality of care is coled by the Provincial Health Office (DPS) and implementer field staff through quarterly visits to each PBF facility (public government-managed workers). The package of services under the PBF scheme is linked to health facility performance across 21 clinical indicators including prevention of mother to child vertical transmission (PMTCT), pediatric HIV, adult HIV care and treatment, tuberculosis, and maternal and child health (MCH). The scheme also includes three quality checklists on hygiene and sanitation, maternal and child health (MCH), and HIV/AIDS. Each facility was mandated to establish a managerial committee in charge of planning and managing PBF funds with guidance from the managing NGO (implementer) and district health office (DPS).
Of the performance payments to facilities, 40% is allocated for reinvestment into the health facility per a pre-approved business plan, while 60% is distributed as salary topups to all health facility staff according to pre-determined criteria, including tenure at the facility, level of education, attendance, and staff performance review. On average, staff bonuses represent 15-25% of a monthly salary for health staff with the highest baseline salaries or those working in larger facilities, and 30-50% of the monthly salary for lowerlevel staff with low baseline salaries or for rural staff in small facilities.

Methods
To gain an in-depth understanding of the perceptions of health workers and facility managers of the effects of PBF, we used a qualitative research design using in-depth semi-structured interviews and focus group discussions. We triangulated the findings with quantitative verification data. We measured the effect that the PBF program had on the health system, namely service delivery, human resources for health, health financing and governance.
We defined these four health system pillars using Witter's framework (Witter et al., 2013). Witter et al. (2013) posits that the interaction between incentives, institutional changes, accountability efforts, and supporting mechanisms for health workers impacts the organization and performance of the health system (Witter et al., 2013). The resulting framework articulates that all health system building blocks are likely affected by the introduction of PBF, but for simplicity modifies the six into four health system pillars: service delivery (including commodities), human resource, governance (including information systems) and health financing. More recently, the boarder definition and conceptual discourse posited by Renmans et al. (2017) and Soucat et al. (2017) these four of those categories in their definitions of PBF (Soucat et al., 2017;Renmans et al., 2017).
The health workers were asked to describe changes they observed within the health facility since the introduction of PBF and to reflect on the positive, negative, or null effects of PBF across the four pillars. Data was collected at 23 health facilities in Nampula (12) and Gaza (11), 73% of which were primary health centers and 82% which were classified as rural or peri-urban. We selected a range of facilities based on size, geographic location (rural, peri-urban, urban), and facility performance data (verified indicator quantities and quality checklist scores). Each facility had participated in the PBF program for a minimum of 6 quarters (18 months).
All interviews were conducted and transcribed in Portuguese, then imported into ATLAS.ti Version 1.0.14 for thematic analysis. Interviews were coded line-by-line and The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

77
analyzed by the two researchers to identify recurrent themes and variations across responses. Initial themes and sub-codes were organized based on our theoretical model ( Figure 1).

Quantification Analysis
Magnitude of effect is a proxy measure to estimate the perceived effect of each change and its attribution to PBF. Magnitude was estimated by the frequency of the code and a coding analysis exercise which assessed and scored each quotation by attribution of the change to PBF (positive and negative, and strength on a three-point scale), saliency and depth (explanation of the change, specificity of the results and self-perceived strength of the change compared to before PBF or in a facility without PBF), other explanations or causes of the observed change (other interventions, contextual changes). While we are aware of the concerns and publications on causality of qualitative work, this analytical technique was used to help organized the top perceived influences of PBF, what was observed and discussed the most by health providers and managers (Maxwell, 2004). We acknowledge that the perceived changes were influenced by PBF, but that contextual components, environmental readiness and other interventions contributed to the changes observed by respondents. Parsing apart attribution to PBF is not achievable with this particularly study design.

Characteristics of respondents
In each facility, health workers (nurses, medical technicians, and physicians), community health workers and facility administrators were sampled based on their exposure to PBF (minimum of nine months) and availability given we collected data through unannounced facility visits during normal clinic hours. Sixty health workers and facility managers were interviewed individually and a total of 30 health workers participated in focus group discussions (Table 1). Health workers typically participated in the PBF program for an average of 9.5 quarters (2.4 years). Semi-structured interview and group discussion guides were used during the interviews, allowing for open dialogue and discussion about the effects of PBF on health system components including service delivery, financing, governance, human resources, access to medicines, and information systems. Data was collected by two Mozambican researchers between November-December 2015.

Geographic Region
Urban 2 (19%) 2 (17%) 4 (18%) Rural 9 (81%) 10 (83%) 19 (82%) Abbreviations: Q, quarters, equivalent of 3 months. Caption: The majority of respondents were medical technicians, who assume some managerial role, in addition to diagnosis, treatment, and counseling of all patients, but particularly HIV patients. The second most common respondents were nurses, falling within three professional designations. A head nurse is typically in charge of many of the managerial activities including the completion of the patient registry books. General nurses concentrate on providing wide range of care prescribed to patients by clinician. MNCH nurses focus on providing preventative and curative care to maternal and child patients, including antenatal care, assisting in deliveries, immediate post-partum care, and contraceptive counseling and prescription. The main function of the lay counselors or Activistas is to provide counseling and referral information for patients in their assigned communities, with special emphasis on HIV+ patients. Information included access points to community support groups (GAACs) and helping patients to find pharmacies to access ARVs. Facility management activities are shared between the facility-based cadre, particularly when the facility is small. The three facility managers listed in Table 1 had no clinical activities and were employed by peri-urban large hospitals.
The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

Results
Perceived changes in health system attributed to PBF The perceived changes associated with PBF, good, bad and disruptive, varied across the health system pillars. Figure 2 depicts the aggregated quantification results as a heuristic tool to visualize the magnitude of change perceived by facility staff. Figure 3 shows the results from sub-categories within each pillar. The most prominent positive or good changes include infrastructure and available equipment funded by the facility reinvestment funds, integration of care efforts (patient double dipping), information system (quality and completeness of the data), and facility and provider accountability through the verification process ( Figure 3). Disruptive changes include task shifting to complete the registry, managerial planning and fiscal autonomy. Inequitable incentive distribution, lack of results-based planning, transparency and communication, and low awareness and buy-in for quality of care checklists affiliated with PBF were perverse changes ( Figure 3). The sections below detail our findings from each health system pillar ( Table 2 includes illustrative quotations for each).

Infrastructure & Equipment
"it would take fifteen years for each facility if we did not have the PBF...in the case of the porch, these comfortable chairs, upgraded bathrooms, yes electrification because we know that a district has whenever a budget is never enough to meet all our needs , but the PBF has reduced a portion of these costs but , for me has great impact , not because it improves the health of structures and a portion of its expenditure rather than to me it has a great impact because it improves the health and also improves working conditions for health professionals infrastructure , yes" -IDI34_Nampula

Integration of Care
"For my area, I make monthly meetings with APEs, APEs work in the community, and even the traditional midwives on a monthly basis to coordinate maternity patients and bring the data on the days they (need to) deliver. We have a meeting that is on the 25th and 15th of each month and we enjoy communicating all the activities that we are going to carry out each month, we communicate to them to make the social mobilization in the communities and we also have (the) Management Committee participate.. We also meet with community leaders and community leaders."-IDI37_ Nampula "Now we are functioning as a The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

81
Quality of Care "Yes, to respect patients…we ask(ed) for a porch because we have a space where patient can be comfortable, for example use it for shade…patients easily get bored in the queue and can get tired because it can be exposed to the sun and can feel worse than at home…. any discomfort to a patient who is not well creates more discomfort and the patient may give up and find that this is not worth it to come to us." -IDI02_Gaza "Good behaviors have changed in general is this that recently talked ... sometime colleagues needed to be bribed, and was (delivering) a bad service, patients had long waits in the queue. So with the PBF, the employee arrives early and on time, they are attentive, they care for the patient, so these are the behaviors that have changed." -IDI35_Nampula "yes because I know that I ... this indicator I practically want to sell, it's my fish I want to sell, I take care of the best possible way it is to be able to buy, if I do not care if this fish has a little ... get in a little sand so nobody will buy, so I do everything that is for the indicators to be the same ones of quality, which is to be able to have a benefit." -IDI12_Gaza

Information System
Data generation "Yes there is improvement because since we knew that the books are visited regularly, now there is great care in filling the books, we improved enough because even the communication between us at the facility level in terms of registration book. There were colleagues before who had problems with data collection because it is difficult to find time. But now because we learned that the book should be visited by us constantly even before the supervisor comes, so there was an improvement, yes, yes" -IDI09_Gaza "We always vaccinated (children) but we did not record all those children we vaccinated and the data that we sent was not accurate...not that it wasn't real ... it was done…..we vaccinated the children, but at some point do not we record, and sometimes we record, but not correctly... before ... it was normal to not keep ... make a bad record, but with this PBF check the fight is to always register properly." -IDI60_Nampula

Analysis and Communication
"During supervision sessions they check how we were in terms of indicators and when they would notice that there is an indicator that we are still one little weak, they explained in the sense of how to improve these indicators, but this in not frequent and is mainly the Management Team." -IDI21_Gaza "We have performance graphs for the government indicators like vaccination, but the PBF graphs are never made or posted for us, so we cannot track our performance. Most of us don't know how the other units are doing for their performance, only our own unit because we do the registers." -FGD39_Nampula Health Workforce Distribution "No there has not been changes to the number of clinical staff at the facility, even though PBF has brought us more work. Our facility does have more Activistas now, working more days, but our staff is the same." -IDI7_Gaza Remuneration "Then they end up being paid as if they were working for the PBF, but there are those who are connected with the indicators and have to work harder weekly or daily on what they are doing in relation to the indicators, and that effects the payment based on performance. Each should reap what they sow." IDI2-Nampula " Well, tension can exist that I work a lot I have the books but you do nothing but when the time comes for encouragement you also want to, but let's see what you did, you did nothing. So it ends up doing what in one way or another those who are not doing, those who are not directly connected work, do other things that can also benefit the health unit's own performance. So they end up being involved as if they were working for the PBF" -IDI31_Nampula "The incentive is very good for us, all of us will have more to bring to our family and to support us. We see that it helps all of us and because the salary is never enough it has changed our family life." -IDI19_Gaza Training "Everyone receives at least one training. EGPAF comes to the facility and provides a day or two of training for PBF, explaining the formulations of PBF, and sit for example and review some data or make some tips for improvement" -IDI13_Gaza Supervision "They (EGPAF) come in helping ... supportive and coaching, help us data also has given some technical support for PBF, often check the books ... are well filled or try to fix together with us... mostly it is supervision for the data verification"-IDI45_Nampula Communication "Communication between units has changed. Communication is better. If there is something we think is not (working) well we try to correct the colleague, but positively and not in a way that may insult you…We try to encourage one another and if something is not going well we will solve the situation together, for instance in our meetings we discussed seeing bad behavior maybe from someone or something bad that is happening and are able to discuss and come up with a solution." -IDI33_ Nampula

Governance
Leadership "Managers are more committed. They are working more, meeting with each other and taking important decisions. He is traveling to district and other places to do his duties." -IDI26_Nampula "We received a new head of management committee and there is now more organization and we feel like they are working as a good representation at district and provincial government." -IDI24_ Gaza Transparency "Little, very little announcements or information is shared by our management committee. We are unsure about how our unit performed, but know that in general we are performing well. Although, how we do compared to other health facilities we do not know. We could be the best or the worst. I don't think the worst, but only the committee or DPS will know." -IDI35_Nampula Performance Monitoring "So for the month we prepare for the verification team. We know they come, they show up on time every quarter, and we meet with them a lot during their visit" -FGD01_Gaza "When EGPAF comes the facility is prepared and we feel we want to perform and provide a good service to them and the patients" -IDI22_Gaza

Health Finance
The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

83
Autonomy "We did not have septic tanks, and we built those pits with the PBF money, for example, when we wanted to make copies of pre-natal records, we use to depend only on the district direction, but now we bought a printer and we created a better space. We started a group to do searches for missing patients, we did it with our PBF money ... that 40%, we did it with that our PBF money and we decided... we depended 100% of the district direction before, but now we decide."-IDI43_Nampula Fiscal Planning "Planning has a space of two more days to bring ideas of what we need to improve more in the health unit. So there is this space where colleagues, especially the management commission, asks the needs of each sector to see if we can improve more and more. There is this space, it may not be exactly in a collective forum but it is sometimes done with the heads of the sectors, they survey the needs and get answers, yes." -IDI25_Gaza "Yes, I am on management board. We meet once a month, or we try. We have to submit a plan for our 40% bonus…how will it be used to improve the facility and quality of care." -IDI14_Gaza "Almost every quarter is a right plan, after receiving the PBF, the necessary things in so sectors sees the priorities and buy it, get it ourselves. One

Infrastructure & Equipment
Every health provider interviewed identified indirect or direct changes observed in facility infrastructure or equipment to PBF (Figure 3). Initially, PBF funds were spent on acute facility needs, like replenishing hygiene and waste disposal supplies, repainting walls, repairing and replacing broken equipment, or repairing infrastructure (e.g. windows, doors, benches). As the PBF program continued, investment was made in large facility improvements like septic tanks, new storage facilities, or electrical rewiring. Facility investments were dictated by the health facility management committee's business plan submitted to the district health office (DPS) quarterly and required approval before funds were received. Purchasing medicines and pharmaceuticals with PBF funds was not permitted.

Integration of care
Health workers cited feeling a greater sense of coordination in the care they were providing to patients, particularly maternal patients, mothers living with HIV, and infants born to mothers living with HIV. Integration or continuity of care was described as ensuring that patients who were already at the facility received all services or counseling they were eligible for, referred to by one health workers as 'patient double dipping'.
Integration of PBF services for mothers without HIV included: antenatal care (ANC) visits, facility delivery, post-partum consultation, and family planning (FP) counseling, in addition to timely vaccination for the newborn within 9-12 months and nutritional counseling. For mothers living with HIV, PBF incentivized integrated care through the testing of their partner, antiretroviral (ARV) prophylaxis, antiretroviral therapy (ART) initiation during pregnancy, polymerase chain reaction (PCR) testing within four to eight weeks of birth, rapid HIV testing within nine to 12 months, ART initiation for newborns 0-23 months, and FP counseling after birth. The integration of care was an initiative discussed at monthly or quarterly PBF meetings.
The results of the impact evaluation show significant increases in patients receiving care for PBF indicators compared to control groups (magnitude of effect).Most notably for the services that facilities reported integrated services (Rajkotia et al., 2017).

Outreach increases coverage
Community outreach efforts were amplified by PBF. Activistas, a cadre of community health worker, were reportedly used for outreach and follow-up activities to attract or retain patients, particularly for maternity services and maternal and pediatric patients living with HIV. This strategy was cited as an explanation for the increase in performance on maternal health, PMTCT and pediatric HIV indicators (Figure 4). One The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

85
strategy employed by facilities was to use PBF funds for fuel or repairing vehicles so that Activistas could visit priority patients to ensure they were properly counseled and aware of when they needed to return to the facility. The changes in coverage were not spurred by increased supply-side access (additional operating hours or new clinical services by the health facility).

Quality of care
Health workers perceived improvements in quality of care via two key observations: reduced waiting times and improved patient comfort. The driver for lower wait times was the adoption of attendance tracking through mandatory sign-in and other mechanisms, increasing health provider accountability and limiting absenteeism. In an effort to increase patient comfort, some facilities used PBF funds to purchase food and water to make the patients and family feel more comfortable during overnight or long stays, and capulanas, decorated cloth worn as a wrap skirt, for women to wear home after giving birth.
The provider's definition of quality contradicts what is measured as performance on PBF quality checklists. The PBF program included three quality checklists, predominately focused on measuring structural attributes of the facility (e.g. presence of supplies, cleanliness of exam room) (Gergen et al., 2017;Josephson et al., 2017). The use and completion of the three PBF quality of care checklists by facilities was rare. Provider's explained that the checklists remained relatively static (same checklists with the same indicators), so if they knew the facility would not score greater than the checklist threshold (>60%), then they not to expend the human resources to complete the checklists and review it with the verifier during quarterly visits. Fewer than 15% of health facilities received a quality bonus from 2014-2015 (University Research Co. and ThinkWell July, 2016).

Data generation
A large majority of health workers reported that patient registries were more complete and the quality of data improved as a response to PBF (Figure 3). Workers felt a responsibility and pressure from managers and facility administrators to complete entries. Prior to PBF, completing facility registers was left until the end of the week, or even month, and the work was not well distributed among service workers. A timelier registry did, however, require significant disruption in task shifting by clinicians away from clinical activities and toward registry completion, and there were reported discrepancies between verified numbers and those submitted by health facility managers in a few instances.
Analysis & Communication Health workers do not have the capacity or resources to analyze PBF performance data. According to administrators, performance data was typically only known and understood by facility management participating in the management committee, which disrupts the link between enhanced performance and payment. Analysis, including performance forecasts and strategies to increase quantity and quality of PBF services based on past performance, was reportedly only shared with health workers in four of the 24 facilities. In a small number of facilities, graphs of PBF indicator performance were posted to a bulletin board so that health workers were able to review the information. Health workers reported that these visuals were helpful, but that, without strategy meetings and discussion on how to improve current performance, this type of information sharing did little to initiate service or behavior changes.
During quarterly meetings, the verification team often reported on performance of select indicators, but this analysis was incomplete and was not used as a mechanism to stimulate performance improvements by health workers. In general, there was a low level of awareness amongst health workers on facility-level performance of indicators they were not directly responsible for and little discussion of how to improve performance based on historical analyses.

Human Resources
Distribution Health workers and administrators perceived little change in the distribution of health workers to PBF facilities (Figure 3). Intervention facilities were reportedly favored by health workers and formal requests for transfers were common. Anecdotally, health workers felt their colleagues had stayed at PBF facilities longer than they would have at non-PBF facilities (retention), however quantitative data concerning human resource retention rates for PBF and non-PBF facilities is not available.

Remuneration
Salary top-ups or bonuses were perceived very positively. Health workers often spoke of incentives as a source of motivation and freedom to support their families. The salaries of health workers in Mozambique remain low, on average (2014) a nurse will receive between 4000-7000 MTs (USD 60-105) per month, a medical technician will receive 7000-9000MTs (US$105-138, double the GDP per capita) per month, depending on tenure, education level, and other adjustments. Many health workers mentioned that, in the absence or withdrawal of PBF, their base salaries would be insufficient to support themselves and their families. In some cases, the base salaries provided by the provincial budget were late by two or three quarters and the PBF bonuses served as the entire salary.
However, the distribution of incentives was often contentious and source of mistrust among colleagues. All health facility staff received a 'piece of the pie' even if they do not work on PBF indicators, including non-clinicians (e.g. janitors, administrative staff). In a third of the facilities, the PBF incentive amount received by each health provider was posted on the health facility bulletin, but this made the sense of unfairness more apparent, The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

87
and created further tension between health workers in different professional positions and service types.

Training and skills building
Neither additional clinical training nor improved adherence to clinical guidelines were reported, however, each provider received the training, which detailed the programmatic components and purpose of the PBF program, at least once. Every participant could describe PBF and the basic tenets of the program including the reporting, verification, and incentive distribution percentages, however components coordinated by the management committee like the quality checklist, payment formula or planning for facility funds were either not communicated or retained by workers.

Supervision
The presence of district supervisors or utilization of supportive supervision techniques was not affected or promoted by PBF. Verification, as one form of supervision and perceived as a strong accountability mechanism, changed significantly. Workers felt that their achievements were measured and recognized through verification.
Performance reviews were designed to be a supervisory management tool. These reviews were intended to be conducted by facility management on an annual basis as part of the criteria for individual PBF payment calculations. The application and use of individual performance reviews was scattered and few health workers recall having one completed for themselves or their colleagues.

Leadership & Transparency
Health workers perceived notable changes in facility leadership with relation to the monthly and quarterly committee meetings (only inclusive of senior facility management). Each participating facility must have an active Health Facility Management Commission (HFMC) responsible for reviewing the facility performance (quality of care and quantity of services) quarterly, resolving problematic management and policy issues, and lobbying for and assisting with resource mobilization. Although health workers were aware that the facility management committees developed a business plans quarterly, they were not part of these discussions nor decision-making processes about facility investments and strategies for increasing PBF services.
Enhanced transparency through participation and communication about the PBF to communities, was not reported. Transparency of facility-level performance data to external institutions was also lacking. For instance, PBF data was not available for facilities to compare nearby facilities' performance, for programs and researchers from partner NGOsto view, nor available to national government or a global audience through existing data platforms like the RBF portal.( The World Bank Group, <https://www.rbfhealth.org/>).

Performance Monitoring
The verification process is the most pertinent change in performance monitoring at the facility level (Figure 3). The reliability and timeliness of the verification process conducted by the implementing NGO and DPS increased the amount of pressure and accountability the facility staff felt towards preparing a complete and accurate registry. Although penalties for false reporting were rare, health workers were aware of this possible adverse event. Although the verification team did not present or broadly discuss the facility's performance outside of the facility management team, the frequency and time spent at the facility every quarter served as an effective performance monitoring technique.

Resource allocation
Financial autonomy was perceived as a key change and driver of program impact. Facility administrators and managers (typically chief nurses and senior medical technicians) had a positive disposition toward having the autonomy to manage their own 'pot' of money based on a process of strategic planning and coordination with the provincial health office and implementer. Conversely, staff at non-autonomous facilities, where district health offices receive the facility's funds and then distribute the staff incentives and facility investments, felt that the districts often delayed their incentives and that they did not receive the full amount earned.
Health workers noted an increase in resource allocation with the introduction of a more robust and deliberate planning process for facility investment and spending. The increase in financial resources due to the PBF facility funds resulting in the physical changes in infrastructure and supplies is the most frequently cited effect. For many of the rural and/or small facilities, the PBF incentive for facility reinvestment is 40-50% of the budget received by the province/central level through input-based financing. Hence, the impact PBF money had on health facilities purchasing power was perceived as very significant.

Discussion
As more donors and country governments in LMICs start embracing PBF as an approach to boost performance in the health sector, there have been repeated calls for rigorous evaluations to grow the evidence base on PBF (Eldridge;Palmer, 2009;Holvoet;Criel, 2017;Renmans et al., 2016b;Paul;Renmans, 2017).
The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

89
While the body of research on PBF has grown significantly in the past few years, recent reviews of the literature have found that, firstly, the evidence about the impact of PBF programs on health system outputs and health outcomes is mixed and, secondly, a lot remains unknown about how PBF acts on the health system Renmans et al., 2016a;Paul;Renmans, 2017). This study speaks to the second observation, using a PBF program in Mozambique to understand how PBF influences actors, institutions, and processes that comprise the health system. The approach we adopted of interviewing health workers to capture their perceptions is similar to what others have used to explore key aspects of PBF programs, such as the implementation process and the effect of PBF on staff motivation (Bhatnagar;George, 2016;Kalk;Paul;Grabosch, 2010). Our application of evaluating PBF across the entire health systems pillars, as defined by Witter et al. (2013), about the changes attributed to the PBF program is novel. While other articles have documented the perceived effects of PBF interventions on individual building blocks like service delivery, (Bonfrer;Van de Poel;Van Doorslaer, 2014;Chimhutu et al., 2015;Skiles et al., 2015;Flink et al., 2016;Kalk;Paul;Grabosch, 2010;Paul, 2009;Bhatnagar;George, 2016) human resources for health, (Matsuoka et al., 2014;Engineer et al., 2016;Lohmann et al., 2017a;Lohmann et al., 2018) and health information system, (Rusa et al., 2009;Kalk;Paul;Grabosch, 2010) no study to date has used a comprehensive health systems lens to systematically explore the perceived effects of PBF on all pillars.
The results from the study show that health workers reported PBF as having had good, bad and disruptive perceived effects on all pillars at variable magnitudes. Overall, the PBF scheme was perceived to have had the greatest effect in the areas of service delivery, health workforce, and information systems. It had more modest effects on health financing and governance, and no effect on the supply of pharmaceutical commodities (by design).
In the area of service delivery, health workers believed the PBF program has resulted in an expansion of service coverage, which is not surprising given that an impact evaluation for the PBF program in Mozambique found increases in a range of health outputs, (Rajkotia et al., 2017) and other evaluation of PBF pilots that showed similarly positive perceived effects on healthcare utilization (Basinga et al., 2011a;Bonfrer et al., 2014;Engineer et al., 2016). Health workers felt the PBF program had strengthened integration of care and the nature of the interaction between health workers and patients. This is similar to findings in Rwanda and Cambodia, where PBF schemes led to an improvement in the interaction between health workers and patients (Kalk;Paul;Grabosch, 2010;Khim;Annear, 2013;Skiles et al., 2015). What emerged most notably is how much health workers appreciation for the infrastructure improvements that health facilities have been able to undertake thanks to the PBF scheme. Health workers were largely unaware of the PBF quality components, including the checklists themselves, and the payment formula based on the facility's quality score. Quality checklists also need to be more process driven, measuring multiple components of quality, including process and outcomes of care. Understanding advances in maternal and child health quality efforts may be a great source of cross-learning, especially because most PBF programs focus on quality of care in MCH (Gergen et al., 2017;Josephson et al., 2017).
In the domain of human resources for health, the PBF payments reportedly increased health worker motivation. This finding is in line with other studies that showed documented positive responses by health workers to the increase in their remuneration (Kalk;Paul;Grabosch, 2010;Bertone;Lagarde;Witter, 2016;Skiles et al., 2015;Lohmann et al., 2017b, Lohmann et al., 2017a. PBF was also credited for improving staff retention rates, as was the case in Cambodia (Matsuoka et al., 2014). However, health workers in Mozambique expressed concerns about the fairness of how the PBF funds were distributed amongst health workers, which echoes concerns by health workers in Tanzania (Chimhutu et al., 2016), Cambodia (Khim;Annear, 2013), DRC (Fox et al., 2014) and Benin (Paul;Sossouhounto;Eclou, 2014). Unlike other PBF programs, where the incentive to improve performance led managers to increase training and supportive supervision for health workers (Bhatnagar;George, 2016), study participants in Mozambique reported receiving limited clinical training and supportive supervision external to the PBF verification and managerial committee meetings. This finding reiterates the need to integrate broader supervision, such as supportive supervision mechanisms, within the PBF design to further enhance facility and provider performance, clinical adherence, patient-provider communication and respectful care. PBF did not change the composition within the facilities, which is likely due to the fact that distribution of health workers is decided by the provincial and central government based on catchment population estimates.
The PBF program strengthened the health information system at the facilitylevel. Similar positive effects of the PBF program on health information captured at the facility level have been reported in Burundi (Peerenboom et al., 2014). However, these improvements came at a cost. Faced with a trade-off between delivering more clinical services and spending time maintaining the register books, health workers reported shifting their focus from the former to the latter because of the PBF verification. There were also notes on discrepancies between verification and reported numbers. Health workers were not concerned about the discrepancies because although penalties were intended to be applied, no health facility ever received penalties during the program's tenure. Instead the verified would note it and warn the health facility manager. This is similar to Rwanda, where Kalk;Paul;Grabosch (2010) found evidence of task-shifting towards record maintenance, and in Burkina Faso (Kalk;Paul;Grabosch, 2010;Turcotte-Tremblay et al., 2017) Further, the PBF program was credited with improving some governance and managerial practices at the facility-level, especially frequent business planning meetings. However, the absence of monthly PBF performance review meetings and the perceived unfairness in incentive distribution made staff and team relations tense at some facilities. Similar to the experience in Cambodia, the PBF and operational cost disbursement were faster than the MoH budget endowment, ensuring timely disbursement of the incentives on the basis of their performance and strengthened capacity for financial management The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).

91
With respect to health financing, health workers credited the PBF program with increasing the fiscal autonomy. This resulted in greater planning at the facility-level on how the PBF funds would be reinvested into the facility (De Allegri et al., 2018). However, these decisions were made largely by the facility managers, with little input from the health workers. Most health workers interviewed were absent from the allocation and resource mobilization processes but keenly aware of the results of these investments, namely the structural and equipment changes. This concept of performance-based financing was broadly and shallowly understood by nearly all health workers, and the link between taking action and planning for increasing service in order to get a bigger bonus was not as clear. This is a core gap in the theory of change and operationalization of PBF, workers and managers both need to understand that performance is directly tied to payments in order to driven performance. Understanding the payment formula would be the first step, however there is significant heterogeneity in the payment formulas, as well as complexity in the calculations.
The methodology adopted in this study of capturing information on the PBF health system perceived effects through semi-structured interviews with health workers has a number of limitations. First, respondents were selected in collaboration with facility management, which could have led to a bias sample of well-adjusted or positive health workers. Moreover, the sample only focused on the supply-side and did not capture beneficiary perceptions or validate any of the changes health providers attributed to PBF. Although the evaluation was conducted by an independent evaluator (ThinkWell), the implementer and occasionally DPS accompanied our data collectors to the facility to facilitate the introduction and coordination of the interviews. The interview guide was open-ended, but all health workers were probed on changes attributed to PBF for all pillars to ensure we were able to capture the perceived effects across the entire system but could have introduced biasin health worker responses.

Conclusion
The changes described demonstrate that PBF has the potential to go well beyond health worker motivation and enhance service delivery to broadly strengthen the health system. However, to achieve its potential, a number of things need to happen. First, program designers and implementers will need to think more critically about how a combination of PBF incentives and technical support can catalyze and institutionalize managerial governance changes at the facility level. Managerial changes and organizational culture should incorporate managerial autonomy to ensuring that the facilities are properly staffed, have adequate supplies, and completed registers. Moreover, program implementers should consider a more robust and active long-term sustainability strategy to institutionalize the program into provincial and national health offices while decreasing control by an external NGO. The Mozambican PBF program failed at numerous level of institutionalization and gaining buy-in from government and other actors, and subsequently the program was ended in 2017 (Gergen;Falcao;Rajkotia, 2018), which reflects the experience documented across many programs (Shroff;Bigdeli;Meissen, 2017;Seppey et al., 2017). Second, the program must be designed such that an increasing proportion of the health facility budget is output-based versus input-based to keep the focus on performance. Third, strengthening the participation of the supply-side could support the strength of PBF's positive effects on the health system.
The Good, the Bad, and the Disruptive of Performance-Based Financing on the Mozambican Health System: Results from a process evaluation Jessica Gergen (ThinkWell); Yogesh Rajkotia (ThinkWell); Nirmala Ravishankar (ThinkWell).
Received: 16/Apr/2018 Approved 1 : 24/Aug/2018 1 Availability of data and material The dataset(s) supporting the conclusions of this article are considered sensitive. The datasets are owned by the United States Center for Disease Control and Prevention. Registration of the study has been completed through the CDC data repository. Analytical files and presentation of the findings have been shared with the Mozambican Ministry of Health.
Funding: This study is funded by the Presidents' Emergency Program for AIDS Relief (PEFPAR) through the Centers for Disease Control and Prevention (CDC), Project Fortalecer Cooperative Agreement U2GGH000422-01. Disclaimer: The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention. Ethical Trial Registration: Ethical approval for the study protocol, data collection instruments and informed consent forms was obtained from the Ethics Review Committee of the Centers for Disease Control and Prevention (CDC) [IRB 2015-190], and the Ethics Review Committee of the Mozambique Ministry of Health.