Strengthening Supportive Supervision in Immunization Programmes

Thanks for this resource.

Let's, though, acknowledge that there is little evidence that could justify the statement that "Supportive supervision is highly effective in improving staff performance, satisfaction, and retention".

There is no question that health workers, particularly those in remote, rural areas or in fragile contexts, require a lot of support to successfully carry out their tasks. However, it would seem that supportive supervision often ignores or takes for granted situations where physical infrastructures are deplorable and/or working conditions unbearable. The available evidence, indeed, does not help us to advocate for supportive supervision very loudly, I would say.

Xavier, thanks for your thoughtful posting. Gavi's pro-equity evidence map also points to inconclusive evidence around supportive supervision. It's a good read, covering a myriad of well known interventions framed around IRMMA: https://zdlh.gavi.org/resources/evidence-map.

Thanks for this resource and I am very happy to have a reply from you,Jenny. I hope you are doing well!

Actually, I was checking the evidence maps, and I think we need to redouble our care to avoid misinterpretations. It is risky to draw effectiveness conclusions from rapid reviews without assessments of the risks of biases and to issue recommendations on the basis of that evidence. I wonder why after decades of discussing the issue of supportive supervision (our review has a study from 1984) we do not have yet compelling evidence (i) about its effects and (ii) about how supervision should or could be implemented. Or research on supervision is poorly done and/or supervision is not an appropriate intervention to support health workers in many settings.

In my view, we need to (i) bring in standards for health systems research to minimise research wastage (https://www.thelancet.com/series/research) and (ii) abandon the fallacy that some system interventions can substitute the shortcomings of health services operating under unacceptable baseline conditions. I do not have the power to push for either, but I would be certainly ready to contribute to both.

I fully agree with Xavier's concerns and support his encouragement of evidence-based practice (previously referred to as evidence-based medicine), including the use of rigorous systematic reviews and grading of underlying evidence, as well as his acknowledgement of the need to apply more intentional and deliberate health services research practices to areas within immunisation service delivery that are complex (perhaps more than we care to acknowledge). We can celebrate the successes achieved with existing delivery systems have operated, albeit perhaps inefficiently, for more than 50 years in many countries while at the same time continuing to challenge what we think we know to be true (...known knowns, known unknowns, unknown unknowns, unknown knowns...Mark Twain: “It isn’t what you don’t know that gets you into trouble as much as what you think you know that isn't so.”) and pushing for ever greater efficiencies in increasingly resource constrained environments. There is much within supportive supervision to further understand, both pressure testing what we think we know (particularly in different resourced environments and different organisational structures) as well as exploring new opportunities and models that are organically evolving with a new generation of frontline staff who have and are maturing in a very different world from the beginning of EPI.

Supportive supervision

Modibo Dicko

My belief is that supportivesupervision is very effective in maintaining and improving the performance of health workers managing the immunization supply chain (iSC) in the field. This conviction dates back a long time. Indeed, I remember a training in iSC management that I organized in 1997 in Dakar for national logisticians from all French-speaking countries in Africa. We asked the participants to list three factors that they consider essential to maintain their performance in daily work. There were people in the room (especially from the Central African Republic and Chad; but they were not the only ones) who had 4 to 9 months' overdue salaries. So we naturally expected to see salary appear at the top of the list of motivating factors. Well, no! To our surprise, it was the supervision that came first. We asked participants why. Their answer was clear: "If you work hard and no one cares about your results, why are you going to strive to perform?" For them, the lack of supervision was evidence of the lack of interest of their superiors in their work. And that demotivated them enormously.

Another piece of evidence came when WHO asked me to analyse the results of effective vaccine management assessments (EVMA) conducted in 2010-2012 (before the improvement plans) and in 2017-2019 (after the implementation of the first improvement plans) respectively. On the attached slides, it appeared clearly that the score for criterion E9 (Support functions, including supervision) had decreased from 65% in 2010-2012 to 58% in 2017-2019. I have called that the "Precipice of Power" since supervision is the responsibility of hierarchical superiors. It can also be seen on the slides that the scores of the input criteria (E3, E4 and E8) have increased sharply, creating the two "Mountains of Wealth and Knowledge". It can also be noted that the scores of the process criteria (E5, E6, E7), which involve the human factor, have increased much less than the former, which has created the "Depression of Willing" between the two mountains mentioned above. My conclusion was that the neglect of support functions (including supervision) had most certainly contributed to slowing down the progress of the process criteria because of the demotivation that the participants of the 1997 training had already deplored.

Analysing the results of the EVM assessments carried out in 2019-2023, it can be seen that the score of support functions (now M1 to M4 with M3 devoted to supportive supervision) has increased up to 65% compared to the score of support functions in 2017-2019, which was only 58%. At the same time, the scores of the process criteria have also increased by an average of 6%. I reiterate my deduction that the improvement in support functions, including supervision, has resulted in an increase in the scores of the process functions.

In conclusion, we can see that:

• Logisticians admit that the lack of supervision demotivates them;
• The decline in supervision is concomitant with a slowdown in the increase in process function scores;
• As the score of the supporting functions increases, so does the increase in the scores of the process functions.

For me, it's “C.Q.F.D.” (“Ce Qu’il Fallait Démontrer” meaning "What had to be demonstrated")! In other words, supportive supervision is an effective factor in improving the performance of health workers because of the motivation it instills in them.

Supervision formative

Modibo Dicko

Ma conviction est que la supervision formative est très efficace pour maintenir et améliorer la performance des agents de santé gérant la chaîne d’approvisionnement vaccinale (CAV) sur le terrain. Cette conviction date de longtemps. En effet, je me rappelle une formation en gestion de la CAV que j’avais organisé en 1997 à Dakar pour les logisticiens nationaux de tous les pays francophones d’Afrique. Nous avons demandé aux participants de lister chacun trois facteurs qu’ils jugent indispensables pour maintenir leur performance dans le travail quotidien. Il y avait dans la salle des personnes (notamment de la République Centrafricaine et du Tchad; mais ils n’étaient pas les seuls) qui avaient 4 à 9 mois de salaires en retard. Nous nous attendions donc naturellement à voir le salaire apparaître en tête de liste des facteurs de motivation. Eh bien, non! À notre grande surprise, c’est la supervision qui est venue en premier. Nous avons demandé aux participants pourquoi. Leur réponse était sans appel: « Si tu travailles et tes résultats n’intéressent personne, pourquoi vas-tu t’évertuer à être performant? » Pour eux, l’absence de supervision était l’évidence de l’absence d’intérêt de leurs hiérarchies pour leur travail. Et cela les démotivait énormément.

Une autre pièce à conviction viendra lorsque l’OMS me demanda d’analyser les résultats des évaluations de la gestion efficace des vaccins (EGEV) conduites respectivement en 2010-2012 (avant les plans d’amélioration) et en 2017-2019 (après la mise en œuvre des premiers plans d’amélioration). On voit nettement que le score du critère E9 (Fonctions d’appui, y compris la supervision) a baissé de 65% en 2010-2012 à 58% en 2017-2019; ce que j’ai appelé le « Précipice du Pouvoir » vu que la supervision relève des supérieurs hiérarchiques. On constate que les scores des critères d’intrant (E3, E4 et E8) ont connu une forte augmentation, créant ainsi les deux « Montagnes de l’Avoir et du Savoir ». On constate également que les scores des critères de processus (E5, E6, E7), qui impliquent le facteur humain, ont augmenté beaucoup moins que les premiers, ce qui a créé la « Dépression du Vouloir » entre les deux montagnes ci-dessus citées. Ma conclusion fut que la négligence des fonctions de soutien (y compris donc la supervision) avait très certainement contribué à ralentir le progrès des critères de processus à cause de la démotivation que les participants de la formation de 1997 avaient déjà déplorée.

En analysant les résultats des évaluations GEV effectuées en 2019-2023, on constate que le score des fonctions d’appui (maintenant M1 à M4 avec M3 qui est consacré à la supervision formative) a augmenté jusqu’à 65% comparativement au score des fonctions d’appui de 2017-2019 qui n’était que de 58%. On constate parallèlement que les scores des critères de processus ont eux aussi augmenté de 6% en moyenne. J’en conclus de nouveau que l’amélioration des fonctions de soutien, y compris la supervision, s’est traduite par une augmentation des scores des fonctions de processus.

En conclusion, on constate donc que :

• Les logisticiens avouent que l’absence de supervision les démotive;
• La baisse de la supervision est concomitante avec un ralentissement de l’augmentation des scores des fonctions de processus;
• L’augmentation du score des fonctions d’appui s’accompagne de celle des scores des fonctions de processus.

Pour moi, c’est C.Q.F.D. (« Ce Qu’il Fallait Démontrer »)! Autrement dit, la supervision formative est un facteur effectif d’amélioration de la performance des agents à cause de la motivation qu’elle leur insuffle.

Thanks for these insights. It is very important to draw as much evidence from different components of supervision as possible, inlcuding the EVMA data.

I am a bit reluctant to grant a lot of credibility to opinions or self-reported outcomes. If health workers are more concerned about supervision than about a decent salary, I would be concerned about the questionnaire or assessment tool. It could well be that health workers replied what they knew was expected to them, or they may have thought that there was nothing to do about the salaries so it was not worthwhile to go in that direction. I think that income to decently raise a family is more important than supervision.

Colleagues- Thank you everyone for this interesting discussion! 

I am a longtime believer and have been an implementer of various succesful supportive supervision interventions, and was surprised to learn from this discussion of the lack of evidence to prove its effectivess. In my programmatic experience I have seen it to be successful in improving healthcare worker motivation and improving skills/behaviors (particularly related to supply chain). 

From a research perspective I suspsect it is something that can be difficult to measure properly given the large number of variables regarding HOW supportive supervision can be done-- its quite a general concept which can be adapted to many technical/health areas, can be done using digital tools or paper-based, can be done with minimal orientation for the supervisor or a formal training, and can be implemented at various levels of the health system all the way down to the community level. Like many programmatic interventions, its possible for it to be poorly exicuted, or well exicuted. Would would be great is if the research could tell us the 'secret sauce' that makes supportive supervison most effective, and in what situations/scenarios we get the most incremental benefit from it. As Xavier rightly pointed out, supportive supervision is not going to solve deeper problems like if a HCW isnt satisfied with their pay or the facility infrastructure is falling apart, so there are some situations where its not going to have much impact.  

An example of a recent successful expereince we have had with a supportive superision tool in Malawi, where we piloted a tool/process to strengthen CHW knowledge and skills regarding immunization supply chain practices (handling, storing vaccines, cold chain, etc.). In Malawi, CHWs do most of the vaccinations in the country, and dont recieve regular top up training related to immunization supplies, so this is a way to ensure they are handling and storing vaccines safely, forecasting their needs correctly, etc. Our tool has built in messaging to help guide the supervisors, so they have all the technical information at their fingertips to mentor the CHWs on the spot.

https://www.villagereach.org/resource/digital-health-tools-vaccine-supply-chain-malawi/

We will be publishing our experience with this tool so look forward to contributing to the evidence base in a positive way regarding supportive supervision for supply chain.

Best regards,

Rebecca Alban

Senior Manager, VillageReach

Thanks to all contributors for providing so many interesting insights.

The Gavi pro-equity evidence map on supportive supervision that Jenny mentions gives a nice summary of the main issues and is worth reading. The study labels the evidence for supportive supervision’s effectiveness as “inconclusive”. However, this review is focused specifically on "whether supportive supervision led to improved data utilization, data collection, and decision-making". It does not consider the effectiveness of supportive supervision on improving overall HCW performance or immunization service-delivery (or health outcomes).

The focus on data for decision-making reflects a common problem encountered when trying to synthesise the findings of supportive supervision interventions - namely, that success tends to be measured differently for each. Success in one may be improved HCW motivation, but it may be vaccine knowledge, cold chain management operations or reduced wastage in another. This perhaps relates to Xavier's idea of prioritising "standards for health systems research".

A systematic review from 2020 on successful supportive supervision interventions in LMICs (Deussom et al.) reached similar conclusions and gave the following recommendations on what interventions with the best chance of success should look like (these give a few clues as to what might be in the "secret sauce" Rebecca mentions):

• integrate evidence-based, QI tools and processes;
• integrate digital supervision data into supervision processes;
• increase use of health system information and performance data when planning supervision visits to prioritize lowest performing areas;
• scale and replicate successful models across service delivery areas and geographies;
• expand and institutionalize supervision to reach, prepare, protect, and support frontline health workers, especially during health emergencies;
• transition and sustain supervision efforts with domestic human and financial resources, including communities, for holistic workforce support.

For me, the most striking observation from the review was that 44 of the 57 studies were externally funded pilot projects. I wonder how many were sustained or integrated into national health systems? Probably very few. This supports Xavier's advice that we should "abandon the fallacy that some system interventions can substitute the shortcomings of health services operating under unacceptable baseline conditions" and Rebecca's that supportive supervision can't solve "deeper problems".

Findings from EVM assessments, now including almost 3000 health facilities across 75 countries, offer useful insights into the prevalence of supportive supervision of HCWs in LMICs and the obstacles that restrict this provision. The EVM framework, which considers the inputs, outputs, and performance of individual ISC functions, provides a ready-made, standardised conceptual model. For reference I have attached an export of the 21 input and output requirements in 'M3 Supportive supervision' that EVM assesses. More information on all 874 EVM requirements can be found here:

https://evm2.who.int/Public/Requirements

Another very interesting paper relevant to the above is this one. It supports the idea that evidence on the effectiveness of supportive supervision is mixed and identifies “contextual, institutional and logistical challenges” to the successful implementation of supportive supervision (EVM C1-6), notably over-reliance on externally funded pilot projects (EVM C6) and lack of clearly defined oversight and accountability processes (EVM Outputs). 

---

Avortri, G.S., Nabukalu, J.B. and Nabyonga-Orem, J. (2019) “Supportive supervision to improve service delivery in low-income countries: is there a conceptual problem or a strategy problem?,” BMJ Global Health, 4(Suppl 9), p. e001151. Available at: https://doi.org/10.1136/bmjgh-2018-001151.

Deussom, R. et al. (2022) “Systematic review of performance-enhancing health worker supervision approaches in low- and middle-income countries,” Human Resources for Health. BioMed Central Ltd. Available at: https://doi.org/10.1186/s12960-021-00692-y.