TechNet-21 - Forum

This forum provides a place for members to ask questions, share experiences, coordinate activities, and discuss recent developments in immunization.
  1. David Brown
  2. Immunization information systems & coverage monitoring
  3. Wednesday, October 26 2016, 09:37 PM

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

proof / prüf / (noun)

1. evidence or argument establishing or helping to establish a fact or the truth of a statement

synonyms: evidence, verification, corroboration, authentication, confirmation, certification, documentation, validation, attestation, substantiation

--------------------------------------------------------------------------------------------------------------------------------------------------------------------

When you pick-up and read a vaccination coverage survey report, what do you look at first? Be honest. You skip the background and methods sections of the report and go straight away to the table that details the vaccination coverage. More specifically, your eyes jump to the estimated coverage value for the third dose of diphtheria–tetanus–pertussis containing vaccine (DTPCV3), because, again we are being honest here, that is the door behind which the investment money is hidden.

And when you get to the DTPCV3 column, you have three estimates to choose from: coverage based on documented evidence in home-based records (HBRs) (e.g, vaccination cards, child health books, etc), that based on caregiver recall (relied upon when documented evidence in HBRs is not available) and that based on the combination of both sources. It might look something like this:

 

Sample Vax Coverage Survey Table rs

Reading the table above, you might then get excited (depending on the chair you sit in) because the estimated DTPCV3 coverage based on the combination of documented evidence and caregiver recall is 90%. That is quite good.

But what if I told you that everything was going to change and the new number to focus on in the table above is not the 90% DTPCV3 coverage by either source of information, but rather the new number to focus on is two lines above in the same column — the DTPCV3 coverage based on documented evidence in HBRs, which in this case is 71%. Wow! There is a big difference in the country above between the DTPCV3 coverage based on both sources and that based on actual documented evidence observed in the HBRs by the survey field teams, 90% vs 71%. As a programme manager or as a supporter of this programme, my decisions about priorities are very different in this case if I am guided by coverage based on documented evidence.

I will put money on the fact that some are asking, “why would we focus on the coverage by documented evidence alone? The 71% DTPCV3 coverage only reflects the contribution to the national coverage estimate based on documented evidence from the 75% of children in the survey above that had a HBR at the time of the survey.” Others may be asking which number is true, 90% or 70%...sorry truth is illusive and something that we do not, and will not, know. But for the same reasons that immigration officers require documented proof of identity using a valid passport on entry to a country or that national health officers at the airports and border entry points in some countries require documented proof of vaccination against polio and/or yellow fever, immunization programmes might hold themselves to account and place greater emphasis on documented proof in HBRs (and facility-based records) of who receives which vaccine, how many doses and when those vaccines were received.

Think about the recommended vaccines for infants in your country. In many countries, the programme recommends infants receive one dose of bacille Calmette-Guérin (BCG) vaccine, a birth dose of polio and hepatitis B vaccines, three doses of diphtheria–tetanus–pertussis containing vaccine (DTPCV) delivered in combination with three doses of hepatitis B (HepB) and Haemophilus influenzae type B (Hib) vaccine, three doses of oral polio vaccine (OPV), one dose of inactivated polio vaccine (IPV), three doses of pneumococcal conjugate vaccine (PCV), two doses of rotavirus (Rota) vaccine, one dose of yellow fever vaccine and one dose of measles containing vaccine (MCV). I will then highlight for you that current research suggests very mixed results of how accurately caregivers can recall their child’s vaccination history. In the 1980s and early 1990s, recalling your child’s vaccination history with some level of accuracy was perhaps possible (there were only five vaccines), but that is less likely today. I know I have a hard time recalling the vaccination histories of my two children.

From a measurement perspective, this is a problem for vaccination coverage surveys which are being relied upon (for better or worse) in many countries as a key mechanism for monitoring programme performance because of poor routine administrative vaccination coverage recording and reporting systems.A review of the most recent DHS and MICS results for documented evidence of vaccination in home-based records and the estimated vaccination coverage among children aged 12-23 months at the time of the survey in 32 Gavi-eligible countries in the WHO African Region suggests that documented evidence of vaccination history was available in only about HALF (!) of the estimated 31 million children born across the 32 countries during 2015 (Figure below), assuming the recent survey results by-and-large reflect the situations in the countries during 2015. Current HBR ownership ranged from around 30% (Chad) to around 95% (Rwanda) with a median value of 70% (IQR: 56-81) (Table below). Ten countries, accounting for just under one-quarter of total births across the 32 countries, were observed to have current HBR ownership levels >80%, with Gambia, Sao Tome and Principe and Rwanda having levels of 90% or greater. Twelve countries had current HBR ownership levels <60%. In Democratic Republic of Congo, Ethiopia and Nigeria—three countries collectively accounting for a little more than 40% of the total 2015 birth cohort across these 32 African countries—a random visit to a household with a child under two years of age in these three countries would identify documented evidence of vaccination history for less than one-third (!) of children based on most recent survey results.

With the problems alluded to above and an understanding that the contribution of coverage based on caregiver recall is almost certainly biased (although we do not know either the direction or the magnitude of the bias), I hope these figures give you pause.

The thought of moving away from utilization of vaccination history based on caregiver recall in coverage survey towards only accepting documented evidence of vaccination history, either from home- or facility-based records, might serve as a wake-up call to those communities where poor recording and reporting in home-based records and facility-based records continues. Over the long-run, this would be an important shift. Is such a shift likely to happen? Perhaps not, although those who conduct nationally representative population-based surveys would love to throw out the vaccination recall section of the survey questionnaire.

A more likely scenario is that over the coming 3-5 years the financial and technical support partners of immunization programmes, perhaps even the national immunization programmes themselves, will agree that good clinical and public health practice benefits from good documentation reflecting the importance of complete, timely, and accurate recording of information. And in doing so, these same partners will begin to take greater notice of the proportion of evidence that is derived from documented evidence in home-based records and facility-based records in vaccination coverage surveys. They will agree that current inefficiencies (i.e., missed opportunities for immunization, extra-immunization) that result from suboptimal documentation are no longer acceptable and that a more effective and efficient system is imperative. They will accelerate investments to understand and correct immunization information system deficiencies in ways not yet considered since continuing on the current path is insufficient and wasteful. And, they will agree that too much that needs to be known remains unknown, and that the poor quality data that exist in many areas of the world fundamentally shapes what we know about the performance of immunization programmes which in turn shapes how decisions are made. And this should change.

 

Figure current HBR ownership 32 AFR Gavi

David Brown Accepted Answer

In a prior post I alluded to the reporting of ever HBR ownership prevalence alongside current HBR ownership prevalence in Demographic and Health Survey country reports. It is useful to share an example from the 2015-16 Malawi Demographic and Health Survey which I find to be a very good example of a model table presentation of ever and current HBR ownership prevalence by background characteristics. Moving forward, it will be useful to learn from those within the immunization programmes how this information is used and leveraged to improve home-based record systems.

Attachments (1)

Thanks for this discussion.

It is a very good point that with the increasing complexity of vaccinations, vaccinationrecall may be more and more biased (a good 'new' research issue: is recall bias, under thismore complex vaccination scenarios,consistently towards over- or under-estimation?). The idea of enquiring about 'ever' HBR ownership is very interesting. It came to my mind that here we will have then 'recall' bias for those who do not currently own a HBR, and similar considerations as we have in relation to vaccination recall. But I can imagine that adding this item in vaccination surveys would not be too difficult and much better than no having any idea at all of what happened with previous HBR ownership (together with a few questions to ascertain what happened with the HBR for those who had it once). An idea worth promoting, I think. Equally important is HBR use: they need to be taken alongside the child when vaccinating and in other health care encounters with the services.

(I totally agree that methods should go first; not only when producing evidence but also, for the same reasons, when reading it!).

Xavier Bosch-Capblanch

Dear David,

Thank you for this very interesting and relevant commentary on Home-based records (vaccination/health cards). Home-based records (HBRs), along with health facility registers, are the main tool we have to ensure that children (and people in general) go back to get all vaccine doses needed. Reducing drop-outs alone has the potential to really improve vaccination coverage (by the way, in this day and age, I truly think that it is time to move our threshold for an acceptable drop-out rate from 10% to at least 5%...this is a personal opinion).

HBRs are the main tool we have to ascertain vaccination status with some confidence in a survey. I want to also emphasize that having HBRs allows for conducting secondary analyses from existing surveys, analyses such as vaccination timeliness, simultaneity of vaccine administration between vaccines recommended at the same age, and missed opportunities, for example, whether people receiving a measles-containing vaccine (MCV) and that are missing a DTP/penta dose are given that dose during the MCV visit. Honduras took advantage of an existing survey (DHS) and conducted some of these secondary analyses (see attached summary). This was possible in Honduras because the percentage of HBRs seen in that survey was high.

Finally, a recent publication of a study on missed opportunities for vaccination conducted in the Dominican Republic concluded that about half of the children assessed had not brought their vaccination card when visiting the health facility (this are non-vaccination visits) and that the main reason for missed opportunities was health workers not checking the cards. I paste the abstract below. The study can be accessed at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5080475/

Garib Z, Vargas AL, Trumbo SP, Anthony K, Diaz-Ortega JL, Bravo-Alcántara P, Leal I, Danovaro-Holliday MC, Velandia-González M. Missed Opportunities for Vaccination in the Dominican Republic: Results of an Operational Investigation. Biomed Res Int. 2016;2016:4721836.

Abstract

Background. Despite the success of the Dominican Republic's National Immunization Program, homogenous vaccine coverage has not been achieved. In October 2012, the country implemented a study on missed opportunities for vaccination (MOVs) in children aged <5 years. Methods. A cross-sectional study of 102 healthcare facilities was implemented in 30 high-risk municipalities. Overall, 1500 parents and guardians of children aged <5 years were interviewed. A MOV is defined as when a person who is eligible for vaccination and with no contraindications visits a health facility and does not receive a required vaccine. We evaluated the causes of MOVs and identified risk factors associated with MOVs in the Dominican Republic. Results. Of the 514 children with available and reliable vaccination histories, 293 (57.0%) were undervaccinated after contact with a health provider. Undervaccinated children had 836 opportunities to receive a needed vaccine. Of these, 358 (42.8%) qualified as MOVs, with at least one MOV observed in 225 children (43.7%). Factors associated with MOVs included urban geographic area (OR = 1.80; p = 0.02), age 1-4 years (OR = 3.63; p ≤ 0.0001), and the purpose of the health visit being a sick visit (OR = 1.65; p = 0.02). Conclusions. MOVs were associated primarily with health workers failing to request and review patients' immunization cards.

Attachments (1)
David Brown Accepted Answer

Ha! I am going to respond to my own posting, because, well it just seems like the right thing to do. In all seriousness, I meant to include two definitions at the end: ever HBR ownership prevlanece and current HBR ownership prevalence.

Ever home-based record ownership prevalence indicates the percentage of eligible target individuals (e.g., children 12-23 months) who have ever received a home-based record (e.g., vaccination card, child health book, etc) in their lifetime. This can be important in helping identify supply side problems with HBR provision (i.e., availability). Mums cannot keep something that they do not receive...stayed tuned for a forthcoming posting on this issue.

Current home-based record ownership prevalence indicates the percentage of eligible target individuals for whom someone (e.g., a caregiver) was able to produce a home-based record (e.g., vaccination card, child health book, etc) for the survey team to view and abstract vaccination history information from. This can be important in helping identify challenges in how HBRs are valued and utilized by caregivers. It may lead to an exploration of how health workers are communicating the importance of the document, or it may lead to an exploration of how the HBR is poorly designed to meet its functional needs. There is a new open access (FREE) article describing a user-centered design workshop that took place in Sri Lanka in March 2016 on PubMed (https://www.ncbi.nlm.nih.gov/pubmed/27743647).

I (We, let's all get invovled here) believe both ever and current HBR ownership prevalence are important indicators to add to the many other programme indicators that are being monitored. Soon we think that both values will be reported in DHS and MICS reports too.

~db



There are no replies made for this post yet.
However, you are not allowed to reply to this post.

Not signed up?

Joining TechNet is free, takes seconds, and provides many benefits.

Join today

Already a Member?