I am setting up an alarm to indicate to high a temperature in a vaccine refrigerator.
I would appreciate it if someone could tell me the reccomended temperature and the time that
temperature must be maintained to set off the alarm?
My second question is if you what that time and temperature are based on?
Larry Schlussler Sun Frost
I read John’s summary and could not see validity in any of the points he made except for his first sentence which was incomprehensible to me and my colleagues and hence I couldn’t tell if I agree or disagree.
“VVM’s are developing towards a more complex mechanism that will not be easy to follow with parallel MKT driven alarm systems.”
John also states that a ‘severity index” makes no sense because the vaccines may have different ages. John, if you were responsible for two refrigerators and one was at 9 C and the second was at 20 C, which refrigerator would have repaired first? In the refrigerator at 20 C all the vaccines, both new and old, would be degrading five times faster.
All alarm systems currently used are based on the temperature variations of the refrigerator. Unfortunately the system currently used does not always correlate with how the vaccines are affected. For example a refrigerator at 20 C for 9 hrs will not set of the alarm off whereas a refrigerator at 9 C for 10 hrs will. As mentioned previously vaccines at 20 C will degrade five times faster than vaccines at 9 C. This is a failure of the current alarm system.
The results obtained by an analysis incorporating MKT temperature are consistent with indications on a VVM. The “Arrhenius Graphs” used by TEMPTIME to calibrate VVM’s are the same graphs used to correlate MKT temperature with vaccine life. The use of MKT temperature is accepted by the pharmaceutical industry, also WHO uses MKT temperature measurements in it’s “daynight” tests of refrigerators.
Just because the calculation of MKT temperature involves some math does not mean the results can not be presented in a form easily interpreted. Both a VVM and the proposed methods are based on the Arrheneius equation. A health worker does not have to understand the math and chemistry behind the Arrhenius equation to interpret the meaning of the dot on a VVM. What is difficult for health workers with all levels of training is to be presented with a graph showing temperature variations for a month and determining how the temperature variations affected the stored vaccines. Using MKT analysis the question could be easily answered.
One possible form to present the results is:
Life of a new VVM2 vaccine at 8 C is.......................................................................................................................108 days
If the storage conditions experienced in the last 30 days are continued the life of a new VVM2 vaccine will be…..80 days
A similar output could be used for an alarm system based on 24 hrs of performance:
Life of a new VVM2 vaccine at 8 C is…………..................................................................................................……108 days
If storage conditions experienced in the 24 hrs continue the life of a new VVM2will be……………………..……….20 days
If the life of the vaccine drops below 100 days an alarm could be turned on. The severity of the problem could be seen by looking at the projected life of the vaccine. This system would be more responsive to significant temperature changes than the time temperature method. For example if the temperature in the refrigerator suddenly increased to 20 C the alarm based on MKT temperature would go on in about 2 hrs while the time temperature alarm would take 10 hrs.
While writing this reply I realized that a user friendly and informative way to present data on the performance of the refrigerator would be to display the current temperature and a graph of the refrigerators temperature for the past 48 hrs, a button could be pressed to present earlier data.
The display would be located on the front of the refrigerator. This data and its interpretation would be much easier to understand than the data extracted from a 30DTR. Forty eight hour display thermometers exist and could be built at a low cost. This device could also be programmed for the alarm system and 30 day performance indicator described.
To make further progress on this topic I think it would probably be more expedient to have some one on one conservations.
Hi John
Could you please clarify this sentence and the rest of your concluding paragraph.
VVM's are developing towards a more complex mechanism that will not be easy to follow with paraell-driven MKT alarm systems.
Larry
Sun Frost
The proposals of the side-meeting at Technet seemed to me a neat closure on the many months of discussion among stakeholders, allowing manufacturers of RTMDs and 30DTR based devices to ‘go forward’. But the debate continues and I think I have a relevant point to add.
When VVMs were being developed, vaccine manufacturers were cautious of any suggestion that they would be seen as monitors of vaccine potency. They were more comfortable with the decision that VVMs should be used as a managerial tool for vaccine handling with a generous ‘safety’ margin, that are selected, purchased and fitted by manufacturers themselves. For this reason alone I think we should reflect on the attached list of issues addressed by the side-meeting proposals:
The literature is divided (Vaisala , “Mean Kinetic Temperature in GxP environments”, Life Science. European Pharmaceutical Review Volume 21, Issue 6 , 2016) on the advisability of using MKT for refrigerated storage but should we claim to be implementing MKT driven alarms to correspond with time and temperature exposure, it seems likely that we would need to return to a discussion with the vaccine manufacturing industry, thus reopening also the issues of VVM implementation.
In reply to Larry's statement "We could eliminate these problems by doing digitally what a VVM does in an analog fashion..."
I wanted to point out that Temptime demonstrated a combined VVM/QR code that can be read by a phone mobile app they also demoed. The VVM is embedded in the QR code ink, and the phone's camera picks up the spectrum change of the VVM as it also captures the rest of the QR code data. This is a year or two out from introduction, and they need to figure out how to get it on the Vx container lable real estate (the demo used a label on a small vial-sized box package), but digitizing VVM seems to be done.
After looking at many helpful comments on TechNet I wanted to summarize my current thoughts on temperature monitoring and alarm systems. A temperature monitoring/alarm system should:
A VVM is a simple elegant device which determines the loss of life due to a temperature excursion. A VVM is essentially an analog computer which displays its output as the density of a dot on a piece of paper. The VVM takes into consideration how long the vaccine is exposed to a particular temperature even if the temperatures are varying.
For example, if a vaccine is exposed to three days at 9 deg C, two days at 20 deg C and five days at 12 deg C the vaccine will degrade five times faster than at 9 deg C. A VVM will calculate in an analog fashion the net effect of these temperature variations.
What would be extremely useful is if we mimicked what a VVM does digitally. The output of the device could be a dot of varying density or more practically the device would indicate the loss of life of a vaccine. The loss of life could be displayed for vaccines of different sensitivities.
The basic problems with currently in use monitoring and alarm systems is that the criteria that they use for analysis is not correlated with how the life of a vaccine is effected. This is why after a temperature excursion the change in life indicated by a VVM often does not correlate with what the alarm or monitoring system is indicating.
We could eliminate these problems by doing digitally what a VVM does in an analog fashion and basing our alarm and monitoring system on MKT temperatures.
Larry Schlussler
Sun Frost
Smart Phones and 30DTR
Joanie Robinson pointed out that there are many locations using 30DTR's and "without remote data acess", making data analyssis more challanging,
To make use of this already in place recorder, data from a 30DTR could be down loaded into an app on a smart phone to carry out the appropriate calculations.
Monthly Performance
From 30 days of recorded data the MKT temperature could be calculated. If the MKT temperature is 8 deg C or lower the screen would indicate good performance. If the MKT temperature is above 8 deg C the screen would indicate the loss of life due to higher temperatures. The results could be displayed for vaccines of vaccines of differing sensitivities.
The screen could show the life of a new vaccine at 8 deg C and the reduced life as a consequence of the 30 day temperature excursion. This information would help manage vaccines in a refrigerator which is not working perfectly.
Currently there this information is not available.
If data is being collected remotely the same calculations and display could be used.
A graph of the unprocessed data could also be displayed, this graphical presentation would be valuable for trouble shooting a problem refrigerfator.
Performance During Transport
During transport the temperature of a vaccine carrier could be recorded on a 30DTR at the conculsion of the trip the data could be down loaded to an appropriate app. The app would than calculate the loss of life due to temperature excursions. The data could than be displayed could presented in the same manner monthly performance is displayed. The loss of life of vaccines of all sensitivities would then be clearly indicated.
Severity Metric
If a conventional alarm is activated the loss of life of the vaccines in the next 24 hours could be found by downloading the data in 30DTR and processing the data to determine the loss of life in the next 24 hours. this would give an indication of the severity of the problem. If data was transmitted to a central location determining the life lost in the next 24 hours would be simpler. There are probably additional ideas on how to use the calculating potential of an smart phone.
Larry Schlussler
Sun Frost
Hi Larry
I recommend you the temperatures inside the refrigerator to set up the alarm are 8°C high and 2°C low, and the time is 5 minutes, the elapsed time maximun while the sensor is put out of the refrigerator with the basket, in case that the vaccinator needs pick up the vaccines on the lowest baskets. but if the vaccinator is cautious, doesn´t need to get out the sensor only put it beside.
When the thermostat are adjustable by user, the temperatures oscillating between 3.5° C and 5.5°C , but in the last version of refrigerator where doesn´t have the thermostat knob, the temperatures oscillate between 7.0°C and 7.8°C.
Best regards
Hi Jonie,
Your point is correct. Even though the life of the vaccines decrease the same % for vaccines of different sensitivities, for a given excursion the percent of life lost will be different. Thanks for pointing that out.
Severity Metric
If a vaccine degrads 3% and an alarm goes off, it is valuable to know that the vaccine has degraded 3%, however the alarm does not give any information on the rate of decay. The rate of decay would tell you how much time you have to make a repair. My previous suggestion was to look at the time it took for the alarm to go off and use that as a measure. However I think a better projection could be made by looking at the MKT temperature for an 8 hour peroid before the alarm went off and than calculate the % loss of life for the next 24 hours. For a VVM 2 vaccine if the MKT temperature for the previous 8 hrs was 10 deg C the percent of life lost for the next 24 hrs will be 1%, if the vaccines were warmer say 20 deg C the percent of life lost for the next 24 hrs will be 5%. Taking Jonie's comments into consideration, for a VVM14 vaccine if the MKT temperature for the previous 8 hrs was 10 deg C the loss for the next 24 hrs will be.1%. If the temperature for the previous 8 hrs was 20 deg C the % loss in the next 24 hrs will be .5%. If the data is presented on a screen it would be easy to show the loss of life the next 24 hrs for vaccines of all sensitivities.
Hi Larry,
Just because both a VVM2 and a VVM14 have a 10% decrease in life from 5deg to 20deg, that doesn't mean that they have the same loss of life per excursion.
For example, consider a 1 day excursion at 20degC. Given the numbers you provide in your last comment, the VVM2 would lose 1 of 20 days, or 5% of its life. However, the VVM14 would lose 1 of 200 days, or only 0.5% of it's life. So therefore, in the event of a single excursion, vaccines with different stabilities would experience a different % loss of life.
Joanie
I thought it would be worth repeating that for a given temperature excursion the percent of life lost by vaccines
of different sensitivities will be the same. For example, the life of a VVM2 vaccine at 5deg C is 200 days and it's life at 20 deg C
is 20 days. A VVM14 vaccine has a life of 2000 days at 5 deg C and 200 days at 20 deg C. Both `decreased by a
factor of 10. This makes data reporting and computations simpler when using an analysis based on MKT temperatures.
Severity Metric
I think what John and Martin were illustrating with a severity metric is how much time do you have to make a repair
before the vaccines are significantly degraded. The alarm could initially be set off when 3% of the life of the vaccine
is lost. A 3% loss could be obtained in a refrigerator at 10 deg C or in a shorter time in a 25 deg C refrigerator.
At 25 deg C it will only take another .3 days to degrade 3%
in the refrigerator at 10 deg C it will take 9 times longer or about 3 days.
If the alarm simply posted the time it took to lose the initial 3% of life, the time to lose the next 3% would usually
be about the same, this would help a technician prioritize his time
Alarm Based on Vaccine Life
I found Umit's discussion of vaccine freezing very informative.
In Umit's section " Alarm and Vaccine Stability" Umit is under the impression that the alarm is set to trigger
when the "excursion is severe enough to significantly effect the life of the vaccine". It could of course be set
to a very small percentage.
Densitometer and VVM's-
My mention of using a densitometer on a VVM was to show that the percent of life lost by a vaccine
could be determined by temperature data used in an analysis based on MKT temperature or by the change
in density of a dot on a VVM. I did not think using a densitometer was practicle in the field. Umit pointed that
it was "out of porportions of the field reality"
What gave me the idea for this example was supprisingly a paper by Umit et al "Use of Cool Water Packs......
to Prevent Freezing During Transport". In this paper Umit used a densitometer in the field to determine the
change in optical density of a VVM dot to determine the percent change in the life of a vaccine during transport.
Since all the time-temperature data was available, at the time of reading the paper, I thought it would have been
easier to determine the percent of life lost based on an analysis using MKT temperature.
Hello!
Umit, thank you for your post! It was very informative and clarifies a lot. We often have long discussions with colleagues who want to set conservative alarms with RTM systems to protect vaccines. They seem to define the WHO alarm recommendations as a serious risk indicator for status of vaccines and have trained all their staff to follow this protocol. It is all from good intentions of wanting to be more proactive.
The result is that when countries want to set up the alarm threshold and duration for their RTM system, they often choose to go with a more conservative temperature range than the WHO recommendations in order to give health facility staff time to respond to those alarms. Is there a document published on defining the WHO temperature threshold (< -0.5C for one hour and > 8C for 10 hours) and how it should inform continuous temperature monitoring and notification practices?
The use of MKT has been a long discussed topic in our office. We would like to summarize what has been proposed so far in this thread, with some liberal interpretations on our side. Anyone have additional use cases or scenarios they can imagine with it?
Umit’s post brings much clarity to a lot of questions we face frequently and for us it makes sense to stick with the WHO alarms at the clinic levels for now... and continue these discussions as they are always great!
Look forward to hearing all your thoughts!
Martin.
Dear All,
I was the responsible officer when the alarm settings for 30-day electronic refrigerator temperature logger were decided. I will explain the hows and whys behind this decision. I will also touch upon the confusion I see in the messages between the alarm and the stability. Lastly, I would elaborate on alarm philosophy as a control measure from risk management perspective.
Why to abandon thermometers?
Until the revolutionary introduction of 30-day electronic refrigerator temperature loggers, temperature monitoring in refrigerators was done with the help of thermometers. We perfectly know that thermometer is NOT a monitoring device; it is only good when you look at it. Oxford dictionary defines monitor as follows:
“A device used for observing, checking, or keeping a continuous record of something.”
That means monitoring includes an action that is to pay continued close attention to [something] for a particular purpose. Here ‘continuity’ is the key, and thermometer has no continuity. If you see say 4.5 deg C reading in a thermometer in a health centre refrigerator on Monday morning, it does not mean that things were okay during the weekend. In this sense, readings within the recommended temperature range give a ‘false’ confidence to workers as if things were in control. Although it would not tell you any other details, reading say -2 deg C means something and has some value – meaning that something is wrong at that particular moment. Unfortunately, there are still many immunization and other medicine programmes that use thermometers as the main monitoring devices in unit level refrigerators. If you were hospitalized with a heart problem, would you be happy to be connected to a “thermometer like” heart monitor that does not record and nurses and doctors only check it in the morning and in the evening?
Temperature alarm settings for refrigerators (applicable to 30-day electronic refrigerator temperature logger)
From this perspective, we wanted to eliminate thermometers from the programme as monitoring devices. It was an IQPC cold chain meeting, during a lunch I talked to representatives of temperature monitoring device manufacturers and told them about the dream device I had in my mind – describing the features you are familiar with today. Though there was a general agreement that this could be done, nobody promised anything. And just before the Xmas period of that year, the representative of one of the temperature monitoring device companies visited me with a prototype device saying that I can hang it my Xmas tree… The following year we have finalized the specifications and verification protocol following relevant PQS SOPs and prequalified the very first device.
There were a lot of deliberations in-house and outside the WHO on how to set up the alarms. We believed that the alarm should be based on refrigerator performance criteria (please note that this performance criteria for the refrigerator is also established taking into account the excursions not have an impact on the most heat and freeze sensitive vaccine). In this regard, the specifications and verification protocols that are used for refrigerators were taken as the basis for this decision. Because the logic was if we allow a certain amount of excursion in prequalifying refrigerators, we cannot set up a limit that is tighter.
Acceptable temperature range for refrigerators is described in the PQS specifications as follows:
“The acceptable temperature range for storing vaccine is +2°C to +8°C. However, transient excursions outside this range will be tolerated, within the following limits:
These figures were worked out for the repeated cycles of the day/night test that is equivalent to 30 days (with the assumption that vaccines in such refrigerators are used within a month) that such excursion does not have any significant life lost in the most heat sensitive vaccines.
Based on these calculations, we agreed to have the lower limit set to -0.5 deg C for 1 hour single event. This is an “operational” point rather than being an “actual” freezing point for the most freeze-sensitive vaccine that is HepB. Freezing is a very complex process and cannot only be explained with the help of temperature. For example, in our validation of the shake test study (2010), published in World Health Bulletin (http://kartoglu.ch/papers/11_Validation_of_shake%20test_for_detecting_freeze_damage_to_adsorbed_vaccines.pdf), we have exposed all freeze-sensitive vaccines to -2 deg C for 24 hours, and no freezing was observed in any of them at the end of this period. Microscopic examination of these vials was identical to those kept at 2-8 deg C. This was confirmed with phase contrast microscopy (PCM), scanning electron microscopy (SEM) as well as atomic force microscopy (AFM) that the lattice between the antigen and aluminum adjuvant was intact and not broken in these vaccines. Since freezing points of freeze-sensitive vaccines have pretty big range, the programme had to have an “operational” point for freezing, taking the most freeze-sensitive vaccine HepB freezing point as the trigger point. But here we should note that HepB having -0.5 deg C freezing point does not mean it will freeze when it is exposed to this temperature, exactly like water does not necessarily freeze at zero degree… In this regard, we need to distinguish between “being actually frozen” and “being exposed to freezing temperatures”. So, with the lower alarm, we are warned that the refrigerator had “freezing temperatures” that might put freeze-sensitive vaccines at risk if not fixed.
Many commercial freeze-indicators have zero degree as the trigger point. However, taking account the accuracy of temperature monitoring devices (+/- 0.5 deg C), we know that half of the alarm cases will be above zero degree at the positive temperature side. With the Gaussian distribution logic, setting the trigger point to -0.5 deg C, we manage to have all alarm cases below zero degree C.
As for the higher temperatures, in addition to 5-day day/night test we have taken into account the hold-over time that is defined as “The time in hours during which all points in the vaccine compartment remain between +2°C and +10°C, at the maximum ambient temperature of the temperature zone for which the appliance is rated, after the power supply has been disconnected.” This is how the 10 hours single excursion event above 8 deg C came up.
Alarm and vaccine stability
I see that there is an effort in correlating the stability of the vaccine with the refrigerator alarms. I believe this is the whole argument. In reality, the alarms are intended to monitor the refrigerators, not the contents of the refrigerators. These alarms (and their frequency) tell you whether or not a refrigerator is operating appropriately. Any PQS prequalified fridge should stay within the alarm limits most of the time (an occasional alarm could occur if an external stimulus was applied).
Some contributors on the discussion thread seem to be making an argument that boils down to this “10 hours above 8 deg C will cause false alarms and the alarms should only go off when an excursion is severe enough to significantly reduce a vaccine’s remaining life, therefore we need to investigate for better alarms”. While I do agree that false alarms are very problematic, I don’t agree with this argument that any form of the 10 hours over 8 deg C alarm could be considered as false. Because alarms are set to reflect problems (hazardous situations) in refrigerator performance before they could cause harm to its contents. The alarm should, in principle, be set in a way to serve as a warning so control measures could be taken to bring back the performance of the refrigerator to the optimum level. By doing (fixing) this, you prevent the hazardous situation harming the content. It is not possible to correlate the problem directly with the stability of the vaccines since the hazardous situation is occurring at the last mile, where all products have already been travelled in the supply chain prior reaching to this particular refrigerator and most likely experienced various other excursions. Even you have full access of the detailed raw data from the device; you cannot reach certain conclusions regarding remaining shelf life of the products since you don’t have detailed data on the temperature storage conditions of each product prior to being placed into the current refrigerator. In this regard, vaccine vial monitor (VVM) is the only tool that can be instrumental in telling us what has happened to that particular vial before [from the high-temperature exposure perspective].
If you are running a study, following the product along the supply chain, you then perfectly correlate the temperature exposure data with the vaccine stability. But this has nothing to do with refrigerator alarms. A good example of such approach can be reviewed in recently published manuscript on following oxytocin from the manufacturer (Germany) to the service points in Ghana (http://kartoglu.ch/papers/00_stability_of_oxytocin_GHANA.pdf).
Suggesting a spectrodensitometer to be used to evaluate VVM is out of proportions of the field reality. First of all, such devices are used for “validation” purposes, mainly by the VVM manufacturer for quality and release purposes and by vaccine manufacturers when they receive orders. They need regular calibration. In addition, you cannot use spectrodensitometer to read VVMs on the vials. VVMs must be removed from the vials and put to flat surfaces to be read. Even you do this, at the end, your decision will only be “use” or “don’t use”. Who needs optical density values of reference ring and active surface of the VVM at the health centre level? Not mentioning the cost of the densitometer, I consider this suggestion as not realistic, impractical and technically inappropriate.
To summarize, the fridge alarms are for alerting users to offending fridges, the VVM is for monitoring cumulative heat on vaccine life.
Alarm philosophy and quality risk management perspective
The word “alarm” comes from Italian all'arm, meaning "to the arms", "to the weapons", telling armed men to pick up their weapons and get ready for action, because an enemy may have suddenly appeared. Alarm is a signal about a problem or condition, and requires an “action”. The action that is required in the original word in Italian is “picking up the weapons”.
In this regard, we do not support 2 and 8 deg C instant alarms for health centre and pharmacy type service provider refrigerators. Let me explain this with an analogy. If you want to get up at 6 am, what time do you set up your alarm clock? Would you ever set your alarm to 4 am? Because you are not going to wake up and get up at that hour, and there is no point of waking up and re-setting the alarm to 6 am at that time. If you think of other alarms you are familiar with in everyday life, do you know any alarm that there is no any action you should take? Think of a fire alarm, smoke alarm, low tire pressure [for cars] alarm, engine alarm, burglar alarm…
So, my point here is that what are the things you should do (your DO-LIST) when you have an instant alarm of 8 deg C in a health centre refrigerator? If you cannot come up with a DO-LIST, your alarm is not set appropriately. Similarly, there is not much you can do when you have an alarm of immediate 2 deg C. In our case, if you have an alarm of >8 deg C for 10 hours, this can only happen under certain conditions (external stimulus): Someone left the fridge door open (or the door is broken), someone played with the thermostat to a wrong direction, serious power cut for more than 24 hours, or someone loaded big amount of (warm) thermal mass inside (new shipment). So, your do-list could start with “check whether there is electricity”, and then “check the power cable whether it is connected properly”, and so on… If someone can give me a do-list for 8.1 deg C alarm, I would be delighted to learn. As for the cumulative heat exposure of the products, you may refer to the VVM readings and take appropriate actions.
As for the low alarm of -0.5 deg C, in addition to checking the thermostat position, the environment being extremely cold, you can conduct shake test to decide what to do with your freeze-sensitive vaccines. Here you have a do-list. Can you come up with a do-list for 1.9 deg C alarm for a health centre refrigerator?
I am sure some of you would remember the study in Albania to evaluate the value of SMS-enabled alarm systems at different levels of the cold chain. I do not want to get in details of this study, but when selecting a type of control measure, we need to think about implications. We cannot ask or introduce a technology for control because it is modern or top of the list. For me there is one simple question that needs to be answered here: “If there is an SMS alarm that you receive 3 am in your bed, whatever you would do, does it need to be done at that particular time or would there be any difference if you attend it in the morning when you reach the facility?” If what needs to be done is not something that must be done immediately upon receiving the alarm, then I do not see any reason to receive an SMS in the middle of the night.
From the risk management perspective, temperature monitoring devices are control measures for “detection”. They detect the predefined hazardous situation and alert workers with various types of alarm. Once we define the hazardous situation, we can think about the likelihood and the severity issues to calculate the risk score of the event. If the risk score falls in the red or amber category, we will have to do something and come up with control measures. We may not be able to eliminate hazards nor substitute them in all cases, but we may uncouple a process or reduce the number of steps or risk exposures that could occur. We may isolate the hazard so it presents fewer potential risks. We may also change conditions. More importantly, we may decrease the frequency of an event happening as well as decreasing the consequences should an event occur. In this sense, we can come up with redundant controls (like in the Swiss-cheese model) against each and every hazard. For example, PQS decision of having a sealed thermostat (preventive control measure) is to eliminate wrong thermostat setting (hazard). This control measure reduces the likelihood of temperature excursion events. Placing vaccines in groups based on their stability on different shelves is also a control measure. Shake test is a mitigation nature control measure and is used only once the hazard [exposure to freezing temperature] occurs. Shake test would also serve as a preventive control measure if you detect damage and discard vials [so, children do not receive this ineffective vaccine]. VVM is also a very good example of a control measure for both detection [the problem] and prevention [administrating heat damaged vaccine, vaccine wastage].
Warm regards,
UMIT
camp Jakarta
VVM's and SDAT Complementary Tools
The performance of a refrigerator over a one month period could be determined by analyzing the data logged
temperatures or by using an optical densitometer to determine the change in density
of a dot on a vaccine vial. If the performance of the refrigerator is what is of interest
it would be simpler to analyze the logged data with a tool such as SDAT , if the status of a
particular vaccine is of interest it would be easier to examine the dot on the VVM.
John’s Comments
Percent of Life Lost for Vaccines of Different Sensitivities
I think a point that John was making was that due to a temperature excursion the % of life lost would be different for vaccines of different sensitivities. It may seem counter
intuitive but for a given temperature excursion the % life lost will be the same for
vaccines of varying sensitivities. If a sensitive vaccine with a 100 day life loses 10% of it’s life it will have a 90 day life. If a vaccine with a 1000 day life experiences the same
temperature excursion it’s life will be reduced by 10% to 900 days. This fact simplifies
calculations and the presentation of results with SDAT.
MKT Temperature
MKT temperature is a weighted average
temperature where the higher temperatures count more than the cooler temperatures. If the temperature remains constant the MKT temperature will be the same as the temperature measured with a thermometer. John suggests the possibility of adding temperatures together I don’t think this addition would have any significance .
One scentence I could not understand “ The MKT gives a standardized energy unit related to vaccine Arrenhius behavior.” Temperature and energy are independent concepts , please clarify the thrust of what you were saying.
Current Alarm Set Points and 30DTR’s
If the 30DTR allows for downloading of data, the data could be analyzed and the percent of life lost for a month calculated. This could be done with a small computer or
microprocessor .
Unfortunately an alarm set for 10 hrs and 8 deg C will cause a large number of false alarms. Here is another example of a problem excursion. If the typical temperature of a refrigerator is 4 deg C and the temperature strays to 9 deg C for 11 hrs the alarm will sound , however for that days exposure the life of the vaccine will not be lowered when compared to the life of a vaccine continually kept at 8 deg C. If a time- temperature
alarm setting must be used there are probably better choices than 8 deg C for 10 hrs.
These choices should probably be inveatigated
To add some thoughts to this thread:
I think it is interesting to think about basing refrigerator temperature alarms on MKT and potential loss of vaccine life, as it provides a way to evaluate the severity of temperature excursion events in a more meaningful way than simply the amount of time over or under a specific temperature threshold. I want to additionally note the following:
I think there are two weaknesses embedded in this proposal for alarm rationale:
How about treating each 'accident' separately. The alarm would warn of an accident - the definition of an accident is quantified as:
The MKT gives a standardized energy unit related to vaccine Arrenhius behaviour.
Correct me if I am wrong, but I think the MKT value can be added from accident to accident. So the period of the alarm(s) in , say, a month could be used to give a value fo the performance of the fridge. If a batch of vaccine is in question, then because tteh system knows the movement of vaccine by lot.no. it can also assess the values of a chain of stores to assess the damage to the vaccine lot.
We can more easily rationalize the alarm limits as:
I suggest that you and I offer to draft a publication or a Technet discussion point on this?
I also think it is a good time to revive this debate and perhaps review the thresholds through a thorough study.
As Joanie stated, the threshold for alarms with 30DTR devices is preset by the product manufacturers based on WHO recommendations and even though countries are free to program central or remote devices with the alarm system they want, my impression is that countries would appreciate more evidence-based documentation linking the thresholds to the actual damage done to vaccines. As Larry wrote, damage to the vaccines is the criterion. When temperature studies show very significant freezing or exposure to heat that would trigger alarms but yet the VVMs and shake tests are negative, countries wonder if perhaps we set the bar too high.
The thresholds were set, based on different vaccination programs, 10 years ago. New vaccines and progress in vaccine stability have changed some of the parameters so perhaps it is time for review and reconsideration.
Hi John,
Thanks for the thread. I found it very informative. Here are some thoughts I had on alarms and temperature monitoring.
Proposed Monitoring and Alarm System
The purpose of an alarm is to minimize the percent of life lost during a temperature excursion. Basing the trigger point of the alarm directly on the percent of life lost would eliminate ambiguity. The proposed device could also display the percent of life lost for the entire temperature excursion. The change in color of the VVM would than correspond with the percent of life lost and displayed on the alarm.
Time Temperature Alarms
An alarm based on time and temperature does not correlate with the percent of life lost. If an alarm is set to be triggered for temperatures greater than 10 deg C for 10 hrs there can be temperature excursions which result in a larger percent loss of life but do not trigger the alarm. For example, exposing a vaccine to 20 deg C for 2 hours or 9.5 deg C for 30 hours would both result in greater loss of life but the alarm would not be triggered. In the thread Joanie Robertson noted that “VVM changes do not correspond with alarm changes”. Basing the alarm on percent of life lost would eliminate this discrepancy.
When stored at 8 deg C the life of new polio vaccine is about 120 days. If the vaccine was in a 10 deg C environment for 10 hours the loss of life of the vaccine would be 0.2 days less and the dot on a VVM would be essentially unaffected. If the acceptable loss of life was extended to 1 day a number of false alarms could be averted while the loss of life of the vaccine would still be small.
Loss of Life Calculations
The loss of life of a vaccine is determined with the aid of MKT temperature. We call our analytic method SDAT [Simplified Data Analysis Tool]. MKT temperature and the color density of a VVM are both based on the Arrhenius equation which describes the relationship between temperature and the rate of degradation of a vaccine. This is why results obtained by both methods are in agreement with each other.
SDAT and the Analysis of Monthly Data
If a refrigerator is not running properly and time temperature data is available for the month the percent of life lost can be calculated based on the monthly temperature data. The results can then be displayed for vaccines of different sensitivities. Without doing an analysis based on the Arrheius equation I don’t think the effect of the temperature excursion on the life of the vaccine can be determined. Using SDAT would greatly simplify analyzing the months data.
An analysis of the plotted raw data would be useful to troubleshoot a refrigeration problem to determine if there is, for example, a bad battery or a door left open.
SDAT and Transport
SDAT would be valuable for evaluating equipment such as vaccine carriers and refrigerators and also help to evaluate temperature excursions during transport. A display based on SDAT would show the percent of life lost for a journey. A small self contained unit could be built to do the calculations and display the results. If vaccines are transported in a carrier and during a one week trip the temperature in the carrier linearly climbs to 21 deg C the SDAT analysis will show that the life of new Polio vaccine will be reduced from 120 days to about 30 days. Thirty days may be long enough to carry out an effective vaccination program. Using SDAT will allow longer trips to be made with confidence and also help in evaluation of a vaccine carrier.
I would like to continue this work, however, I could not do it without financial support. If there are any ideas on how I can further support this work please let me know.
Larry Schlussler Phd
Sun Frost
Your question is well timed Larry! The best answer I can relay to you is a summary of a current email thread (below).
I will now try to learn from Umit the basis for the current WHO settings so that we can discuss the best answer.
EMAIL THREAD (condensed)
JOHN 24/08
The attached thread is just ten years old .... and still unresolved! From time to time I find myself defending the WHO alarm criteria which are:
…against country managers or those that advise country managers who would prefer much tighter criteria when they are used to trigger visual warning messages, or audio- alarm bells and when they automatically send warning SMS messages to supervisors or technicians. I usually claim that WHO made a thorough review of vaccine heat sensitivity when choosing the values for the criteria and that field studies such as the Optimize project in Albania clearly showed the loss of impact of alarms that sounded too many times and for too long.Quite reasonably, I am usually asked for a document references that underpin the key time/temperature thresholds. Although I can cite the Albania study and the annotated bibliography on freezing vaccines, the crucial decision to choose these values seems to be un-documented.
Can any of you who participated in the ten year old discussion direct me to a source and, if not, what do you think can be done to re-inforce these WHO norms? Its important because more remote monitoing systems are automatically tracking alarm signals and ‘temperatures restored’ signals to drive dashboards calling for managerial action. Freezing alarms have become particularly influential for judging the vaccine handling performance of area/country supply chains.
SOLO, WHO 24/08
To me the alarms limits are given for management purposes and not necessarily to serve as a norm. But since there should be a mean of comparability across countries, the criteria are set by WHO (I think you were the author...NOT ME, JOHN) for reference. You are right, with time and experience these criteria/limits need to be reviewed and adjusted to new reality.
SOREN SPANNER 25/08
I remember that Umit worked a lot with Berlinger, he would know the reason for the WHO criteria. As I understand it, countries wants stricter criteria, which to my best knowledge makes no sense. Will just make much more unnecessary alarms which again will make people ignore the alarms and finally switch it off.
JOANIE ROBERTSON, PATH 25/08
I agree that it’s best to have limits that will avoid overalarming, because as we all know, people will quickly stop paying attention, especially if VVMs changes do not correspond with alarm conditions over time. Probably worth a review every now and then. A couple addition points:
JAMES CHEYNE 30/08
Perhaps this is an opportunity to open up a public debate of advantages and disadvantages of 'tight' limits on when an alarm sounds (0C and 9C) and looser limits (say, 0C and 15C). And tightness of the alarm delay: say, 5 mins or 30 mins.
