BUILD IN EVALUATION AND CONTINUOUS IMPROVEMENT.
Public health solutions and programs start out based on what we know at the beginning. However, once the program is implemented and engagement continues, we learn more and can see how well our strategy and tactics are working. Knowing that we want to improve what we are doing leads us to build evaluation into our program from the very beginning. The program must be examined frequently – as it develops, not waiting until it is finished – to see what is working and how it could be improved. We should solicit improvement ideas from everyone working on our project and listen to people at all levels, asking them how things can be improved.
PART 5.1
In the battle to eradicate smallpox in India in 1974, finding and containing thousands of outbreaks was the key to getting smallpox under control. In mid 1974, the number of new outbreaks was starting to decrease slightly. The strategy of surveillance and containment - which included a massive workforce, tireless efforts, and rewards for identifying new cases - was paying off. But then we started to notice that members of previously vaccinated families were still coming down with smallpox. We should not have been seeing these new cases.
FEATURING
Dr. Bill Foege, Senior Advisor, Bill & Melinda Gates Foundation
Dr. Susmita Parashar, Associate Professor, Emory University School of Medicine
PART 5.2
The polio eradication program relied on strong surveillance efforts to inform vaccination efforts. Surveillance teams worldwide constantly searched for cases in which people had limbs that could not be moved and just hung limply, or cases of “flaccid paralysis.”When they found these cases, they would target vaccination to those regions. This was a very effective strategy when polio was widespread, because almost every case of flaccid paralysis was caused by polio. As vaccination coverage increased, cases of paralytic polio plummeted. As cases of flaccid paralysis began to disappear, polio was declared eliminated from many countries and regions. But in many areas polio persisted. It was just harder to find. First, there were other causes of paralysis among children. As polio became much less frequent, the other causes of flaccid paralysis became more frequent than polio, so flaccid paralysis was no longer a good indicator of how widespread polio was in a particular area. And second, periodically, polio outbreaks still continued to occur and these would then seed the virus to different parts of the world and threaten eradication. As cases of paralysis decreased, it became tougher to find the few cases that continued to occur. For every case missed, workers were missing several hundred cases of silent or asymptomatic infection in the community and these silent cases continued to spread the virus.
FEATURING
Dr. Manish Patel, Influenza Prevention & Control Team Leader, CDC
PART 5.3
Infants were dying in rural health centers because they had complications that couldn't be dealt with in those hospitals. Doctors thought that a way to address this would be to transfer them to a larger hospital with more neonatal capacity. But, when the transfer solution was tried, it turned out that more infants were dying. Lives were not being saved. Babies continued to die because the transportation to the secondary or tertiary facilities took so long. The infants could not survive the trip.
FEATURING
Dr. Nana Twum Danso, Senior Vice President, Institute for Healthcare Improvement and Founder and CEO of MAZA
PART 5.4
Gun violence takes an extraordinary toll on individuals, families, and communities in the United States. But the NRA has been successful in working through Congress to make sure that much of the critical data for assessing the problem, and finding interventions that work, is not collected, maintained, or made available.
FEATURING
Dr. Mark Rosenberg, President Emeritus, The Task Force for Global Health
BUILD IN EVALUATION AND CONTINUOUS IMPROVEMENT.
When we plan and implement a program for the first time, we don't know how well it will work. But we can study how well it is working and improve it as we go.
You may not be successful the first time, but failure is not a terrible thing. We are bound to make mistakes, but we can see these as opportunities to learn how we can do better.
Don’t wait until your program is completed before thinking about how you will evaluate it. Plan your evaluation from the very beginning. This will help you see the kinds of data you should be collecting.
Evaluation is a never-ending process. It should be part of a cycle that begins with planning, doing, assessing, then improving and repeating the cycle.
If you are honestly evaluating your work, you will always find something that needs to be improved.
Measurement and evaluation are necessary for success. We must keep searching for answers and adapting to new information.
New tools and new approaches may be necessary to find the last remaining cases of a disease, to reveal more about disease transmission, and to improve surveillance. So we say that “The more successful you are at preventing new cases, the closer you have to look to find more cases. The closer you look, the more you see. The more you see, the more you learn.”
LESSON 5:
THE BIG IDEAS
To learn more about this topic, refer to the additional resources below. For in-depth concepts, refer to the deeper dives. Click the glossary link to find key terms in this lesson.
Excerpt from Vision Zero Handbook – Vision Zero in Disease Eradication
Plan-Do-Study-Act (PDSA) Directions and Examples | Agency for Healthcare Research and Quality
How to Improve | IHI - Institute for Healthcare Improvement
Deeper Dives
Exemplars research is studying best practices from around the world and what has worked to reduce neonatal and maternal deaths. Learn more: Neonatal and Maternal Mortality.
Adara Development is working in Uganda to bring evidence-based models and Centers of Excellence to support mothers and babies from pregnancy through the moment they return home after birth, reducing infant deaths. Learn more: Working to save babies – and mothers – in Uganda.
How can Africa’s lessons on polio surveillance and mass vaccination support Mozambique and Malawi with new outbreaks? Learn more: Africa’s polio lessons may help Mozambique and Malawi contain new outbreaks.
LESSON 5:
Additional resources
THIS LESSON WAS MADE POSSIBLE BY THE FOLLOWING CONTRIBUTORS
Kate Adelung, Nina Alcacio, Michael Bartenfeld, Sarah Borgman, Amanda Brayman, Hannah Burris, Amy Carzo, Prarthna Desai, Gabriel Diamond, Veronica Garcia, Gates Notes, Eric Hansen, Lynn Heinisch, Mary Hilpertshauser, Bruce Lowry, Amy Mayberry, Nancy Messonnier, Matthew Morse, Ben Pyne, Sumon Ray, Svetlomir Slavchev, Joel Stanojevich, Emily Staub, Kelly Stewart, Anushka Swalef, Lisa Valente, Bill Warren, Lisa Wiley, and Sydney Yang
Images and Videos provided by: Bill & Melinda Gates Foundation, CDC Public Health Image Library, Emory University, FIA Foundation, Gavi, The Vaccine Alliance, Getty Images, MAZA, Personal collection of Manish Patel, Personal Collection of Susmita Parashar, Rich Addicks, Richard Stanley Productions, Shutterstock, The Carter Center, The Task Force for Global Health, and the World Health Organization
OUR FUNDERS
CONRAD N. HILTON FOUNDATION, CDC FOUNDATION, FORD FOUNDATION, GATES VENTURES, ROCKEFELLER FOUNDATION, SKOLL FOUNDATION, THE CARTER CENTER, THE TASK FORCE FOR GLOBAL HEALTH