A consensus process on the use of exercises and after action reports to assess and improve public health emergency preparedness and response.

INTRODUCTION: The objective of disaster preparedness is to ensure that appropriate systems, procedures, and resources are in place to provide prompt, effective assistance to disaster victims, thus facilitating relief measures and rehabilitation of services. Disaster preparedness efforts include the identification of possible health scenarios based on the probability of hazards and vulnerability of the population as a basis for creating a disaster plan. Exercises that simulate emergency response, involving the health and other sectors, have been suggested as useful tools to test the plans on a regular basis and measure preparedness efforts; the absence of actual testing is likely to negate even the best of abstract plans. Problem Exercises and after action reports (AARs) are used to document preparedness activities. However, to date, limited analysis has been performed on what makes an exercise an effective tool to assess public health emergency preparedness (PHEP), and how AARs can be developed and used to support PHEP improvement efforts. The scope of this project was to achieve consensus on: (1) what makes an exercise an effective tool to assess PHEP; and (2) what makes an AAR an effective tool to guide PHEP improvement efforts.

METHODS: Sixty-one PHEP experts were convened by the use of Nominal Group Techniques to achieve consensus on a series of characteristics that exercises should have when designed to assess PHEP and on the recommendations for developing high-quality AARs.

RESULTS: The panelists achieved consensus on a list of recommendations to improve the use of exercises and AARs in PHEP improvement efforts. Such recommendations ranged from the characteristics of the exercise audience to the evaluation methodology being used and the characteristics of the produced AAR such as its structure and content.

CONCLUSIONS: The characteristics of the exercise audience, scenario and scope are among the most important attributes to the effectiveness of an exercise conducted for PHEP evaluation purposes. The evaluation instruments used to gather observations need an appropriate matching between exercise objectives and the response capabilities tested during the exercise, to build the base for the production of a good AAR. Improvements in the design and creation of exercises and AARs could facilitate better reporting and measurement of preparedness outcomes.