Hazards Planning Center

Measuring Progress

Measuring Success in Recovery

Executive Summary

Recovery from disasters is a key capability for federal, state, and local governments. To support this capability, practitioners at all levels need useful and validated metrics to measure and monitor how well a community is recovering from a disaster over time. Practitioners' need for these metrics is echoed by the hazards research community, which over the last decade has made the case for more systematic ways of measuring the disaster recovery process across events and over time to improve planning for, and recovering from, disasters (Berke, Kartez, and Wenger 1993; Peacock et al. 2008).A high-quality recovery process informed by data can provide an opportunity to build future resilience by taking advantage of the increased interest and resources present

Printer version (pdf)

Key Points

#1: The identification of standards and metrics for assessing the effectiveness of recovery efforts is a major challenge.

#2: The preexisting disaster recovery plan is one way to identify indicators and metrics for measuring recovery.

#3: Metrics must capture important differences in both the magnitude and speed of recovery for socially vulnerable populations.

KEY POINT #1: The identification of standards and metrics for assessing the effectiveness of recovery efforts is a major challenge.

Valid and reliable metrics that can be utilized across disasters, over time, and in different geographic locations are a necessary part of increasing resilience by providing data to inform planning, preparedness, and early interventions.

The sense of urgency that prevails after a disaster may lead local governments to make decisions in the short term that foreclose long-term options to reduce a community's vulnerability to future disasters. However, sustainable recovery should provide an opportunity to improve upon, rather than recreate, pre-disaster vulnerabilities (Smith and Wenger 2006). To do this, systematically collected and shared data that characterize the baseline condition of a community and track recovery over time is needed.

The identification of a robust set of recovery indicators, with quantifiable metrics for assessing the effectiveness of national recovery efforts, has been identified as a major challenge requiring federal investment (National Research Council 2012). Recovery indicators should be linked to the guiding framework of federal recovery policy as outlined by the Federal Emergency Management Agency (FEMA) in the National Disaster Recovery Framework. This linkage will help ensure and support local, state, and regional coordination with the federal government as each works towards recovery. If the "whole community" (FEMA 2011) is to successfully work together to keep the nation safe and resilient in the face of disasters, this type of coordination around federal recovery policy will support comparability within communities and across disasters. Working as a "whole community" can also help build networks of practitioners across potentially siloed functions such as planning, emergency management, and public health. These networks are needed to successfully take action on the multiple fronts required to reduce vulnerability and build resiliency in the future.

Working within the framework of federal recovery policy, there is great opportunity for the development and use of measures that are flexible, easy to assess, costeffective, and useful for decision making in the practice, policy, or research setting. Following established best practices, recovery metrics should be able to be measured and assessed repeatedly over time. They should be sensitive to changes in community recovery status over time and within key demographic and geographic subpopulations. Finally, the effect of both community- and individual-level experiences should also be considered.

A number of recovery measures have already been developed and pilot tested as part of a variety of assessments, scorecards, and toolkits. Many of these are likely underutilized by both practitioners and researchers. For example, the National Oceanic and Atmospheric Administration's Coastal Resilience Index allows community leaders to identify potential problems that should be addressed in a disaster recovery plan before the next disaster, as well as to identify resources to address them; the Rockefeller Foundation's City Resilience Framework provides measures that help users relate resilience to 12 measurable aspects of health, economy, leadership, and systems; and the Disaster Recovery Tracking Tool, developed by the University of North Carolina at Chapel Hill's Coastal Hazards Center of Excellence and FEMA-New York provides users with 79 metrics for tracking recovery in 10 focus areas. A community's own pre-disaster recovery plan can also establish baseline conditions, define measures, outline monitoring programs, and design policies that support the use of data for assessing the effectiveness of recovery.

For communities that have successfully characterized a baseline and adopted measures to assess progress toward a resilient future, recovery from a disaster can provide an opportunity for economic development, restoration of social networks, or revitalization of the role of civic organizations. The process of identifying recovery metrics that fit a community's setting, context, and values is a challenge that can be surmounted. First, federal recovery policy can provide an effective coordinating structure for the key functional areas of assistance typically needed by a community after a disaster. Second, existing indices and measures can be adapted to address specific risks and opportunities. Finally, robust, high-quality plans can provide the fact base that can assist in the development of indicators, as well as a road map for the use of data in the recovery process.

KEY POINT #2: The preexisting disaster recovery plan is one way to identify indicators and metrics for measuring recovery.

The failure to measure recovery outcomes against a local recovery plan limits planners' ability to evaluate and revise recovery policies and programs and decreasesthe chances of having a positive impact on ultimate recovery outcomes.

Rebuilding a community after disaster strikes is a major undertaking. The most effective way to accomplish holistic post-disaster recovery is to be prepared before a disaster strikes. Developing a post-disaster recovery plan requires envisioning the potential obstacles to reconstructing a community in a compressed timeline — not just rebuilding what was there prior to the disaster event, but redeveloping a more sustainable and disaster-resilient community with participation from all stakeholders impacted by the disaster (Berke and Campanella 2006).

Recovery indicators should offer relevant feedback throughout the post-disaster recovery plan implementation process. When indicators are linked to recovery plans, the information derived from tracking indicators can be used to assess achievement of progress toward plan goals, the performance of plan policies, and the updating of policies and programs. In contrast, indicators designed to track recovery outcomes directly, bypassing recovery plan goals and policies, are less likely to convey information that can be converted into information relevant to decision making. Indicators that are isolated from planning will not contribute to meaningful assessments of the cost, speed, efficiency, and sustainability of recovery policies and programs. Indeed, "Indicators alone are idle information which hardly convey any meaningful message for policy-making. It is the analysis of indicators against the wider context and policy objectives that provides the added value of converting information into intelligence." (Wong 2005)

The key to designing indicators linked to recovery plans is to determine how change can be measured in an understandable manner with data that can be collected using a reasonable amount of resources. Other important questions to consider in linking indicators to a recovery plan include:

  • Does the indicator account for goals that reflect stakeholder needs and aspirations?
  • Can the indicator be customized to fit an individual agency charged with implementing an element in the recovery plan?
  • Is the cost/benefit relationship sensible (in terms of data availability or resources needed to collect data versus value)?
  • Can a source of information accountable for providing the data be identified?
  • Will the measure be sustainable to track plan performance over a period of years?

Indicators should be designed for a wide array of goals and policies that might be included in a recovery plan. Two general classes of goals include transformative goals that deal with building back better and restorative goals that are aim to alleviate losses efficiently and quickly. Table 1 illustrates examples of goals, policies, and indicators that track achievement of goals.

Development of recovery indicators can also help coordinate recovery planning. Communities can choose among different combinations of several types of plans that can be used to influence where and how local recovery should occur, and at what rate:

  1. an emergency management plan addresses emergency response activities that focus on immediate actions to protect property, remove debris, and begin repair of water, sewer, and other essential services needed to meet basic human needs;
  2. a comprehensive plan directs the long-range location, type, density, and rate of physical development; and
  3. a stand-alone recovery plan offers an integrated guide that ties short-range emergency actions with longrange redevelopment decisions (Florida DCA 2009).

Measuring recovery outcomes based on a comprehensive and well-conceived set of indicators permits planners to identify congruencies, gaps, and conflicts among plans and program implementation. This allows planners to revise recovery policies and programs early on, thereby improving the chance of having a positive impact on the ultimate recovery outcomes while also saving time, resources, and opportunity costs.

Table 1: Examples of Plan Goals, Policies, and Indicators

  Policy to Achieve Goal Indicators
Transformative Goals    
Blight removal Nonconforming use designation % of nonconforming buildings discontinued
Smarter Growth Zoning districts that allow rebuilding in priority areas with mixed uses and high densities % of homes and businesses in mixed use centers
% of homes in proximity of transit route
Enhanced public safety Fee-simple land acquisition to remove development from hazard zones
Infrastructure provisions that replace damaged facilities to safer locations
% of housing units relocated from hazard zone
% of total linear feet of water and sewer lines steered away from hazard zones
More equitable distribution of services and facilities    
Restorative Goals    
Economic resumption to keep businesses alive Temporary tax abatement program % preexisting businesses operating
% jobs restored
Replace/repair development to prior conditions Temporary repair permitting
Emergency demolition regulations to remove unsafe structures that may impede neighborhood redevelopment
% of occupants in homes after disaster (by week or month)
% of structures rebuilt (by week or month)

KEY POINT #3: Metrics must capture important differences in both the magnitude and speed of recovery for socially vulnerable populations.

Uneven recovery trajectories for socially vulnerable populations can leave entire neighborhoods susceptible to rapid change, population displacement, and redevelopment, undermining community plans and exacerbating preexisting disparities.

Although natural disasters magnify and accelerate processes already occurring in communities (Olshansky, Hopkins, and Johnson 2012) not all processes compress or accelerate at the same rate during the recovery. The result can be a distortion in the relationships between redevelopment and decision processes. Recovery may vary greatly among subpopulations based on social vulnerability factors (Peacock et al. forthcoming). Socially vulnerable households include those who are minority, low-income, female-headed, elderly, and/or renters. Often these characteristics are present in combinations (e.g., both minority and low-income), compounding the households' vulnerability (Morrow 1999). Socially vulnerable households are more likely to live in less desirable and likely more risky (i.e., low-lying) neighborhoods, in lower-quality and less well-maintained homes. As a result, these households are more likely to receive higher-than-average levels of damage. Further, socially vulnerable populations often have no or inferior insurance and poorer access to financial resources that can aid in recovery.

As a result, socially vulnerable households will take longer to recover, and their housing units are less likely to reach full restoration of pre-disaster values. Further, because these populations are often spatially concentrated, more uneven recovery trajectories can leave entire neighborhoods susceptible to change and redevelopment (Zhang and Peacock 2010). The local context of housing recovery (i.e., the sociodemographic composition of the population, the local housing market, and the mix of housing types) may result in significant variation in the way that social vulnerability factors matter. In some areas, race or ethnicity may be a critical factor in predicting recovery times and levels, while in others, income may be more critical (Peacock et al. forthcoming). Housing tenure — whether one owns or rents — is also critical, since renters, particularly those that are low-income, have lower levels of housing security (Rohe, Van Zandt, and McCarthy 2001) and have fewer resources (lower incomes, less job stability, and less savings) with which to address maintenance, repair, and recovery (McCarthy, Van Zandt, and Rohe 2001; Van Zandt and Rohe 2011). Consequently, recovery metrics must capture important differences in both the magnitude and speed of recovery for these subpopulations within the community.

While housing recovery is only one area of recovery, it is probably the strongest indicator of overall community recovery and, when disaggregated, can reflect recovery for constituent subpopulations. Aggregate measures and average values of housing recovery will miss important differences. For example, in many communities, single-family housing will dominate the housing stock. Looking at average values of housing recovery will mask important differences that may be occurring within multifamily or other types of housing that is more often inhabited by renters, who are also more likely to be lower-income, minority, and otherwise socially vulnerable. Consequently, it is important to disaggregate housing recovery by housing type, neighborhood sociodemographic characteristics (race/ethnicity, income, female-headed households, age, and tenure), and neighborhood location. Extending this to other metrics of recovery is also important. For example, business recovery should also be disaggregated to uncover important systematic differences in how businesses with different characteristics (owner characteristics, sizes, industries, locations) may be recovering or languishing.

Tax appraisal data from county appraisal districts is a terrific source of data that permits tracking of the pace of housing recovery by housing type. Most often, recovery is measured by the percentage of pre-disaster value reflected in the appraised value. Restoration is achieved when the unit is appraised at a value equal to its pre-disaster value. Appraisal data can also be used to identify where long-term vacancies are occurring and where land uses are changing, both signs of impending and occurring redevelopment. While sales data are another great source, they are not available in every state and not for every property. Both types of data can be merged with block group or tract-level sociodemographic data from the American Community Survey or U.S. Census to assess differences in both pace of recovery (how quickly the values return to pre-disaster values) as well as the magnitude of recovery (whether or not they achieve or surpass pre-disaster values) among neighborhoods based on income, race/ethnicity, tenure, female-headed households, and age. Building permit data can help understand where rebuilding is taking place and where it may not be. Importantly, spatial analysis using GIS permits the assessment of differences at the neighborhood level to help identify areas that may not be recovering adequately.

Disasters can provide a focusing event that heightens awareness of inequalities present before, during, and after the event. Tracking inequalities with effective metrics will enhance the ability of recovery agents to highlight needs and qualify for additional assistance based on disaster impacts and recovery needs. During recovery, there is a window of opportunity to use both community commitment and the influx of resources that often occurs to undertake transformational action to reduce rather than exacerbate preexisting inequalities.

Conclusions

  1. Long-term, coordinated, systematically collected, and shared data on recovery is needed to effectively improve community resilience to future disasters
  2. The use of metrics to track recovery is necessary to see what's getting better and what isn't. Data can also provide policy makers and community stakeholders with the information needed to identify priorities and make sound decisions about their future.
  3. Metrics linked to recovery plans are more likely to convey information that can be converted into meaningful assessments of the cost, speed, efficiency, and sustainability of recovery policies and programs.
  4. Metrics can help focus assistance on individuals and communities where it is most needed — and can potentially highlight inequalities in recovery that should be addressed.

Resources

Berke, Philip and Tom Campanella. 2006. "Planning for Post-disaster Resiliency." The Annals of the American Academy of Political and Social Science, 604: 192–207.

Berke, P., J. Kartez, and D. Wenger. 1993. "Recovery after Disaster: Achieving Sustainable Development, Mitigation and Equity." Disasters. 17(2). pp. 93-109.

Federal Emergency Management Agency. 2011. "A Whole Community Approach to Emergency Management: Principles, Themes and Pathways for Action." Available at www.fema.gov/media-librarydata/20130726-1813-25045-0649/whole_community_dec2011__2_.pdf.

Florida Division of Community Planning (FL DCA). 2009. Post-Disaster Redevelopment Planning. Available at www.floridajobs.org/fdcp/dcp/PDRP/Files/PDRPSeaLeveRiseAddendum.pdf.

McCarthy, G., S. Van Zandt and W.M. Rohe. June 2001. "The Economic Benefits and Costs of Homeownership: A Critical Assessment of the Research." Working Paper 01–02. Washington, D.C.: Research Institute for Housing America.

Morrow, B. H. 1999. "Identifying and mapping community vulnerability." Disasters, 23(1), 1–18.

National Research Council. 2012. Disaster Resilience: A National Imperative. Washington, D.C.: National Academies Press.

Olshansky, R. B., L.D. Hopkins, and L.A. Johnson. 2012. "Disaster and recovery: Processes compressed in time." Natural Hazards Review, 13(3), 173–178.

Peacock, W.G., H. Kunreuther, W.H. Hooke, S.L. Cutter, S.E. Chang, and P.R. Berke. 2008. "Toward a Resiliency and Vulnerability Observatory Network: RAVON." Available at archone.tamu.edu/hrrc/publications/researchreports/RAVON. 2008.

Rohe, W.M., S. Van Zandt, and G. McCarthy. 2001. "The Social Benefits and Costs of Homeownership: A Critical Assessment of the Research." Working Paper 00–01.

Washington, D.C.: Research Institute for Housing America. Morrow, Betty Hearn, and Elaine Enarson. 1997. "A Gendered Perceptive: The Voices of Women." Chapter 7, pp. 116–140, in Hurricane Andrew: Ethnicity, Gender and the Sociology of Disasters. London: Routledge.

Smith, G.P., and D. Wenger. 2006. "Sustainable Disaster Recovery: Operationalizing an Existing Agenda." Pp.234– 274 in Handbook of Disaster Research, ed. H. Rodriguez, E.L. Quarantelli, and R. Dynes, New York: Springer.

Van Zandt, S., and W.M. Rohe. 2011. "The Sustainability of Low-income Homeownership: The Incidence of Unexpected Costs and Needed Repairs Among Low- Income Home Buyers." Housing Policy Debate 21(2): 317–341.

Wong, Cecilia. 2005. Indicators for Urban and Regional Planning: The Interplay of Policy and Methods, Taylor & Francis.

Zhang, Y., and W.G. Peacock. 2010. "Planning for housing recovery? Lessons learned from Hurricane Andrew." Journal of the American Planning Association, 76(1), 5–24.

This briefing paper was written by Jennifer Horney, Philip Berke, and Shannon Van Zandt. Horney is an associate professor of Epidemiology and Biostatistics at Texas A&M Health Science Center School of Public Health, College Station. Contact her at 979-436-9391 or horney@sph.tamhsc.edu. Berke is a professor of Landscape Architecture and Urban Planning at Texas A&M College of Architecture, College Station. Contact him at 979-845-7813 or pberke@arch.tamu.edu. Van Zandt is an associate professor of Landscape Architecture and Urban Planning at Texas A&M College of Architecture, College Station. Contact her at 979-458-1223 or svanzandt@ tamu.edu.

Cover photo: © Dustin Henry.