The Hidden Costs of "Unnecessary" False Alarms
Michael R. Smith
WeatherData, Incorporated
January 15, 2003
The Problem
Even with the considerable progress in atmospheric science during the twentieth century, there remains considerable room for improvement in the accuracy of the public warnings of tornadoes, flash floods, large hail and damaging thunderstorm winds. But even if we had perfect knowledge of the process of tornadogenesis, for example, inadequate atmospheric sampling and measurements would still prevent warning accuracy from ever reaching 100 percent.
Stewart (2000) states the problem succinctly:
Every prediction contains an element of irreducible uncertaintyÉ actions
that are based on predictions lead to two kinds of errors. One is when an
event that is predicted does not occur, i.e., a false alarm. The second is when an event occurs but is not predicted, i.e., a surprise. There is an inevitable tradeoff between the two kinds of errors; steps taken to reduce one will increase the other.
This article examines the effects of false alarms and proposes two classifications of false alarms. Further, some rudimentary estimates of the costs of false alarms are presented.
Two Types of False Alarms
To use tornado warnings as an example, if one were to turn the clock back fifty years, tornado warnings were in their infancy. The "hook echo" (Bigler 1956) was yet to be discovered. The initial Weather Bureau tornado warnings were issued on a countywide basis, in part, because our knowledge was so lacking it was not possible to isolate the portions of the thunderstorm where a tornado was likely to occur.
There has been substantial progress in public severe storm warnings since the 1950's. The Critical Success Index (AMS, 2000) is used by meteorologists as a combined measurement of the accuracy of storm warnings and includes both surprises and false alarms as defined above. Table 1 conveys the CSI for tornado warnings for the past 15 years. Perfect, all storms detected with no false alarms, is 1.00.
| Year | CSI |
|---|
| 1986 | .137 |
| 1987 | .128 |
| 1988 | .128 |
| 1989 | .147 |
| 1990 | .209 |
| 1991 | .179 |
| 1992 | .223 |
| 1993 | .199 |
| 1994 | .197 |
| 1995 | .216 |
| 1996 | .199 |
| 1997 | .194 |
| 1998 | .179 |
| 1999 | .250 |
| 2000 | .210 |
| 2001 | .259 |
Table 1. Tornado Warning CSI.
Given the progress made in storm warnings the past ten years, I believe it is now possible to define two types of meteorological false alarms: One type is due to limitations in meteorological science. For example, Doppler radar may indicate moderate rotation in an appendage in the right rear quadrant of a supercell thunderstorm and a tornado warning issued, but, for unknown reasons, because of our incomplete knowledge, no tornado occurs. Given the current state of atmospheric knowledge and atmospheric measurement does not allow for anything close to perfection, I define these as unavoidable false alarms.
However, in contrast to 50 years ago, meteorologists now know that a tornado is very unlikely to occur in the left rear quadrant of a supercell that exhibits little rotation. Because of the incremental improvement of knowledge of tornadic behavior, tornado warnings can be confidently issued on a sub-county basis using latitude and longitude coordinates and these coordinates are included in National Weather Service warnings. (NWS, 2002) Unfortunately, once the warning leaves the NWS, the warning system, in most parts of the United States is not sophisticated enough to deliver sub-county warnings effectively (i.e., to warn people in the specific path of the storm and not warning those who will be unaffected by the storm). I define these as unnecessary false alarms.
False alarms, regardless of type, create a number of undesirable consequences: loss of credibility of the warning system, increased risk to citizens (i.e., there will inevitably be traffic accidents during a false alarm hurricane evacuation that would not have occurred if an evacuation had not been ordered), and economic loss. The economic effects can be large. Pielke (1999) states, "Floyd holds the dubious distinction of being the first billion-dollar storm in which the costs of the evacuation rival the costs of the storm's impacts."
While some of the overwarning (an informal term used by meteorologists to describe an alarm that is false because it is too large as opposed to meteorologically incorrect) associated with Floyd was unavoidable (i.e., uncertainty associated with the point of and strength at landfall), part was also unnecessary. If a perfect landfall forecast was made, the current warning system cannot differentiate and communicate the major differences in the level of threat in the 30 miles separating the right front quadrant of the eye from the left front quadrant just outside of the eye structure.
Thunderstorms and tornadoes can also cause large unnecessary overwarnings. Figure 1 is a tornado warning for Kingman and Sedgwick Counties in Kansas. Wichita, the largest city in Kansas, is in Sedgwick County.
WFUS53 KICT 112312
TORICT
KSC095-173-120015-
BULLETIN - EAS ACTIVATION REQUESTED
TORNADO WARNING
NATIONAL WEATHER SERVICE WICHITA KS
612 PM CDT THU APR 11 2002
THE NATIONAL WEATHER SERVICE IN WICHITA HAS ISSUED A
* TORNADO WARNING FOR...
SEDGWICK COUNTY IN SOUTH CENTRAL KANSAS
KINGMAN COUNTY IN SOUTH CENTRAL KANSAS
* UNTIL 715 PM CDT
* AT 612 PM CDT...NATIONAL WEATHER SERVICE DOPPLER RADAR INDICATED A
SEVERE THUNDERSTORM WITH STRONG ROTATION 26 MILES WEST OF DOWNTOWN
WICHITA...OR 5 MILES NORTHWEST OF CHENEY...MOVING SOUTHEAST AT 10
MPH. THIS STORM COULD PRODUCE A TORNADO AT ANY TIME.
* THE SEVERE THUNDERSTORM WILL BE
NEAR CHENEY AROUND 635 PM CDT...
IF NO UNDERGROUND SHELTER IS AVAILABLE...MOVE INTO AN INTERIOR ROOM
ON THE LOWEST FLOOR OF A STURDY STRUCTURE.
REPEATING...A TORNADO WARNING HAS BEEN ISSUED FOR UNTIL 715 PM CDT.
LAT...LON 3772 9779 3768 9791 3757 9782 3762 9767
/CARUSO
(Figure 1)
At the end of the tornado warning is the set of latitude and longitude coordinates plotted in Figure 2.
(Figure 2)
Only a tiny portion of Sedgwick County is threatened; it is largely rural, well west of the city limits of Wichita. Because Sedgwick County has an "all or nothing" system for sounding tornado sirens (either all 134 sound or none of them sound), the emergency manager was forced to trigger the sirens over the entire county, including Wichita.
I was exercising at a health club in west Wichita at the time the sirens sounded. The health club emptied out almost immediately, even though the management of the health club did not make an evacuation request. I was told by the manager of a restaurant in east Wichita, which employs a sophisticated revenue projection system, that the tornado warning cost him $1,700 in lost revenue, even though the sky was partly blue at that location during the tornado warning (i.e., potential restaurant patrons would not have perceived a threat if not for the tornado warning). The Wichita yellow pages list approximately 700 restaurants. Using a conservative figure of $800 per restaurant (in west Wichita, lightning occurred with light rain, which might have given the tornado warning even more credibility than on the east side), the "false alarm cost" to the Wichita restaurant industry that evening was around $560,000. If this figure is roughly correct, the loss of sales tax revenue was on the order of $30,000 for the restaurant industry alone. When one considers health clubs, restaurants, movie theatres, etc., it is likely the false alarm cost was on the order of $1,000,000 with around $55,000 in lost tax revenueÉall unnecessary (i.e., it wasn't lack of meteorological skill, but rather an ineffective warning system that caused the loss). Assuming the figure of $1,000,000 is approximately correct, it is more than double the cost of upgrading the siren system to selective siren activation which would prevent these losses.
In no way are these comments meant to criticize the Wichita office of the National Weather Service nor are they meant to criticize Sedgwick Co. Emergency Management. The point here is that both did their jobs properly that April evening, but even so, unnecessary costs resulted because of the constraints of the warning system they have to work with.
Wichita is a medium sized city with a 2000 population of 344,284. Consider the public false alarm cost for a city and county the size of Dallas in a similar situation. And, of course, these figures do not include business costs (i.e., shutting down an assembly line to shelter workers) nor do they consider potential secondary costs (i.e., a traffic accident involving one of the health club patrons rushing home to be with children). These costs also do not consider the cost in credibility to the warning system.
Discussion and Conclusions
I reviewed these figures with a Kansas emergency manager. He stated he "had never previously considered the economic costs" of false alarms. While he did not state a reason, it is suspected that since it was not possible to differentiate false alarm types until recently, economic loss was considered an inevitable part of the warning process.
Because meteorologists understandably focus on the scientific knowledge required to minimize false alarms, the unnecessary false alarm, due to inadequate warning technology and techniques has not received the attention it deserves. There are systems available that can greatly cut down the geographic size of the areas in which siren activations, reverse 911 calls and other types warnings are distributed.
From all points of view (credibility, economic development, etc.) it is recommended that meteorologists, the media, public officials and business partner to find ways to greatly decrease or even eliminate unnecessary false alarms.
References:
National Weather Service Instruction 10-511, Oct 1, 2002.
Bigler, 1956: "A Note on the Successful Identification and Tracking of a Tornado by Radar", Weatherwise, Vol. 9, pp 198-201.
American Meteorological Society, 2000: Glossary of Meteorology, Second Edition, Boston, Mass.
Pielke, 1999, "Floyd, the Fire Drill." Weatherzine. Number 18, October 1999.
Stewart, 2000, "Uncertainty, Judgment, and Error in Prediction," Prediction: Science, Decision Making and the Future of Nature, Island Press.
|
