Making the most of the forecast

By Andrew Charlton-Perez

Although it’s something we all often take for granted, our ability routinely and skilfully to predict the weather several days in advance is fantastic evidence of the advances in meteorology over the last 50 years. Measures of the skill of operational forecasts since the 1980s like those prepared by the European Centre for Medium-Range Weather Forecasts show a gradual increase in skill,with a modern seven-day forecasts as skilful as a five day forecast was in the mid-1990s.

Another of the major changes in weather forecasting over this time has been the development of ensemble forecasts, which allow us to evaluate the uncertainty inherent in our predictions. Ensemble forecasts are now available on a range of time and spatial scales from hours to seasons ahead. One challenge with the production and dissemination of ensemble forecast information is how best to provide end-users with uncertainty information. One method, as used in the National Severe Weather Warnings service developed by the Met Office, uses a matrix with likelihood and impact axes to assign colour-coded weather warnings for particular weather hazards.

There is, however, a great deal of benefit that can be gained from having access to uncertainty information from forecasts in less extreme weather situations. As part of the NERC funded RACER project and with support from the University Undergraduate Research Opportunities Programme (UROP) I’ve been working with colleagues in the Dept. of Psychology and Dept. of Typography and Graphic Communication to try to understand how well members of the public understand weather forecast uncertainty.

An early part of this project involved conducting a survey of 278 people at various locations around Reading. We focussed on trying to understand how people interpret Probability of Precipitation (PoP) forecasts, since this aspect of forecasts has been studied before with groups of the general public in other countries. We started by asking people how they interpret the statement “There is a 30% chance of rain tomorrow”. In common with the studies of Gigerenzer, Morss and Peachey, only around 30% of the people in our study correctly stated that this is intended to indicate that it will rain on 30% of days like tomorrow, with significant numbers of people either misinterpreting the statement or providing an alternative explanation.

Figure 1. Interpretation of the statement ‘There is a 30% chance of rain tomorrow’ by respondents. Possible answers were ‘It will rain in 30% of the region’; ‘It will rain for 30% of the time’, ‘It will rain on 30% of days like tomorrow’; ‘I don’t know’ and ‘Other’. Responses are expressed as percentage of total number of answers (sample size: 271).

We also explored another part of the precipitation forecast that can also lead to misinterpretation, the use of verbal descriptors (‘Light’, ‘Moderate’, ‘Heavy’) for the intensity of forecast precipitation. Since these words are subjective, we also conducted a second survey to compare their interpretation by members of the public with seven professional meteorologists in Reading. We asked both groups to provide two different interpretations of the verbal descriptors, the rain rate in mm hr^-1 and the time they would estimate puddles to form in minutes.

Figure 2. Mean estimates of rainfall intensity (mm h^-1) and time (minutes) for puddles to form for three rainfall descriptors for general public (light grey) and expert (dark grey) groups. (95% confidence interval for each mean estimate is shown by the error bars). Sample sizes, general public group for rainfall amount in mm (Light=218, Moderate=216, Heavy=217) and time for puddles to form (Light=241, Moderate=250, Heavy=250).

As you can see from Figure 2, experts and the public interpret the descriptors differently, with the public estimating much higher rainfall rates than experts but a longer time for puddles to form (at least for the ‘Moderate’ and ‘Heavy’ categories). The estimates given by experts were consistent with Met Office definitions of rainfall descriptors (where, for example, the ‘moderate’ rainfall rate for precipitation other than showers is between 0.5 and 4 mm hr^-1).

Our focus as we work on this topic over the next two years will be on developing improved communication methods for weather forecasts and other natural hazards that overcome public misinterpretation of both the forecast probability and its severity (or the likelihood and impact axes of the National Severe Weather Warnings matrix discussed above). This is an exciting time to be working on the communication of weather forecasts, not only because of the growth of ensemble forecasting but also because we are in the midst of a fundamental change in the way people consume weather forecasts. One of the other questions we asked people was to tell us their preferred source for weather forecast information with the results shown in the table below.

Table 1. Preferred source of weather forecast information for respondents (sample size: 265) split into respondents whose stated age is 40 or below and whose stated age is above 40. Raw number of responses is shown in brackets. A Chi-Squared test shows that the difference in the distribution of source preference for the two age groups is highly significant (df = 3, N = 265, X²= 57.846, p = 0.00).

As you can see from the table, there is a big difference in the way people under 40 and those over 40 consume weather forecasts, with 58% of people under 40 using a mobile phone as their first choice in comparison to only 19% of people over 40. The kind of hyper-local, frequently updated forecasts typical of mobile phone apps are already the predominant way that many people access our science. This poses both fantastic opportunities and challenges for the way we deliver forecasts and how to provide users with the kind of bespoke information now common in many other fields.

Does this mean the end of the TV or radio weather forecast? A final interesting piece of information we gathered was that of those who expressed a first preference for weather forecasts delivered by mobile phone, more than 50% also use TV forecasts at least twice per week. Perhaps current consumption of weather forecasts might be better characterised as a nationwide, 2-3 day outlook delivered by TV augmented by mobile phone app for local details?

If you would like more details of this work, it will be published in Meteorological Applications (link: http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1469-8080) or you can contact me for a pre-print (twitter:@CharltonPerez)