Satellite data used to provide life-saving weather forecasts in tropical Africa

Posted on 22 March, 2021 by danaallen

By: Peter Hill

Much of the population of tropical Africa are vulnerable to severe weather, often caused by intense storms that can generate heavy rainfall, strong winds and flooding. For instance, thousands of fishermen drown each year in Lake Victoria as a result of accidents caused by storms. As a result, improved weather forecasting systems in tropical Africa could save lives and protect livelihoods.

The Global Challenges Research Fund (GCRF) African Science for Weather Information and Forecasting Techniques (SWIFT) project aims to enable African weather forecasting services to develop such improved weather forecasting systems. A partnership between meteorologists from Senegal, Ghana, Nigeria, Kenya and the UK, including several scientists at the University of Reading, SWIFT is striving to improve forecasts from timescales of a few hours to a few weeks ahead.

Much of my work in the SWIFT project involves very short-range predictions – from 0 to 12 hours ahead – based directly on observations, something meteorologists term “nowcasting”. The simplest nowcasts take weather observations and extrapolate them forwards in time, using the assumption that the weather will continue to develop along the same trajectory as the recent past. Nowcasts can be crucial for severe weather events, providing timely information to enable authorities and the public to respond appropriately to safeguard lives and livelihoods.

One of the major obstacles to nowcasting in tropical Africa is the lack of rainfall radar observations, which are used for nowcasting in other parts of the world, including the UK. Passive satellite observations, which measure the naturally occurring energy at the top of the atmosphere, provide a less direct measure of weather systems. Yet in the absence of other observations, this satellite data can provide vital information for nowcasting purposes.

To this end, the SWIFT project has made satellite-based nowcasts for tropical Africa freely available from a new website. Figure 1 provides examples of two such products. These nowcasts are based on software provided by the European Nowcasting Satellite Applications Facility (NWCSAF). However, these products have been calibrated and validated for mid-latitude European weather systems and it is therefore necessary to evaluate how well they perform for tropical Africa.

Figure 1: Examples of two NWCSAF products over tropical Africa. (a) shows the convective rainfall rate in different regions (b) shows the rapidly developing thunderstorms convection-warning product over the Guinea Coast region.

To understand the suitability of this NWCSAF software for tropical Africa, I compared the two products shown in Figure 1 to higher quality satellite rainfall estimates that incorporate data from multiple sources including direct rainfall estimates from rain gauges at the surface. This higher quality data cannot be used for nowcasting because it is not available sufficiently quickly.

The comparison demonstrates that both NWCSAF products provide useful information, despite some limitations. For instance, the convective rain rate product has valuable skill for predictions at least 90 minutes ahead (Figure 2). The rapidly developing thunderstorms product can also identify the occurrence of heavy precipitation, correctly identifying around 60% of strong (5 mm of rain per hour) events at least one hour before they occur. These products could be used to inform flood warnings, disaster response, or provide warnings to fishermen.

Figure 2: Skill metrics for the convective rainfall rate product, compared to predictions based on the historical occurrence of rainfall events. Hit rate means the proportion of true rainfall events that are successfully identified, and false alarm ratio means the proportion of the predicted rainfall events that do not occur in reality. “Retrieval” here is the skill of the satellite products, versus higher quality data. This higher quality data is regarded as “truth” but is not available sufficiently quickly to be useful for nowcasts. The “extrapolation” is the forecast made by projecting the observed storms forward in time. The “climatology” skill is skill from assuming today’s storms can be predicted using previous years storms on the same time of day and time of year.

This analysis is crucial in providing forecasters with confidence in the products which GCRF African SWIFT has made available to them to issue warnings. It has also highlighted some aspects of both the convective rain rate and rapidly developing thunderstorm – convection warning products that could be improved upon. Future work will aim to further develop these products to provide better nowcasts for tropical Africa.

Ongoing work within the African SWIFT project is also enabling African groups to generate these products locally, as well as supporting forecasters to understand and use these products effectively to minimise adverse impacts of severe weather on lives and livelihoods in Africa.

References:

Roberts, A.J., Fletcher, J.K., Groves, J., Marsham, J.H., Parker, D.J., Blyth, A.M., Adefisan, E.A., Ajayi, V.O., Barrette, R., de Coning, E., Dione, C., Diop, A.-L., Foamouhoue, A.K., Gijben, M., Hill, P.G., Lawal, K.A., Mutemi, J., Padi, M., Popoola, T.I., Rípodas, P., Stein, T.H.M., Woodhams, B.J. 2021. Nowcasting for Africa: advances, potential and value. Weather (In press).

Hill, P. G., Stein, T. H. M., Roberts, A. J., Fletcher, J. K., Marsham, J. H. & Groves, J. 2020. How skilful are Nowcasting Satellite Applications Facility products for tropical Africa? Meteorological Applications, 27(6). DOI: https://doi.org/10.1002/met.1966

Posted in Climate, Remote sensing | Tagged , | Leave a comment

Flood forecasting for the Negro River in the Amazon Basin

Posted on 15 March, 2021 by danaallen

By: Amulya Chevuturi

Figure 1: Photograph of the Negro River and the Amazon rainforest.

The Amazon is the largest river basin in the world, with large free-flowing rivers, draining about one-sixth of global freshwater to the ocean. The Amazonian floodplains have been long settled and used by indigenous populations, providing essential ecosystem services and natural resources for human needs (Junk et al., 2014). Increasing frequency and magnitude of floods in the last two decades has caused considerable environmental and socio-economic losses in many regions of the Amazon basin (Marengo and Espinoza, 2016). Although some studies have estimated flood risk for the Amazon basin (de Andrade et al., 2017), most towns and cities in this region still lack operational flood forecasts and integrated flood risk management plans.

The main aim of the PEACFLOW (Predicting the Evolution of the Amazon Catchment to Forecast the Level Of Water) project was to develop skilful forecasting systems for high water levels of Amazonian rivers, at sufficiently long lead time, for effective implementation of disaster risk management actions. In this project, we focused on developing forecast models for annual maximum water level for the Negro River at Manaus, Brazil (Figure 1), as a pilot case study, using a multiple linear regression approach. We used various potential predictors from preceding months: rainfall, water level, Pacific and Atlantic Ocean conditions and linear trend, all of which strongly influence the water levels in the Amazon basin. Flood levels in the Negro River occur between May and July and are strongly influenced by the rainfall during November to February, as its large floodplains delay the flood wave by months (Schöngart and Junk, 2007). This delay and the regularity between the rainfall and peak water level allows for the development of skilful statistical forecast models that can issue forecasts by March or earlier.

Figure 2: The Negro, Solimões and Madeira Rivers (blue lines) and their catchment basins (regions bounded by black lines) contributing to the river water level at Manaus (yellow circle; 3.14°S, 60.03°W).

In collaboration with Brazilian scientists, from various partner institutes, our team developed forecast models of the annual maximum water level (flood level) for the Negro River at Manaus by finding the best model fit over the training period of 1903 to 2004. For our models, rainfall over the catchment of the Negro River as well as from the catchment of nearby Solimões and Madeira Rivers (Figure 2) is the predominant predictor. We developed three models in this project, which use observations as input and can be implemented operationally to provide flood forecasts for Manaus. We compared the models developed in this project against current operational forecasts, provided by Brazilian agencies (CPRM and INPA), for the period of 2005 to 2019. The three PEACFLOW models issue forecasts of flood levels in the middle of January, February and March each year, but the skill of the models increase with decreasing lead-time (Figure 3a). Our results show that the models developed in this study can provide forecasts with the same skill as existing operational models one month in advance.

We also gained an additional month of lead-time when we relaced the observed input data with the ECMWF seasonal ensemble forecast. We developed two operational models using this data, which provide probabilistic forecasts at the beginning of January and February (Figure 3b). The probabilistic forecasts for the maximum water level, using ECMWF input, show good skill for extreme flood likelihood.

Figure 3: Comparison of models developed in PEACFLOW project and existing models (CPRM and INPA) and observed values for annual maximum water level at Manaus for models using (a) observations and (b) seasonal forecasts as input.

The methods developed in this project can also be used to develop forecast models for flood and drought levels over other regions of the Amazon basin. We provide the fully automated PEACFLOW models in a GitHub repository at https://github.com/achevuturi/PEACFLOW_Manaus-flood-forecasting. We retrospectively forecasted the annual maximum water levels at Manaus for 2020, and we are actively forecasting for 2021 (Table 1). Our forecasts this year show the maximum water levels crossing 29m, which is the extreme flood threshold for Manaus, at which the government declares emergency conditions.

Table 1: Forecasts for 2020 and 2021 using PEACFLOW models at different lead-times. Observed annual maximum water level at Manaus for 2020 was 28.52m.

 

References:

de Andrade MMN et al. (2017) Flood risk mapping in the Amazon. Flood Risk Management, 41. DOI:  https://doi.org/10.5772/intechopen.68912

Junk WJ et al. (2014) Brazilian wetlands: their definition, delineation, and classification for research, sustainable management, and protection. Aquatic Conservation: marine and freshwater ecosystems, 24, 5–22. DOI: https://doi.org/10.1002/aqc.2386

Marengo JA and Espinoza JC (2016) Extreme seasonal droughts and floods in Amazonia: causes, trends and impacts. International Journal of Climatology, 36, 1033–1050. DOI: https://doi.org/10.1002/joc.4420

Schöngart J and Junk WJ (2007) Forecasting the flood-pulse in Central Amazonia by ENSO-indices. Journal of Hydrology, 335(1),124–132. DOI: https://doi.org/10.1016/j.jhydrol.2006.11.005

Posted in Amazon, Climate, Flooding | Leave a comment

Can We Use Artificial Intelligence To Improve Numerical Models Of The Climate?

Posted on 8 March, 2021 by danaallen

By: Alberto Carrassi

Numerical models of the climate are made of many mathematical equations that describe our knowledge of the physical laws governing the atmosphere, the ocean, the sea-ice etc. These equations are solved using computers that “see” the Earth system at discrete points only, for instance at the vortexes of a grid where the physical quantities are defined. The density of the grid defines the model resolution: the denser the grid the higher the resolution and, in principle, the better the match between the simulated and the real climate.

Resolution is inevitably finite and to a large extent constrained by computer power. As a consequence, our numerical climate models do not see what occurs in between grid points and offer only a partial description of the reality. This source of model error is called “subgrid” or “unresolved scale” model error.  Reducing or correcting for this is a major endeavour of our scientific community, and a lot has been achieved in the past decades thanks to increased computational power and the improvement of our understanding of the subgrid processes and on their effects on the resolved scale.

Inspired by the astonishing success of artificial intelligence in so many different areas of science and social life, in our recent study (Brajard et al., 2021) we investigated whether artificial intelligence could also be used to improve current numerical climate models by estimating and correcting for the unresolved scale error. Artificial intelligence, and machine learning, in particular, extracts and emulates behavioural patterns from observed data. Being driven by data alone, the forecasts based on machine learning predict behaviour based on behaviour that has previously been observed. Therefore, the quality and completeness of the data used in the training is extremely important.

To overcome this limitation our approach relies on data assimilation, another key component of the nowadays operational weather or ocean prediction routine. Data assimilation is the process by which data are incorporated into models to get a more accurate description of reality. After many years of research and development, data assimilation now provides a range of methods that handle noisy and sparse data with great efficiency.

In our approach, we combine data assimilation and machine learning in the following way. First, we assimilate the raw (sparse and noisy) data into the physical model. This step outputs a sequence of pictures, like a “movie”, showing the climate over the given observed period, whose accuracy depends on the unresolved scale error in the model. The difference between this movie and the model contains information about the unresolved scale error that we wish to correct. In the machine learning step these differences are used to train a neural network to estimate the model error. At the end of the training, we have a neural network that has been optimised to produce an estimate of the model error given the model state as input. The final step consists of constructing a new, possibly more accurate, hybrid numerical model of the climate, made of the original physical model plus the data-driven model obtained using this method.

Figure 1: Shows the model prediction error as a function of time: the longer the time horizon (time length of the forecast) the larger the error. The dashed black line shows the original physical model. The solid lines refer to hybrid (physical plus data-driven) models based on a complete and perfect dataset (black) or on a different amount (p) of noisy observations. The hybrid models perform much better than the original model.  *MTU – Model Time Unit

The data assimilation-machine learning approach has been tested in idealised models and observational scenarios with very encouraging results. A key advantage of the method is that it relies on data assimilation methods that are already routinely applied in weather and ocean prediction centres: we expect this type of approach to be widely implemented operationally in the future.

References:

Brajard, J., A. Carrassi, M. Bocquet, and L. Bertino, 2021. Combining data assimilation and machine learning to infer unresolved scale parametrization. Philosophical Transactions of the Royal Society A, 379(2194), 20200086. doi: https://doi.org/10.1098/rsta.2020.0086 

Posted in Machine Learning | Tagged | Leave a comment

Putting a 120-Year-Old Barograph To The Test

Posted on 1 March, 2021 by danaallen

By: Kieran Hunt

Cast your mind back to 1900. The World’s Fair. Great Britain has just won 48 medals at the Summer Olympics including a clean sweep in the steeplechase. Queen Victoria’s reign continues through an unprecedented 63rd year. British heroes, the Queen Mother and Douglas Jardine, are being born. At 43 Market Street, Manchester (now the home of a massive Urban Outfitters, apparently), an aging Italian immigrant, Joseph Casartelli, owns a workshop specialising in the construction of measuring instruments. Now, forward 120 years (I’ll spare you the scene-setting this time), and I was delighted to receive one such instrument, a barograph, as a Christmas gift from my convivial father-in-law.

Barographs of this era comprise two basic components. On one hand, there is an aneroid barometer – typically a stack of partially-evacuated alloy cells that expand and contract as the pressure decreases or increases. On the other, a clockwork drum is set to rotate about once per week. The two are connected by a scribing arm holding an ink nib. When operating, the nib rests against a paper chart wrapped around the drum, marking pressure changes with time. (Figures 1-3)

Figure 1: Close-up photos of the Casartelli & Son barograph. Top left: inside the clockwork drum. Top right: The spindle on which the drum sits. Bottom: the drum in place, with the scribing arm and ink bottle visible. The aneroid cells are conveniently sealed inside the oak casing and thus not shown here.

Figure 2: The barograph operational setup, showing the drum with paper affixed, scribing arm, and connection to the aneroid in the base.

Figure 3: A page from Percy Jameson’s “Weather and Weather Instruments”, published by Taylor in 1908, showing an engraving of a similar barograph. He’s also not happy about the “concealed works”.

The Storm

As luck would have it, the arrival of Storm Bella (Figure 4) on Christmas night meant that I could test the barograph immediately. With a coffee to steady the post-Christmas hangover (note: it did not steady my hands), I carefully filled the nib with ink, attached the paper to the drum, and woke the clockwork from its multi-decade slumber. It wasn’t that easy of course, it actually took me two hours to figure out that the clockwork wasn’t working, but increasingly firm shaking (the instructions called for “rotation about the horizontal plane”, make of that what you will) soon set it in motion.

Figure 4: Photo of Storm Bella irritating British residents, in this case the owner of a Rolls Royce. Credit: PavementsForThePeople via BBC.

The Results

Figure 5 shows the barograph trace from just after the initial fall in pressure associated with Storm Bella through its development, and eventual recovery by New Year’s Day. Now, a confession in two parts: Boxing Day was a Saturday and the log papers start on Mondays – not wanting to reset the equipment two days into the experiment, I took the liberty of adjusting the calendar. I also confused 12pm with 12am during the initial setup. Bearing these in mind, I overlaid pressure data from the atmospheric observatory at the University (shown in red in Figure 5).

So, how did it do? Well, there are two major differences compared with the observatory record – the first is an initial offset of about 5 hPa, the second is an overestimate of the minimum pressure: 979 hPa on the barograph compared with 963 hPa at the observatory (a difference of 11 hPa when accounting for the initial offset). I had hoped the initial offset was due to elevation differences, but the observatory is only 20 m higher than my house, accounting for just 2 hPa. The rest was almost certainly due to clumsy alignment, a regrettable by-product of my unsteady hands and a remarkably sensitive scribing arm lever. I suspect a similar alignment problem caused the overestimated pressure minimum – in setting the scribing arm position, too much force between the nib and drum results in friction that prevents the scribing arm from moving freely. If we adjust the observatory data to account for these issues (Figure 6), by shifting it up and squashing it a bit, the barograph does a clearly exceptional job of capturing the hour-to-hour pressure changes, keeping within 1 hPa of the observatory values for the whole week.

Figure 5: The barograph trace from Storm Bella (dark blue). Overlaid is the pressure reading from the automatic sensor at the University Observatory (red). If you look carefully at the beginning of the trace, you’ll see my various attempts to get the clockwork moving.

Figure 6: As Figure 5, but with the observatory data shifted and compressed to take into account various barograph calibration errors.

Conclusion

Calibration issues could probably be overcome with a bit of practice, but I probably wouldn’t recommend using it to land a plane. In the right hands, though, it probably could still be used operationally. Amazing.

References:

Science Museum Group 

Posted in Climate, History of Science, Measurements and instrumentation | Leave a comment

Using deep learning to observe river levels using river cameras

Posted on 15 February, 2021 by danaallen

By: Sarah Dance

In recent times, machine learning is being increasingly used to make sense of digital data. In environmental science, we are only at the beginning of this journey (Blair et al 2021). However, we have already found one useful application, providing us with new observations of river levels.

We have successfully investigated novel deep learning approaches to extract quantitative river level information from CCTV cameras near a river (Vetra-Carvalho et al 2020, Vandaele et al 2020).  These provide a new, inexpensive, source of river-level observations.

Unlike river gauging stations, cameras are used to observe the overall environment instead of directly measuring the water level. The cameras are placed at a distance from the water body to ensure a large field of view, so they have a higher chance of withstanding floods. Many carry back-up batteries so that they can function even if the main power supply is disrupted.

Figure 1: (left) A river camera image. (right) An automated semantic segmentation mask for the same image. Flooded pixels are shown in white and unflooded pixels in black.

Figure 1 shows an example river camera image on the left. On the right we show the results of applying a deep learning technique (automated semantic segmentation using a convolutional neural network).  The deep learning method determines which pixels correspond to flooded areas (white) and unflooded areas (in black). Using this information and some extra information about the heights of the image pixels, we are able to work out the water level from the camera image in an automated way. This method could be used to provide invaluable new source of observations for flood monitoring and forecasting, emergency response and flood risk management.

References

Blair, G.S., Bassett, R., Bastin, L., Beevers, L., Borrajo, M.I., Brown, M., Dance, S.L., Dionescu, A., Edwards, L., Ferrario, M.A. and Fraser, R. et al., 2021: The role of digital technologies in responding to the grand challenges of the natural environment: the Windermere Accord. Patterns, 2(1), 100156. https://doi.org/10.1016/j.patter.2020.100156

Vandaele, R., Dance, S.L. and Ojha, V., 2020: Automated water segmentation and river level detection on camera images using transfer learning. In: 42nd German Conference on Pattern Recognition (DAGM GCPR 2020), 28 Sep – 1 Oct 2020. (In Press)

Vetra-Carvalho, S., Dance, S.L., Mason, D.C., Waller, J.A., Cooper, E.S., Smith, P.J. and Tabeart, J.M., 2020: Collection and extraction of water level information from a digital river camera image dataset, Data in Brief, 33,106338, https://doi.org/10.1016/j.dib.2020.106338.

 

Posted in Climate, Flooding, Machine Learning | Leave a comment

Why do clouds matter when we measure surface temperature from space?

Posted on 8 February, 2021 by danaallen

By: Claire Bulgin

We can use satellites up in space to measure the surface temperature of the Earth over the land and sea.  Satellites have now been making measurements for 40+ years and these data are really helpful for understanding trends in surface temperature as our climate changes.  Measuring surface temperature from space is not without its challenges though, and one of the biggest of these is cloud.

So why do clouds matter?  Basically, they block the view of the Earth’s surface from the satellite.  If we try to measure the surface temperature and there is a cloud in the way, what we really measure is in part the temperature of the cloud.  How much it affects our temperature measurement depends on how transparent it is, and how high up in the atmosphere it is. 

So what do we do?  We really only want to measure the temperature when the sky is clear.  This means that we first screen our data for cloud, and then only use the clear-sky observations.  However, this screening process is not always 100% accurate.  Some clouds are difficult to spot even from space!  Consider cold, white cloud over a cold, bright snow surface as in the example of Figure 1.  This was the winter of 2010 where nearly the whole of the UK was covered in snow in early December.   Some clouds are very difficult to pick out above the snow surface.

 

Figure 1:  Snow and clouds over the UK on 08/12/10 in an image from the MODIS Terra satellite (NASA Earth Observatory, 2010). 

So what do we need to do in those cases where screening is difficult?  We need to understand what impact these clouds could have on our measured surface temperature.  In a recent study, we compared a number of different cloud screening approaches against a cloud screening done manually by an expert.  By looking at the differences between each cloud screening approach and the cloud screening done manually, and how these vary, we can build up a picture of how much getting the cloud screening wrong can introduce uncertainty in our measurement of land surface temperature.

Perhaps not surprisingly, we find that the uncertainty in land surface temperature is higher as the amount of clear-sky in the area we are looking at decreases.  This is shown in Figure 2.   The left hand plot shows that the uncertainty in land surface temperature is on average higher when only 20 % of the sky is cloud-free (2 °C) than when 90 % of the sky is cloud free (0.75 °C). This shows that near cloud edges (where a high fraction of the surface we are looking at is covered by cloud) the uncertainty in our measured surface temperature from cloud screening is higher than in areas with fewer clouds.  The uncertainties are larger at night because cloud screening is more difficult without observations at visible wavelengths.

Figure 2: Left: Uncertainty in measured land surface temperature from clouds as a function of the clear-sky fraction (left).  Right: The number of observations for each clear-sky fraction (Bulgin et al, 2018).

If we choose a consistent percentage of clear-sky pixels from our images, we can also assess how the uncertainty varies as a function of the underlying surface type.  In this study we were able to look at five land surface types: Cropland, evergreen forest, bare-soil, shifting-sand and permanent snow and ice.  We found that for a standardised clear-sky fraction of 74.2 %, uncertainties over snow and ice were largest at 1.95 °C, whilst for cropland they were much smaller, only 0.09  °C.  The other surfaces had uncertainties between these two extremes: 1.2 °C for forest, 0.9 °C for bare soil and 1 °C for shifting sand (Bulgin et al, 2018).

References:

Bulgin, C. E., Merchant, C. J., Ghent, D., Klüser, L., Popp, T., Poulsen, C. and Sogacheva, L. 2018.  Quantifying uncertainty in satellite-retrieved land surface temperature from cloud detection errors. Remote Sensing, 10, 616, doi:10.3390/rs10040616.

NASA Earth Observatory (2010).  Snow in Great Britain and Ireland.  Images courtesy of Jeff Schmaltz, MODIS Rapid Response Team.  Accessed 29/01/21.

Posted in Climate, Clouds, Remote sensing | Leave a comment

Weather forecasts to save species

Posted on 1 February, 2021 by timpoulter

by Vicky Boult

Extreme weather events impact the lives and livelihoods of people all over the world, a story we hear more and more often as climate change increases the frequency and severity of weather events. But as climate change throws more challenges our way, so science rallies to address them.

Scientific advances in weather forecasting now mean many extreme weather events can be reliably anticipated. Forecasts therefore provide an early warning system, alerting individuals, organisations and governments to an approaching hazard. Early warnings allow for early action. For example, early warnings of flooding give at-risk communities time to evacuate, and early warnings of drought give farmers the opportunity to choose drought-tolerant seeds for planting. Early action means people are better prepared, so that when an extreme weather event hits, the impacts are less damaging.

Traditionally, humanitarian organisations would respond after an extreme weather event, launching an international appeal for funds to support relief efforts. However, major humanitarian actors have come to recognise the benefits of acting early, before an extreme weather event occurs. Better preparation through early action reduces the humanitarian burden, saving lives, infrastructure and livelihoods, whilst also sparing limited funds and resources (because impacts are reduced and there are cost-saving efficiencies in acting early).

The International Federation of Red Cross and Red Crescent Societies, the world’s largest humanitarian network, have called this movement Forecast-based Action. Forecast-based Action is recognised as one of the most important ways to minimise the impacts of climate change on human lives.  

Part of my role within the Department of Meteorology involves working with the Red Cross’ National Societies to develop Forecast-based Action protocols for drought across Africa. The aim is to identify forecasts which reliably anticipate drought and can therefore be used to trigger early actions and reduce drought impacts.

Personally, I find the Forecast-based Action movement incredibly exciting. Not only because Forecast-based Action has the potential to save lives and livelihoods, but also because I believe the Forecast-based Action concept can be applied to reduce the impacts of climate change elsewhere.

Before joining the Meteorology department in early 2019, my studies and research focused on ecology and conservation; during my PhD, I studied the influence of food availability on the movement and abundance of African elephants. Now with both my conservation and my meteorology hats on, I realise there is huge potential to adopt Forecast-based Action in conservation.

Just like people around the world, species too are increasingly at-risk of extreme weather. Drought can lead to food shortages for herbivores, cyclones can destroy precious coral reefs, and bush fires can burn through woodland habitats. Despite the obvious implications of extreme weather events for species survival, research in ecology and conservation has focused more on the gradual effects of climate change on species over much longer timeframes, and relatively little research has focused on the impacts of extreme weather. Whilst a long-term view has highlighted the threats posed by climate change for biodiversity, such warnings are not very actionable and, in the meantime, extreme weather events push species closer to extinction.

What might Forecast-based Action look like in conservation? Here’s one example.

Sea turtles lay their eggs on sandy beaches around the world. Nest temperatures below the sand are important for hatching success; if temperatures get too hot, the developing turtles inside the eggs die. So, if a weather forecasts indicates very high temperatures are expected at a turtle nesting beach, conservationists can take early action to prevent losses. One option is to install temporary structures over nests to provide shade and prevent overheating. Alternatively, nests can be excavated, and eggs artificially incubated at safe temperatures. In this way, forecasts can help to improve turtle hatching success and contribute to the conservation of the species.

Image: Forecast-based Action to improve hatching success in sea turtles. Cartoon adapted from International Federation of Red Cross and Red Crescent Societies media.

Realising Forecast-based Action in conservation will require overcoming several challenges, including deciding when to act on forecasts (even the best forecasts can be wrong) and how to fund early action. But these challenges have already been addressed in the humanitarian sector, and conservation should use lessons learnt there for guidance.

As climate change demands increasing intervention from conservationists to prevent species extinction, I hope that advances in weather forecasting developed in meteorology and innovation in practice established in the humanitarian sector, might inspire and guide more anticipatory action in conservation.

Posted in Climate | Leave a comment

What’s that data? Why and how the geoscientific community is forging metadata standards.

Posted on 25 January, 2021 by timpoulter

By Sadie Bartholomew

Part I: Why

Earth science is a data-driven endeavour. Advanced simulations of the Earth system, whose configurations are themselves complex controlled data, churn out petabytes of analysis datasets whilst raw data from in-situ and satellite observations stream in continuously. Both upstream sources can then undertake a myriad of different paths in processing and post-processing workflows, becoming derived products or being ingested for reanalysis or as a background state to further model runs downstream.

Figure 1: Weather and climate research involves an abundance of data covering many different variables. Individual keywords from potential variables are indicated in this word cloud (optimised to fit the shape and colour of an image of the Earth) generated (for this post) using the Python library word cloud. Read on to learn precisely what terms and weightings are represented!

For the most part this deluge, which epitomises the various “V” words defining “big data”, comes in the form of array-based data packaged into files in a format called netCDF. A major advantage of netCDF is that it was designed to be self-describing, that is to say each file should outline its own structure and each included array of data should be accompanied by a standalone description of what it represents — by metadata.

An underappreciated subclass of data, metadata is indispensable wherever there is a significant collection of information; imagine trying to locate a book in your favourite library if there were no titles, author names or shelf labels.

Involving a mixture of data drawn from the real atmosphere, surface and ocean as well as varying virtual equivalents, geoscience is more complex than a hypothetical library in data terms. Robust, wide-ranging metadata is therefore essential to making sense of it all.

Take the spatio-temporal (space and time) context of the data arrays, for example. Each represents a snapshot of a location on the real or simulated Earth, a sample across one or more of the latitude, longitude, vertical and time coordinates. What is the coverage of the data across these dimensions and how do the arrays map onto them?

Often the correspondence is not simple. To begin with, data frequently covers intervals, or cells, of space-time rather than discrete points on it. In these cases we need to describe the extent of the cells, the so-called bounds, plus the way in which the data in question represents each full cell (perhaps some average or extremum), the cell-methods.

Moreover the data may not be representative of space-time in a straightforward way, potentially spanning a further axis called climatological time in the case of climatological statistics, or by use of a simplified calendar in lieu of the ordinary (Gregorian) calendar. We need a comprehensive and consistent way to describe all these details.

Another core challenge for geoscientific metadata is identifying the nature of the data itself. Assigning a sensible name to a given dataset may not be too taxing, but with such a diversity of potential variables stemming from the wide variety of physical, biogeochemical and human processes at play, doing so unambiguously is really quite difficult.

NetCDF does not standardise the names of data variables, so without further regulation naming would be subject to variance of interpretation across the membership of a diverse research community, sometimes but not always abiding by other named or informal standards from sub-domains that can be at odds. That scenario would add a significant barrier to data sharing, comparison and archival, not only on a human level, but for the other workhorses involved: the machines crunching all that data. Lack of metadata standardisation would cause relevant software packages to become cluttered with logic to handle the countless interpretations and inconsistencies. It would quickly become unmanageable.

Part II: How

So what solution is there? How can we ensure that, now and sustainably going forwards, anyone is able to describe the data encoded in netCDF in a unambiguous way, with scope to fulfil the potential requirements of researchers from multiple disciplines as well as the strict parsing needs of machines?

Step in the CF-netCDF metadata conventions, more commonly known as the CF Conventions.

A community-led effort that spun up shortly after the millennium and has been gaining traction ever since, the CF Conventions set out to solve challenges such as those presented above by providing a definitive standard for climate (‘C’) and forecast (‘F’) metadata. The fundamental scope is to cover netCDF file metadata, though it has been acknowledged that they have wider application and could be used with other data formats, for example XML.

The conventions are outlined in a definitive and comprehensive document that is updated (in the next tagged version) when there is general agreement from interested parties in the community that a proposed change should be made. In this way it has matured over numerous years and versions, with the latest version 1.8 released in February 2020 and 1.9 due out in 2021.

Another core aspect of the conventions is a table establishing a controlled list of possibilities for the names and associated units of data variables. This is the CF Standard Name Table and it is developed in a similar way, though is versioned separately and on version 76 as of late 2020. The idea is that it should provide any name that researchers could conceivably need to describe the data they own or use. If someone can’t find an appropriate name within the table, they submit an Issue on GitHub to formally propose it as a new name and if accepted will form a CF-compliant entity that can be used by all.

Figure 1 in fact depicts all of the individual words taken from the full set of standard names in the current table (the larger the text, the more the given keyword appears throughout the names).

An illustrative indicator of increased uptake of the CF Conventions is the total number of standard names published over time, as shown in Figure 2. To begin with there were around 700 names, now there are nearly 4500 and counting!

Figure 2: The increase in the number of standard names proposed, accepted and finally added into the CF Standard Name Table since the beginning of the project, up to and including version 75, with an indication of the progression of the table versions.

A further indicator is the popularity of annual workshops dedicated to the CF Conventions. 2020’s ‘CF Workshop’ was due to be held in Santander but due to COVID-19 restrictions was held virtually. Over a hundred people registered from fifteen countries, with at least half attending the sessions across each of three days to participate in ways ranging from learning about the current status and future plans of the conventions to engaging in breakout sessions that made headway towards resolving some weighty open proposals. Much interesting discussion took place, all of which is summarised openly on the website, and I, for one, am looking forward to the next annual meeting.

Many colleagues across the department have been involved or indeed instrumental in the growth of the CF Conventions, including Jonathan Gregory who was one of the five original authors and David Hassell who is the present secretary of the Conventions Committee.

Above all, though, the CF Conventions are run and managed by the geoscience community that they serve. Everyone is encouraged to have their say and get involved, be it by commenting to register thoughts on open proposals for changes to the canonical document or for new standard names, raising their own proposals, or helping to develop tools that facilitate discussion or indeed encourage compliance.

If you’re interested in contributing, please explore the official CF Conventions website or dive into the discussions now contained in an Issue Tracker under the official GitHub organisation ‘cf-convention’.

Posted in Climate | Tagged | Leave a comment

All I want for Christmas is some PV maps…

Posted on 21 December, 2020 by timpoulter

By: Ben Harvey

For many years, the department webpages have hosted real-time plots of large-scale atmospheric conditions based on operational ECMWF analysis data. Over the last few months we’ve been revamping them with a new webpage and higher-resolution images. I thought I’d describe some of the new plots here, in case you found yourself with time on your hands over the Christmas break!

First up, here’s the new web link: www.met.reading.ac.uk/~ben/current_weather.

The first set of pages (‘Dynamical Tropopause Maps’) show a suite of variables at tropopause level. More precisely, they show variables interpolated to the height where the potential vorticity (PV) reaches a certain value. This is a good proxy for the tropopause because the PV, which combines information about the stratification and rotation of air parcels, tends to be low in the troposphere and high in the stratosphere (see http://rammb.cira.colostate.edu/wmovl/vrl/tutorials/satmanu-eumetsat/satmanu/basic/parameters/pv.htm  for a brief intro to PV).

But why is this useful? Jet streams and synoptic-scale weather systems are intimately linked to the shape of the tropopause. Figure 1 shows an example from last weekend. On Sunday 13 Dec there was a strong but wavy jet stream crossing right across the North Atlantic (left panel) – a common occurrence this time of year. The tropopause was over 10 km high to the south of the jet but only 5 km high to its north, with the height dropping very rapidly across the jet core (right panel). There’s often incredible structure in the tropopause height field; try clicking through the plots on the webpage for the past few days to explore the range of patterns that occur.

Figure 1: Wind speed (left) and geopotential height (right) at the tropopause from 18Z on 13 December 2020. The annotation highlights the position of the jet stream.

To help visualise what’s going on, Figure 2 shows a cross section through the North Atlantic jet stream taken from an aircraft field campaign back in Autumn 2016. You can see how the tropopause height drops across the jet stream and how the maximum wind speeds tend to lie on the tropopause itself. Figure 1 effectively follows the black tropopause line in Figure 2, and therefore tends to pick out the strongest wind speeds at each location.

Figure 2: A North-south cross section through the jet stream (bold contours) also showing potential vorticity (shading; units: PVU), potential temperature (thin contours) and the tropopause (black line). This example is from an NWP model forecast. The dashed line shows the track of a research flight aimed at measuring the structure of the tropopause in the core of the jet. (Adapted form Harvey et al., 2020)

Perhaps the most striking features in Figure 1 are the large north-south meanders of the jet stream. These are Rossby waves. They occur because the rotation of the Earth, combined with its curvature, inhibits the north/south motion of air parcels on large scales. Instead, they tend to curve back to their original latitude. Figure 3 (left panel) shows the north/south component of the wind (again at tropopause level) and the alternating green-purple pattern highlights these Rossby waves and their evolution.

Figure 3: Meridional wind at the tropopause (left) and low-level equivalent potential temperature (right) at 18Z on 13 December 2020. The right panel also shows surface pressure (grey contours) and a couple of contours of potential temperature on the tropopause (blue and black). The annotations highlight the jet position from Figure 1.

The next set of pages (‘Lower-troposphere Maps’) show surface conditions, including sea-level pressure, lower-tropospheric windspeed, and equivalent potential temperature. These can be compared to the tropopause maps to understand how the large-scale tropopause structures relate to the surface weather we experience, and vice versa. In our example there are two extratropical cyclones developing beneath the meanders of the jet stream (Figure 3, right panel). You may recall, the deeper cyclone located just to the west of the UK dumped quite a bit of rainfall over Reading on Sunday night (13 Dec).

Finally, the ‘Isentropic PV Maps’ show the PV itself, interpolated to potential temperature surfaces. To visualise this, follow one of the potential temperature contours in Figure 2 (e.g. the one at 6 km altitude at the left edge of the plot). Moving northwards, the surface starts in the troposphere but rises and crosses the tropopause into the lower stratosphere and much higher values of PV. Isentropic PV maps provide deep insight into the evolution of the atmosphere because, to a good approximation, both PV and potential temperature are conserved following air parcels. This means PV features on these maps are simply advected by the winds on each surface (see Hoskins et al., 1985, for further details). Any changes in PV following the winds can be pinned down to the presence of either diabatic heating (e.g. phase changes of water in clouds and radiative heating/cooling) or to frictional effects. As such, the non-conservation of PV turns out to also be very useful for understanding the impact of these processes on weather systems and climate more generally. Have a click through the plots from the last 2 weeks and see which PV features you can track through time, and which ones are created or destroyed.

These webpages are still in development and will be added to (when time allows!). Please send any suggestions for improvements/additions to b.j.harvey@reading.ac.uk.

Hoskins, B.J., McIntyre, M.E. and Robertson, A.W., 1985. On the use and significance of isentropic potential vorticity maps. Quarterly Journal of the Royal Meteorological Society, 111(470), pp.877-946.

Harvey, B., Methven, J., Sanchez, C. and Schäfler, A., 2020. Diabatic generation of negative potential vorticity and its impact on the North Atlantic jet stream. Quarterly Journal of the Royal Meteorological Society, 146(728), pp.1477-1497.

Posted in Climate | Tagged , | Leave a comment

Here comes the rain again…

Posted on 14 December, 2020 by timpoulter

By: Natalie Harvey

British people are well renowned for their obsession for talking about the weather. This is partly because it is a “safe” topic for conversation and partly because it is really fascinating! This is especially true in the UK where our weather is so varied.

Since my son started school in 2017, I have found myself having numerous conversations about the weather and the accuracy of weather forecasts at drop off and pick-up times. These conversations are often critical of the forecasts, which I do my best to dispel in the short time I have. The UK Met Office four-day forecasts are now as accurate as our one-day forecast was 30 years ago, with over 90% of forecasts of next day temperature accurate to within 2 °C [1]. Bauer et al. (2015) [2] give a detailed description of the revolution in numerical weather prediction over the last 40 years.

Another question I hear a lot is “why does it always rain at pick up time?” Now, I live a fair way from my son’s school, so my walk is longer than most and I don’t really feel like it rains that much. However, last Wednesday, at 3pm, it did! I got very soggy. It made me think about the question again as it is something that we can test using rainfall data from the Atmospheric Observatory based on the University’s Whiteknights campus [3] which is a short walk from my son’s school.

Figure 1 – Screenshot from the Met Office weather forecasting app showing rain forecast at 3pm.

In Figure 2a the bars show the fraction of time it rains for rain aggregated over two-hour windows from the last 5 years (2015-2019). Each two-hour window is split into five-minute chunks and each five-minute chunk is counted if at least 0.2mm of rain is recorded. The black bars indicate all days, teal bars indicate summer days (June, July, August) and light blue bars indicate winter days (December, January, February).

Overall, as I thought, it doesn’t rain much per 2-hour window, with each 5-minute chunk being classified as raining just once a month on average. In general, the fraction of time it rains does increase throughout the day, with the signal strongest in the summer months. This is most likely related to the diurnal cycle of convection over land in the summer which brings heavy rain showers. The effect is not very large though, with rainfall during the afternoon around 20% more common than between 8 and 10am, school drop off time.

Figure 2 – (a) Fraction of time rain occurs throughout the day for 2015-2019 in two-hour windows. Black bars indicate all days, teal bars indicate summer days and light blue bars indicate winter days. (b) Mean rainfall per 5-minute window throughout the day when rain is recorded.

Figure 2b shows the mean rainfall (bars) for the times that it is raining in the same 2-hour windows. The grey bars indicate the 5th and 95th percentile of the observed rain at these times. These are included to give an indication of the variability of rainfall and show the highly skewed distribution of rainfall rates. The mean rainfall is reasonably uniform throughout the day with a higher value between 2 and 4pm in the summer months, again this is likely to be related to convective events. During that time window there are some large rainfall events, so it is possible that my fellow parents at the school gate memories are skewed by a few events when they (and their children) got very soggy or maybe it is the rain affecting their mood [4]. This is only a very small-scale study using a small amount of rainfall observations, but it seems to me that it doesn’t always rain at school pick up time but maybe it is best to have your umbrella ready just in case, especially in the summer.

[1] Met Office, 2020 Global accuracy at a local level. Accessed 4 December 2020, https://www.metoffice.gov.uk/about-us/what/accuracy-and-trust/how-accurate-are-our-public-forecasts

[2] Bauer, P., Thorpe, A. & Brunet, G., 2015: The quiet revolution of numerical weather prediction. Nature 525, 47–55. https://doi.org/10.1038/nature14956

[3]Reading University Atmospheric Observatory, 2018: Atmospheric Observatory. Accessed 4 December 2020, https://research.reading.ac.uk/meteorology/atmospheric-observatory/

[4] Flook,J 2019: Can’t stand the rain? How wet weather affects human behaviour. Accessed 4 December 2020, https://www.theguardian.com/science/blog/2019/mar/06/cant-stand-the-rain-how-wet-weather-affects-human-behaviour

Posted in Climate | Tagged , | Leave a comment