Evaluating convective-permitting models over South Africa

By Will Keat

Several operational forecasting centres around the world now run convective-permitting models (CPMs) to forecast rainfall. These kilometre-scale models are sufficiently high  resolution to allow convection to be resolved explicitly (i.e. without the need for parameterisation), and have been shown to lead to improved forecasting skill for convective weather events (Clark et al 2016). This is important for helping to mitigate damaging severe convective weather impacts, such as flooding. A key challenge, however, is developing diagnostic tools to evaluate these models that are appropriate and reliable globally. Since 2016, the South African Weather Service (SAWS) has routinely run convective-scale simulations at 4 and 1.5 km resolutions (named SA4 and SA1.5 respectively hereafter) to assist with forecast operations across southern Africa, allowing the applicability of a variety of diagnostics in challenging convective weather situations to be tested. The domains of these models are shown in Figure 1.

Figure 1: Domains for the SA4 and SA1.5 models. The black dots show radars in the eastern half of South Africa; the grey box is approximately the region used in the analysis shown in Figure 2.

The high temporal and spatial resolution of weather radars makes them an ideal tool for studying convection. In 2010, SAWS invested in a large upgrade to their weather radar network, which provides three-dimensional measurements of backscatter (approximately related to the number and size of hydrometeors) every six minutes over thousands of square kilometres. These radar data provide a unique opportunity for model evaluation on the African continent.

How well do CPMs predict the diurnal cycle over South Africa?
One of the key challenges for CPMs is reproducing accurate timing of convection initiation and the diurnal cycle. One diagnostic used in this project to examine this representation is the fraction of data above a fixed radar reflectivity factor.  Figure 2 shows the diurnal cycle of the cloud-and-precipitation fraction above a 10 dBZ threshold for the radar (top row), SA1.5 (middle row) and SA4 model (bottom row) for 12 (left) and 25 (right) November 2016 during significant convective outbreaks over the Highveld region of South Africa (grey box in Figure 1). The SA4 and SA1.5 models generally compare well in terms of diurnal cycle of cloud profile, reproducing deeper clouds on days when deeper convection is observed with the radars, but some key differences can be observed. The SA1.5 tends to peak 1–2 hours too early, whereas the SA4 tends to peak on time or slightly too late. It appears that the models underestimate the height reached by the various cloud-fraction contours. In terms of cloud amount, generally the SA4 and SA1.5 compare well to each other for individual case days, but whether the models over or underestimate the observed cloud amount is case-dependent.

Figure 2: Diurnal cycle of cloud-and-precipitation fraction for 12 November 2016 (left) and 25 November 2016 (right) using a 10 dBZ threshold, for the SAWS radar observations (top row), SA1.5 (middle row), and SA4 (bottom row).

What next?
Similar behaviour of CPMs (such as incorrect initiation timing and too shallow storms) has been observed in other parts of the world, such as over southern UK (Lean et al., 2008, Stein et al., 2015). Understanding why we observe these differences is essential for continual CPM development. Current work is investigating the realism of vertical atmospheric temperature profiles in the models, since these determine atmospheric stability and are therefore fundamental to convection initiation and evolution. Storm tracking, similar to that used in the DYMECS project over the southern UK (Stein et al., 2015), is also being implemented. It is hoped that analysing differences between storm lifecycles in the radar and models will help to identify reasons for incorrect model behaviour.


Clark, P., Roberts, N. , Lean, H. , Ballard, S. P. and Charlton‐Perez, C., 2016. Convection‐permitting models: a step‐change in rainfall forecasting. Met. Apps, 23: 165-181. doi:10.1002/met.1538

Lean, H.W., P.A. Clark, M. Dixon, N.M. Roberts, A. Fitch, R. Forbes, and C. Halliwell, 2008: Characteristics of High-Resolution Versions of the Met Office Unified Model for Forecasting Convection over the United Kingdom. Mon. Wea. Rev., 136, 3408–3424, https://doi.org/10.1175/2008MWR2332.1 

Stein, T.H., R.J. Hogan, P.A. Clark, C.E. Halliwell, K.E. Hanley, H.W. Lean, J.C. Nicol, and R.S. Plant, 2015: The DYMECS Project: A Statistical Approach for the Evaluation of Convective Storms in High-Resolution NWP Models. Bull. Amer. Meteor. Soc., 96, 939–951, https://doi.org/10.1175/BAMS-D-13-00279.1 

This entry was posted in Climate. Bookmark the permalink.