Comparing the performance of medium-range forecast models
B. Geerts and E. Linacre
Davis (1) verified the prognoses provided by four different agencies for 24
hours hence, 48h, 72h, 96h and 120h (i.e. five days). He compared the sea level
pressure and 500 hPa patterns in the prognoses to the analyses (i.e. the
observations). This was repeated for three consecutive days during a cold
outbreak over Australia during 11 - 13th April 1994. In every case the
comparison was in terms of nine criteria important to operational forecasters.
Conclusions which can be drawn from Appendix 1 in the original paper are these
- On the whole, the best 24h,
48h and 72h prognostics over those three days came from the European Centre for Medium Range Forecasting
(average mark 7.3), followed closely by the Medium Range Forecast
model of the US National Weather Service (7.2), and then the Australian Global
Assimilation and Prediction System (7.1), with the output of the UK Meteorological Office the least
satisfactory on this occasion (5.7).
- With all four medium-range
forecast models, the accuracy of the prognoses was greatest on the first
and least on the fifth day. With rare exceptions, the accuracy of
prognosis declines with the longer lead-times, as would be expected. For
instance, the average mark for the ECMWF prognoses during the comparison
period was 8.0 at 24h, 7.3 at 48h, 6.4 at 72h, 5.7
at 96h, and 4.0 at 120h.
- The accuracy does not
decrease linearly with time. Compared to a linear decay, the 48h, 72h, and
96h forecasts generally are better-than-expected, and the 120h one
worse-than-expected. In other words, the quality of the forecasts decays
rapidly after 4 days.
- The day-to-day variability of
the accuracy of the forecasts is high. Presumably, the models are poorly
initialised on some days, either because of inadequate data quality and/or
quality, or else because an intense storm is poorly sampled.
(1) Davis, C.J., 1997: An analysis of the Global Computer Models during the
cold outbreak situation of 11-13 April 1994 over southeast Australia. Meteor.
Note 211 (Australian Bureau of Meteorology), 23pp.