QUEST Quantitative Evaluation of Regional Precipitation Forecasts Using Multi-Dimensional Remote Sensing Observations 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Quantitative evaluation of regional precipitation forecasts using multi-dimensional remote sensing observations Partnership Susanne Crewell, Thorsten Reinhardt, University of Cologne (IGM) Jürgen Fischer, Anja Hünerbein, FU Berlin (FUB) George Craig, Martin Hagen, Monika Pfeifer, (DLR) Michael Baldauf, Deutscher Wetterdienst (DWD) Nicole van Lipzig, Ingo Meirold-Mautner, Katholieke Universiteit Leuven (KUL), Belgium (QUEST-B) Contributes to PQP Goals Identification of physical and chemical processes responsible for the deficiencies in quantitative precipitation forecast Determination and use of the potentials of existing and new data and process descriptions to improve quantitative precipitation forecast 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Quantitative evaluation of regional precipitation forecasts using multi-dimensional remote sensing observations (QUEST) satellite Observations MSG ~ 5km; 15min Cloud Mask Cloud top pressure MODIS ~ 1km; 1day Optical thickness Radar IPT / Micro- wave GPS Ceilometer DWD radar composite; 1km; 5min Rain rate RANIE combined radar and gauge analysis Polarimetric radar (DLR) 17 stations; Germany; 1min; ranges up to 4km Cloud base height Cloud cover (<4km) 1D vertical; Lindenberg (and Cabauw) temperature profile humidity profile LWC ~ 147 stations; Germany; 15min IWV Surface: rain gauges 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

QUEST: Strategy Observations Retrieval Forward Operator - multi-frequency radiances - polarimetric radar quantities - ground-based and space-borne observations Retrieval Forward Operator - water vapour - cloud properties - precipitation - SynPolRad (polari. radar) - SynSat (MSG, MODIS) - SynSatMic (AMSU, SSM/I) Optical thickness: Matching model and obser-vations involves retrieval and model operator Schröder et al. [2006] Weather Forecasts - three-dimensional description of the forecasted atmospheric state - focus on Lokal-Modell Kürzestfrist (LMK) 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

QUEST: Approach Long-Term Evaluation Case Studies (ongoing) Identification of systematic model deficits Long-Term Evaluation Lokal-Modell Kürzestfrist test suites GOP duration 2007 benefits of high resolution modelling Conditional verification regionalization diurnal cycle weather situation dep. Cross correlation of different variables "How important is physical consistency?" comparison tools test of hypotheses Case Studies (ongoing) Model Sensitivity Runs Tool development SynPolRad SynSat (-Mic) MSG µ-phys. retrievals verification measures .. Hypothesis formulation "What are the crucial variables/processes to observe and to improve?" Model Improvement (new) cloud microphysics land surface turbulence different LMK settings different initial conditions different models (MM4, Meso-NH, RACMO) comparison tools; test of hypotheses case study selection for process studies 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Case studies versus long-term evaluation Detailed analysis Automated analysis Formulation of hypothesis Low significance High significance Case Studies Sensitivity runs feasible / physical explanation Difficult to identify physical mechanism Long-Term Evaluation Subjectively chosen cases Objective selection of cases Tool development 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Deutscher Wetterdienst‘s Lokal-Modell Kürzestfrist (“COSMO-LMK”) Mesh size: x = 2.8 km direct simulation of deep convection convection parameterization for shallow part only assimilation of radar data by latent heat nudging method timestep T=25 s 421 x 461 x 50 gridpoints, lowest model level in 10 m above surface Centre of domain: 10 °E, 50 °N Forecast time: 21 h, started every 3 h Boundary conditions from Lokal-Modell Europa (“COSMO-LME”) with x = 7 km 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Case study 26-08-2004 (M. Baldauf DWD) Accumulated precipitation over 24 hr Radar LM LMK Beschr. siehe S. 38 Beispiel für explizite Konvektion in LMK (Fall '26.08.2004') 3.9 mm/day 5.2 mm/day 3.4 mm/day mm/day 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

General Observation Period (GOP) Year 2007 Central activity of QUEST in second phase of PP. http://gop.meteo.uni-koeln.de 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

GOP Organization and Performance The General Observation Period ─ January to December 2007 ─ encompasses COPS in time and space gather as many data about the atmospheric state as possible within an area covering Germany and it neighboring states. to provide information of all kinds of precipitation types to identify systematic model deficits to select case studies for specific problems to relate the COPS results to a broader perspective (longer time series and larger spatial domain) very different orography within Germany MRR: GOP Highlight Not only COPS and East Germany but also North Sea or Alpine region http://gop.meteo.uni-koeln.de 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

General Observation Period 2007 GOP General Observation Period 2007 Partnership Karl Bumke (IFM-Geomar) Disdrometer observations (WP 3) Susanne Crewell (IGM) Overall GOP organisation Galina Dick (GFZ) GPS observations (WP 5) Jürgen Fischer (FUB) Satellite observations (WP 7) Martin Hagen (DLR) GOP weather radar data (WP 2) Thomas Hauf (UHan) Lightning networks (WP 6) Christian Koziar (Thomas Hanisch) (DWD) Access to DWD observations (all WPs) Armin Mathes (Univ. Bonn) Coordination/QC rain gauges (WP 1) Mario Mech (IGM) GOP data management Gerhard Peters (UHH) Micro Rain Radar (WP 3) Matthias Wiegner (LMU) EARLINET Observations (WP 4) and many more + DKRZ (Claudia Wunram, Hannes Thiermann) + COPS evaluating mesoscale model forecasts of water cycle variables combination of detailed case study investigations and long-term model evaluations systematic model deficits by averaging out stochastic errors (initial and/or boundary conditions) changing model physics in order to attribute the errors to the treatment of specific processes remote sensing data currently not used in routine model verification radar/satellite observations with resolution comparable to Lokal-Modell Kürzestfrist (LMK, ~ 2.8 km) polarimetric radar, millimetre wave radiometry to investigate different hydrometeor species life cycle of clouds and precipitating cells from model and reality with MSG 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

GOP Ingredients: Precipitation Drop Size Distribution Rain gauge Weather Radar WP-GOP-1 Rain gauges; DWD precip analyses (RANIE, REGNIE) WP-GOP-2 Weather Radar WP-GOP-3 Drop Size Distribution DSD several hundred independent observations by DWD, water authorities, environmental agencies etc DWD analyses: RANIE, REGNIE BALTEX one of the continental scale experiments of GEWEX, KNMI DWD radar network and research radars, 3D volume scans, PI, RY, QY, RADOLAN vertical structure at about 15 locations with Micro Rain Radar (MRR) Continuous precipitation observation with high temporal resolution 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

GOP-3 Gerhard Peters 5.-9. März 2007, Bad Herrenalb Lindenberg Zingst Helgoland Lichtenau Wien IMK MPI_4/IG4 DWD_2 UKO DLR MPI_1/IG2 DWD_1 MPI_2 MPI_3/IG1 UHH_1/IG3 MPI UHH_2 UBO_1 UBO_2 LAMP GOP-3 Micro Rain Radar MRR-2 Optical Distrometer ODM470_1 Optical Distrometer FD12P Optical Distrometer PARSIVEL Distrometer JOSS/WALDVOGEL Scanning X-Band Radar (LAWR) DLR Inst. Phys. Atmos., Oberpfaffenhofen DWD_1 R. Assmann Obs., Lindenberg DWD_2 Met.. Obs. Hohenpeissenberg IG_1-4 IfM Geomar, Kiel IMK Inst. Met. Klim., Karlsruhe LAMP Laboratoire de Météorologie Physique MPI_1-4 MPI Hamburg UBO_1-2 Uni Bonn UHH_1-2 Uni Hamburg UKÖ Uni Köln Wien Uni Wien 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln Gerhard Peters

GOP Ingredients: Auxillary Information AMF GPS Lidar WP-GOP-4 Lidar (aerosol, cloud base, mixing layer height) WP-GOP-5 GPS water vapour column WP-GOP-6 Lightning networks WP-GOP-7 Satellite observations (cloud properties, water vapor, aerosol) WP-GOP-8 Meteorological stations EARLINET stations (4), about 100 lidar ceilometer stations in Germany DWD: ca 147 stations in LMK area, ca 200 in LME area + GPS COPS + Switzerland BALTEX one of the continental scale experiments of GEWEX, KNMI European and national networks VLF and VHF MSG, MODIS, MERIS, AMSU, CLOUDSAT, CALIPSO ARM Mobile Facility (AMF), Lindenberg, diverse universities and research institutes 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

GOP-7: Satellites MSG: - cloud mask cloud top pressure (+temperature?), optical depth IR brightness temperature MODIS: - cloud mask - cloud optical thickness τ - liquid water path LWP - effective radius reff - geometric cloud thickness H - IWV - aerosol? MERIS: - cloud mask cloud optical thickness τ cloud top pressure (+temperature?) different LMK settings different initial conditions different models (MM4, Meso-NH, RACMO) comparison tools; test of hypotheses 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Evaluation Areas LMK domain Northsea Baltic Sea Alps North western German lowland North eastern German lowland Low mountain ranges COPS area Countries (D, B, A, CH, NL, F) River catchments (in Germany) LMK domain 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

GOP - First order model evaluation Diurnal comparisons / plots, processed near real-time (“Quicklooks”) Radiosoundings: Plots for each sounding in Germany and neigbouring countries - Stüve diagramm together with corresponding +12h LMK forecast - differences of temperature, specific humidity and wind speed forecasts (+0,+3,+6,+9,+12,+15,+18,+21 h) at each model level GPS, Ceilometer: Daily colour coded maps of BIAS/RMSE of cloud base height (ceilometer) and IWV (GPS); LMK vs. observation Monthly comparisons / plots. Radiosoundings: Bias and RMSE profiles for temperature, humidity and wind for all stations Ceilometer / GPS: Monthly time series of Bias/RMSE for each station or region (depending on number of stations within regions) Ceilometer / GPS: Monthly analysis of mean diurnal cycle and comparison to differnet model runs (lagged ensemble) different LMK settings different initial conditions different models (MM4, Meso-NH, RACMO) comparison tools; test of hypotheses 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Example for GPS Quicklook 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Example for Radiosonding Quicklooks 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Task: Archiving model output Total LMK output too large for permanent storage! => Therefore: Extracting model output relevant for model evaluation : 3 types of data extraction: 1.) statistics over (sub-) areas, timeseries at stations (1-d in time) 2.) column output at individual gridpoints (2-d in height and time) 3.) field output (3-d in x,y,t) 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

1d output (time series) -- statistics of individual quantities (precip, wind, …) in (sub-)domains for direct evaluation & classification für weather-type dependent evaluation -- time series of near-surface variables Werte at Synop stations -- time series of integrated water vapour (IWV) at GPS stations -- time series of cloud base height at ceilometer stations -- time series of precipitation at precipitation stations 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

2d output (column output) All available variables at certain gridpoints with vertically sounding instruments: -- Radiosonding stations -- Micro rain radars (ca 15) -- ARM Mobile Facility -- Earlinet stations -- Cloudnet stations -- COPS Supersites -- Meteorological Observatories 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Field Output for comparison with area covering instruments (radar, satellite) -- brightness temperature of synthetic MSG channels -- radar reflectivity in 850 hPa, max. radar reflectivity in column -- Integrated condensate (TQC,TQR,TQS,TQG) -- height of cloud top and cloud base -- cloud cover (CLCT, CLCL, CLCM, CLCH) -- optical thickness -- precipitation (R, S, G), rates and sums -- radiation balances -- CAPE -- HZERO; 850-hPA temperature, wind; 500-hPa geopotential -- albedo, ground temperature For AMSU: all prognostic variables at overpass times 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Examples of LTE (I): cloud parameters Cloud base Ceilometer Cloud top MSG Cloud cover MSG 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Example of LTE (II): cloud cover Meteosat Second Generation comparison: July 2004 Cloud cover (%) LMK00 / LMK12 BIAS (%) STD (%) Korrelation LMK total 8 / 5 9 / 9 0.80 / 0.80 North Sea 9 / 8 17 / 17 0.72 / 0.70 Alpes 6 / 2 14 / 15 0.78 / 0.81 Lowlands 9 / 7 0.68 / 0.70 Low mountains 7 / 5 15 / 16 0.68 / 0.67 Poldirad area 5 / 2 0.72 / 0.75 COPS area 4 / 0 22 / 20 0.49 / 0.61 Lindenberg Cabauw AMF run 00UTC run 12UTC different LMK settings different initial conditions different models (MM4, Meso-NH, RACMO) comparison tools; test of hypotheses 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Example of LTE (III): cloud cover MSG data – Cloud top pressure Daily cycle Daily cycle of RSME OBS model (run started at 00UTC) Overestimation of cloud top height by model Model simulates realistically no great variation throughout a day. 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Summary LMK LTE Case studies to look into more detail in the problems Boundary layer too thin and too wet IWV generally well predicted IWV Bias of -0.85 kg/m2 for runs started at 12 UTC Clouds too thick Cloud cover in good agreement with MSG Precipitation underestimated by 20% (problem addressed in the maintime by several model changes) Daily cycle not well forecast Case studies to look into more detail in the problems 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Case study example: pol.radar Reflectivity By Monika Pfeifer, DLR LMK, 2 comp. Rain, snow Observation LMK, Thompson, Rain, snow, graupel LMK, 3 comp. Rain, snow, graupel 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Hydrometeor Classification Case study example: pol.radar Hydrometeor Classification By Monika Pfeifer, DLR LMK : 2 comp. Rain, snow Observation LMK: Thompson Rain, snow, graupel LMK :3 comp. Rain, snow, graupel 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln

Evaluate model using long-term observations collected during the GOP Summary QUEST Goals Optimization and refinement of existing evaluation tools Identification of systematic errors in precipitation and cloud fields forecasts Exploitation of the complementary information of the different remote sensing observations; model consistency; cross-correlation of model performance for different variables Evaluate model using long-term observations collected during the GOP Provide an independent test bed for model improvements Improve LMK performance by changes in the treatment of cloud microphysics, turbulence, land surface,… (motivated by results of model evaluation) 5.-9. März 2007, Bad Herrenalb Thorsten Reinhardt, Institut für Geophysik und Meteorologie, Universität zu Köln