Transcript A summary of OSE and OSSE activities at ECMWF. Erik Andersson, Graeme Kelly, Jean-Noël Thépaut, Gabor Radnoti, Peter Bauer and Sean Healy Acknowledgements: EUCOS, EUMETSAT,

A summary of OSE and OSSE
activities at ECMWF.
Erik Andersson, Graeme Kelly, Jean-Noël Thépaut,
Gabor Radnoti, Peter Bauer and Sean Healy
Acknowledgements:
EUCOS, EUMETSAT, JCSDA/NCEP
 Three major sets of OSEs
 Investigating the complementarity between space
based and terrestrial observing systems
 The impact of MetOP instruments
 Impact of GPS Radio Occultation data
 The Joint-OSSE, followed by Conclusions
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 1
ECMWF
1: The Space-Terrestrial Study
Initiated and funded by EUCOS.
i.
BASELINE: all satellite observations currently used in NWP
(radiances, cloud-drift winds, scatt winds) + GUAN R/S + GSN
surface land data + buoys (no ship data)
ii. BASELINE + aircraft data
iii. BASELINE + non-GUAN R/S wind profiles
iv. BASELINE + non-GUAN R/S wind and temp profiles
v. BASELINE + wind-profiler data
vi. (iv) + aircraft data
vii. BASELINE + non-GUAN R/S wind, temp and humidity profiles
viii. CONTROL: the combined observing system
ix. BASELINE + non-GUAN R/S temperature profiles (winter)
x. BASELINE + aircraft temperature data (winter)
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 2
ECMWF
OSE assimilation system configuration
 Resolutions:
 Model resolution T511 (50 km), L60
 Analysis at T511/T159 L60,
 12-hour 4D-Var
 Winter Experiments:
 20041204-00 to 20050125-12 (including 10 day warm up)
 Model cycle 29R1
 Summer experiments:
 20050715-00 to 20050915-12 (including 10 day warm up)
 Model cycle 29R2
 NOAA18 included (AMSU-A and MHS)
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 3
ECMWF
Winter results: Baseline – Control (Z500)
Impact of terrestrial, non-climate, observations
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 4
ECMWF
Control-Baseline (Z500)
Normalised forecast error difference, Day-3
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 5
ECMWF
Z scores: Impact of R/S
1000 hPa
500 hPa
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 6
ECMWF
Baseline – [ Baseline + aircraft]
Z500 Impact of aircraft data
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 9
ECMWF
[Baseline + aircraft] - Baseline (Z500)
Normalised forecast error difference FC+12h
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 10
ECMWF
Z scores: Impact of aircraft
1000 hPa
500 hPa
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 11
ECMWF
Wind scores: Impact of aircraft
850 hPa
300 hPa
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 12
ECMWF
[(i) + R/S T+W] – [ (i) + R/S T+W + aircraft]
Z500 Impact of aircraft in the presence of R/S
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 13
ECMWF
[(i) + R/S T+W + aircraft] – [(i) + R/S T+W] (Z500)
Z500 Impact of aircraft in the presence of R/S
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 14
ECMWF
[(i) + R/S T+W] – [ (i) + R/S T+W + aircraft]
W300 Impact of aircraft in the presence of R/S
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 15
ECMWF
[(i) + aircraft] – [ (i) + R/S T+W + aircraft]
W300 Impact of R/S in the presence of aircraft
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 16
ECMWF
[(i) + R/S T+W] – [ (i) + R/S T+W+Q]
RH700 Impact of R/S humidity
NH
EUR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 17
ECMWF
EUCOS Space-Terrestrial, Conclusions (1)
 Even in presence of satellite observations, degrading
the current terrestrial Observing System has a
significant negative impact on the forecast skill.
 Starting from the degraded baseline (GUAN+GSN+…):
 Additional R/S (T+Wind) and aircraft (T+Wind)
contribute more or less equally to the Observing
System (slight advantage for R/S)
 These two Observing Systems are complementary
Aircraft add forecast skill to R/S
R/S add forecast skill to Aircraft
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 18
ECMWF
EUCOS Space-Terrestrial, Conclusions (2)
 R/S impact:
 R/S winds contribute little on their own
 Radiosonde T contribute marginally more
 This is the combination of wind/T which really provides
the impact of the RS on the forecast skill
 Aircraft impact:
 The results are consistent with that of R/S. Aircraft-T
alone bring relatively little. Combination of T/Wind
makes a big impact
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 19
ECMWF
EUCOS Space-Terrestrial, Conclusions (3)
 Humidity from R/S add very little in terms of scores
 Noticeable but small impact on relative humidity scores
up to day 3
 Impact of wind profilers:
 Winter impact:
 The short range forecasts are improved by the
US and japanese profilers
 The signal blurs away after day 4-5 and large
scale interactions appear
 European profilers do not bring much
 In summer, the impact is smaller than during the
winter period (in absolute but also relative terms)
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 20
ECMWF
2a: Assessment of the space component of the GOS
Initiated and funded by EUMETSAT
Winter period: 20041204-20050125, Summer period:
20050715-20050915 (cycle 29r1)










BASELINE all conventional observations used in NWP
(radiosonde + aircraft + profiler network + surface land data +
buoy observations + ship data)
REFERENCE= BASELINE + AMVs from GEO+MODIS
REFERENCE + HIRS radiances
REFERENCE + AMSUA radiances
REFERENCE + AMSUB radiances
REFERENCE + SSMI radiances
REFERENCE + GEO Clear Sky Radiances (CSRs)
REFERENCE + AIRS radiances
REFERENCE + SCAT winds
BASELINE + GEO AMVs (no MODIS)
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 21
ECMWF
2b: Assessment of the space component of the GOS
Initiated and funded by EUMETSAT
Winter period: 20061205-20070214 (31r1), Summer
period: 20060601-20060815 (31r2)
i.
ii.
iii.
iv.
v.
vi.
vii.
viii.
ix.
BASELINE all conventional observations used in NWP (radiosonde
+ aircraft + profiler network + surface land data + buoy
observations + ship data)
REFERENCE= BASELINE + AMSUA Noaa 16
REFERENCE + AMVs from GEO+MODIS
REFERENCE + AMSUA radiances
REFERENCE + AMSUB radiances
REFERENCE + GEO Clear Sky Radiances (CSRs)
REFERENCE + AIRS radiances
REFERENCE + SCAT winds
CONTROL full operational system (all above observations)
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 22
ECMWF
Average Z 500 anomaly correlation scores
The AMV reference and the AMSU-A reference
S.Hem
All AMVs ref
S.Hem
One AMSU_A ref
WMO Workshop19-21 May 2008: ECMWF OSEs
N.Hem
All AMVs ref
N.Hem
One AMSUA_ ref
Slide 23
ECMWF
Z500 impacts, S.Hem
(REFERENCE_AMV)
AMSU-B, SSMI,
GEO_CSR
HIRS, AMSU-A,
AIRS
Scat, GEO_AMV,
GEO_CSR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 24
ECMWF
Relative Humidity
850hPa impacts, S.Hem
(REFERENCE_AMV)
AMSU-B, SSMI,
GEO_CSR
HIRS, AMSU-A,
AIRS
Scat, GEO_AMV,
GEO_CSR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 25
ECMWF
Vector wind
200 hPa impacts, S.Hem
(REFERENCE_AMV)
AMSU-B, SSMI,
GEO_CSR
HIRS, AMSU-A,
AIRS
Scat, GEO_AMV,
GEO_CSR
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 26
ECMWF
Impact of removing
MODIS from the
REFERENCE_AMV
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 27
ECMWF
Space component, Summary
 Very encouraging that all the space base sensors contribute in a
positive way to the overall performance of the ECMWF forecast
system. Sensors like AMSU-A, AIRS and HIRS are the most
important.
 The humidity analysis requires AMSUB (also MHS), GEO CSRs
and SSMI.
 Amongst the wind data, SCAT has a clear positive impact on the
surface wind in the Southern Hemisphere, and a clear beneficial
impact of AMVs and MODIS winds has been demonstrated.
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 31
ECMWF
3: OSEs for the evaluation of degraded EPS/PostEPS instrument scenarios (EUMETSAT)
The objective is to assess the detrimental impact of potential loss
of the main EUMETSAT Polar System (EPS) instruments on
global NWP.
 REFERENCE: All conventional data + 2*AMSU-A, 2* DMSP,
AIRS and IASI, ASCAT and QuikSCAT.
 REFERENCE – HIRS
 REFERENCE – AMSU
 REFERENCE – MHS
 REFERENCE – SOUNDERS
 REFERENCE – IASI
 REFERENCE – ASCAT
 REFERENCE – METOP
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 32
ECMWF
METOP forecast impact
500 hPa geopotential height RMS error difference
when METOP data withdrawn
T+12 hours
T+24 hours
[%/100]
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 33
ECMWF
Preliminary results on Metop (ongoing study)
 The fit to temperature-sounding instruments (e.g. NOAA18 AMSU-A) is improved when Metop instruments are
present.
 The fit to moisture-sounding instruments (e.g. AIRS, MHS,
GOES) is improved when Metop MHS, all Metop sounders
or IASI are present
 The presence of ASCAT data slightly improves the fit to
QuikSCAT wind data, and products from Envisat and
Jason altimeters
 The synergy of Metop instruments produces a much
stronger impact than its individual instruments
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 34
ECMWF
GPS radio-occultation. Current 6-hour data coverage.
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 35
ECMWF
December 2006: Assimilation of RO data at ECMWF
All 6 satellites
Wave-like bias disappeared
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 36
ECMWF
GPSRO impact on Southern hemisphere geopotential height forecasts
6 COSMICs+CHAMP+GRACE-A (setting only, z > 5 km) – no GPSRO
1100 observations/day, 66 cases, winter 2006/2007, own analyses
control normalised beuwz minus beuwy
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
1000hPa Geopotential 00UTC
Confidence: 95%
Population: 66
control normalised beuwz minus beuwy
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
500hPa Geopotential 00UTC
Confidence: 95%
Population: 66
1000 hPa
500 hPa
0.15
0.15
0.1
0.1
0.05
0.05
0
0
-0.05
-0.05
0
1
2
3
4
5
6
7
8
9
10
11
0
1
2
3
4
Forecast Day
5
6
7
8
9
10
11
9
10
11
Forecast Day
control normalised beuwz minus beuwy
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
200hPa Geopotential 00UTC
Confidence: 95%
Population: 66
control normalised beuwz minus beuwy
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
100hPa Geopotential 00UTC
Confidence: 95%
Population: 66
200 hPa
100 hPa
0.15
0.15
0.1
0.1
0.05
0.05
0
0
-0.05
(Sean Healy)
-0.05
0
1
2
3
4
5
6
7
8
9
10
11
0
1
2
3
Forecast Day
WMO Workshop19-21 May 2008: ECMWF OSEs
4
5
6
Forecast Day
Slide 37
7
8
ECMWF
GPSRO impact on Southern hemisphere geopotential height forecasts
6 COSMICs+CHAMP+GRACE-A (setting+rising, z > 5 km) – no GPSRO
1600 observations/day, 66 cases, winter 2006/2007, own analyses
control normalised beuwy minus beux1
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
1000hPa Geopotential 00UTC
Confidence: 95%
Population: 66
control normalised beuwy minus beux1
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
500hPa Geopotential 00UTC
Confidence: 95%
Population: 66
1000 hPa
500 hPa
0.15
0.15
0.1
0.1
0.05
0.05
0
0
-0.05
-0.05
0
1
2
3
4
5
6
7
8
9
10
11
0
1
2
3
4
Forecast Day
5
6
7
8
9
10
11
9
10
11
Forecast Day
control normalised beuwy minus beux1
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
200hPa Geopotential 00UTC
Confidence: 95%
Population: 66
control normalised beuwy minus beux1
Root mean square error forecast
S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0
Date: 20061215 00UTC to 20070218 00UTC
100hPa Geopotential 00UTC
Confidence: 95%
Population: 66
200 hPa
100 hPa
0.15
0.15
0.1
0.1
0.05
0.05
0
0
-0.05
(Sean Healy)
-0.05
0
1
2
3
4
5
6
7
8
9
10
11
0
1
2
3
Forecast Day
WMO Workshop19-21 May 2008: ECMWF OSEs
4
5
6
Forecast Day
Slide 38
7
8
ECMWF
Advanced infrared sounders: AIRS and IASI
AIRS
 Operational at ECMWF since October
2003.
 324 channels received in NRT.
 One FOV in nine used.
 Up to 155 channels may be
assimilated (CO2 and H2O bands).
IASI
 Operational at ECMWF since June
2007.
 8461 channels received in NRT.
 All FOVS received; only 1-in-4 used.
 366 Channels routinely monitored.
 Up to 168 channels may be
assimilated (CO2 band only).
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 41
ECMWF
IASI forecast impact
500 hPa geopotential anomaly correlation
(56 cases, spring 2007, normalized RMSE difference, own analysis)
IASI
better
IASI
worse
NH
Mean error difference
uncertainty
IASI
better
IASI
worse
SH
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 42
ECMWF
Choosing 10 IASI Water Vapour Channels
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 43
ECMWF
Normalized Analysis Departure Std. Dev.
Fit to other observations
Best value at 1.5K
WMO Workshop19-21 May 2008: ECMWF OSEs
Increasing assumed IASI
H2O channels’ error
Slide 44
∞
ECMWF
The Joint OSSE Nature Run
 Collaboration with NCEP (Michiko Masutani) to plan, produce,
deliver and evaluate the NR. Consultation with the Joint-OSSE
group in the US, EUCOS, ESA, EUMETSAT and ECMWF
 T511 NR: 13 months T511/L91.
 Data set size ~2.5 Tbyte. Shipping to the US on 4 disks
 Yearly, quarterly and monthly comparison with climate and
observations = 831 plots. Posted on NCEP web site
 Extensive evaluation of the NR by US partners
 T799 NR: Two 6-week periods have been run at T799/L91 with
hourly post processing: TC-season, Convective season
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 45
ECMWF
Total Cloud
Cover eskb
nmon=12 0°
nens=1 Global
135°W
90°W June 2005
45°W
45°E Mean: 62.6
90°E 50N-S Mean:
135°E 57.8
Nature Run,
12-month
total cloud
cover
[percent]
95
97.71
60°N
60°N
65
90
95
30°N
30°N
80
65
0°
80
65
35
65
0°
65
70
65
50
65
30°S
30°S
65
35
60°S
60
20
60°S
50
65
65
135°W
90°W
5.165
45°W
0°
45°E
90°E
135°E
Total Cloud
Cover MODIS
June 2005
nmo2001-2001
climatology
50N-S
Mean:
68.9
135°W
90°W
45°W
0°
45°E
90°E
135°E
[percent]
101.2
65
60°N
60°N
80
95
30°N
30°N
80
65
65
65
35
70
0°
0°
65
50
65
65
30°S
30°S
35
60°S
95
95
95
95
60
20
60°S
65
5
-1
135°W
90°W
45°W
0°
45°E
90°E
135°E
ECMWF
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide
47 0°Mean err
Difference
eskb
- MODIS
50N-S
-11.1 90°E
50N-S rms
13.7
135°W
90°W
45°W
45°E
135°E
[percent]
50
65
65
30°S
30°S
35
60
Nature Run, 12-month total cloud cover,
difference with respect to MODIS observations
60°S
95
95
95
95
20
60°S
65
5
-180 -120
135°W
90°W
45°W
0°
45°E
90°E
135°E
-
long
Trop
Difference
eskb
- MODIS
50N-S0°Mean err
-11.1 90°E
50N-S rms
13.7
135°W
90°W
45°W
45°E
135°E
mo
[percent]
80
57.29
60°N
60°N
10
50
40
30°N
30°N
30
70
20
0°
0°
10
60
-10
-20
30°S
30°S
-30
50
-40
60°S
60°S
-50
50
-53.03
-180 -120
135°W
90°W
45°W
WMO Workshop19-21 May 2008: ECMWF OSEs
0°
45°E
90°E
Slide 48
135°E
ECMWF
-
long
Comparison of extra-tropical cyclones in the NR against
cyclones in the NCEP analyses for 5 recent years (green
bars), showing central pressure (hPa, left) and life span
(days, right panel). Courtesy Joe Terry (NASA).
5.
18.
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 49
ECMWF
OSEs and OSSE activities at ECMWF
Conclusions
 Space-terrestrial study (EUCOS) completed
 Assessment of space-component of the GOS (EUMETSAT)
completed
 Assessment of Metop impact (EUMETSAT) is ongoing
 Comprehensive reports are available
 ECMWF Newsletter No 113, page 16-28 (Kelly and Thépaut)
 Metop-IASI and GPS-RO constitute significant new additions to
the GOS
 Joint-OSSE framework being developed, attracting a lot of
interest, generating wide-spread collaboration
WMO Workshop19-21 May 2008: ECMWF OSEs
Slide 50
ECMWF