Transcript Document 7264674

High-Resolution Convective Modeling using WRF
Coupled to NASA's Land Information System (LIS)
Christa D. Peters-Lidard (PI)
Hydrological Sciences Branch
NASA/GSFC, Code 614.3
[email protected]
Wei-Kuo Tao (Co-I)
Paul R. Houser (Co-I)
Sujay V. Kumar
Joseph L. Eastman
Stephen E. Lang
Yudong Tian
Xiping Zeng
1
Outline
•LIS Background
•LIS-WRF Coupling Design
•Computational Aspects
•Science Aspects
•Future Directions
2
Background: LIS Team and Collaborators
C. D. Peters-Lidard1, P. R. Houser1, S. V. Kumar1, Y. Tian1, J. Geiger1,
S. Olden1, L. Lighty1, J. L. Eastman1, J. Sheffield2, E. F. Wood2,
P. Dirmeyer3, B. Doty3, J. Adams3, K. Mitchell4, J. Meng1,4, H. Wei4
1NASA,
Goddard Space Flight Center
Hydrological Sciences Branch, Code 974, Greenbelt, MD
2Department of Civil and Environmental Engineering
Princeton University, Princeton, NJ
3Center for Ocean Land Atmosphere Studies (COLA)
Calverton, MD
4NCEP Environmental Modeling Center
NOAA/NWS, Camp Springs, MD
Code and Documentation at
http://lis.gsfc.nasa.gov
3
Background: LIS “Plugin” Design
domain-plugin
Lat/lon
LIS driver
UTM
lsm-plugin
Gaussian
Forcingplugin
parameter-plugin
GEOS
SSIB
CLM
CMORPH
HySSIB
Noah
land
cover
soils
LAI
elevation
CMAP
GDAS
Persiann
AGRM
Mosaic
VIC
Huffman
NLDAS
ECMWF
4
Background: LIS Execution Modes
Uncoupled
LIS
Station Data
Global, Regional
(Re-)Analyses
or Forecasts
Satellite Products
LSM Ensemble
Noah, CLM2, Mosaic,
HYSSiB, VIC
Coupled
ESMF
WRF
ESMF
GCE
GrADS/DODS
Server
5
LIS Background: Initial Benchmarking at NCEP
Average Diurnal Cycles, March 2003
Latent Heat
Sensible Heat
Ground Heat
Net Radiation
Latent Heat
Europe Average
Net Radiation
Ground Heat
Sensible Heat
CONUS Average
•LIS with experimental mode of GFS on the gaussian horizontal grid of T62
•Forcing: NCEP Global Reanalysis II, AGRMET radiation and CMAP precipitation.
•30% more efficient in computing time
6
WRF-LIS: WRF Version
http://www.wrf-model.org
WRF V2.0 Release (May 18, 2004; V2.0.2 October, 2004,
V2.0.3.1 December 2004)
•What is in WRF V2.0?
•Advanced Research WRF (ARW) dynamical core:
•Eulerian mass coordinate
•One-way and two-way nesting
Key project requirement
•New physics options, including :
•Noah Land Surface Model (LSM),
•Rapid Update Cycle (RUC) LSM,
•Ysu Planetary Boundary Layer (PBL), and
•Grell-Devenyi ensemble cumulus scheme
Not actually ESMF,
•ESMF time manager
but a recoded F90
•Enhanced I/O options
version of ESMF!!
•Enhanced Runtime System Library (RSL)
•New Standard Initialization (SI) V2.0
Built on Message
•WRF 3-Dimensional Variational Assimilation
Passing Interface
•(3DVAR) V2.0
7
Goddard Cumulus Ensemble (GCE) Model
Parameters /Processes
Dy na mics
Vertical Co ordina te
Microphy sics
Num erical Methods
Ini tiali zation
FDD A
Radi ation
Sub- Grid D if fusion
Surface Processes
GCE Mod el
Anelas tic or Comp ressible
2D (Slab - and Axis-symm etric) and 3D
Z (p, terrain )
2-Cl ass Wa ter & 3-Cl ass Ice
2-Cl ass Wa ter & 2-Mome nt 4-Class Ice
Spectral-BinM icro ph ysic s
Positiv e Defin ite Ad vec tion for Sca lar Va ria bl es;
4th- Order f or Dyn am ic Varia bl es
Ini tial Condit io ns with Forcin g
from O bs ervations/Large-Sca le Mo dels
Nud ging
k-D istribut io n and Four-Stream Dis crete-Ordin ate Scatterin g (8
bands )
Expl icit Cloud-Radi ation Interactio n
TKE (1.5 order)
Ocean Mixed Laye r
7-Lay er Soil Mo del (PLACE)
CLM - LIS
TO GA COARE Flu x Mo dule
Paralleliz atio n
OPEN- MP an d MPI
8
9
The Goddard Cumulus Ensemble Model
http://rsd.gsfc.nasa.gov/912/code912/model.html
GCE V1.0 Release (June, 2004)
Project advancement
What is in GCE V1.0?
Place Land Surface Model (LSM)
Message Passing Interface (MPI) parallelization
Cyclic lateral boundary conditions
GCE V2.0 Release (Expected July, 2005)
What will be in GCE V2.0?
Place and all LIS LSMs
ESMF Virtual Machine-based MPI parallelism
Cyclic and open boundary conditions
2D vs. 3D
Key project requirement
Project advancements
10
The Earth System Modeling Framework (ESMF)
http://www.esmf.ucar.edu
Climate
Weather
NASA GSFC PSAS
NCAR/LANL CCSM
Data
Assimilation
MITgcm
GFDL FMS Suite
NCEP Forecast
NSIPP Seasonal Forecast
C. DeLuca/NCAR, J. Anderson/NCAR, V. Balaji/GFDL, B. Boville/NCAR, N. Collins/NCAR,
T. Craig/NCAR, C. Cruz/GSFC, A. da Silva/GSFC, R. Hallberg/GFDL, C. Hill/MIT, M. Iredell/NCEP,
R. Jacob/ANL, P. Jones/LANL, B. Kauffman/NCAR, J. Larson/ANL, J. Michalakes/NCAR,
E. Schwab/NCAR, S. Smithline/GFDL, Q. Stout/U Mich, M. Suarez/GSFC, A. Trayanov/GSFC,
S. Vasquez/NCAR, J. Wolfe/NCAR, W. Yang/NCEP, M. Young/NCEP and L. Zaslavsky/GSFC
11
ESMF Status
http://www.esmf.ucar.edu
Planned (Actual)
Milestone
May 2002
Draft Developer’s Guide and Requirements Document completed
1st Community Requirements Meeting and review held in D.C.
July 2002
ESMF VAlidation (EVA) suite assembled
August 2002
Architecture Document: major classes and their relationships
Implementation Report: language strategy and programming
model
Software Build and Test Plan: sequencing and validation
May 2003
ESMF Version 1.0 release, 2nd Community Meeting at GFDL
November 2003
First 3 interoperability experiments completed
April 2004 (July 2004)
Second API and Version 2.0 software release, 3rd Community
Meeting (Version 2.0.2 released in October 2004; and Version
2.1.0rp2 released in March 11, 2005)
November 2004
(Expected Nov 2005)
All interoperability experiments complete; all testbed applications
compliant
January 2005 (Expected
Jan 2006)
Final delivery of source code and documentation
12
ESMF coupling schematic
ESMF Conceptual
Design
Component Coupling:
e.g., LIS-GCE
Components Layer:
Gridded Components
Coupler Components
ESMF Superstructure
Model Layer
Model
Component
Model
Sub-component
ESMF Infrastructure
ESMF Infrastructure
BLAS, MPI, NetCDF, …
BLAS, MPI, NetCDF, …
Component Coupling:
e.g., LIS-WRF
ESMF Superstructure
Fields and Grids Layer
Low Level Utilities
External Libraries
13
LIS-GCE and LIS-WRF coupling
LIS-GCE
LIS-WRF
14
Evaluation Case Study:
International H2O Project (IHOP), May-June 2002
Central US, Southern Great Plains
15
IHOP “Golden Day” Synthetic Case
GOES Imagery & Sounding Data June 6, 2002
13:55 UTC
GOES=Geostationary Operational Environmental Satellite
x
16
Synthetic Case Experimental Design
Vegetation Condition
Soil Moisture Condition
Dry
Wet
Half Wet/Half
Dry
Bare Soil
Baredry
Barewet
Barewetdry
Vegetated
(Grassland)
Grassdry
Grasswet
Grasswetdry
17
Synthetic Case Evaluation
Comparison of Input Radiation
18
Synthetic Case Evaluation
Comparison of Energy Terms and Balance for Grasswet Case
19
Synthetic Case Evaluation
Factor Separation: Impact of Wet Soil and Grass Relative to Bare, Dry
Variable
Wet Soil
Grass
Interaction
GLW (Wm-2)
Longwave Radiation
-4.7
3.1
1.4
SWDOWN (Wm-2)
Shortwave Radiation
-2.0
4.3E-02
-1.6
HFX (Wm-2)
Sensible Heat Flux
-155
39
-54
LH (Wm-2)
Latent Heat Flux
176
-5.8E-03
116
Q2 (kg/kg)
2-m Air Humidity
3.82E-03
-1.80E-04
1.54E-03
T2 (K)
2-m Air Temperature
-5.49
0.26
-0.72
PBLH Planetary Boundary
Layer Height (m)
-490
191
-149
20
Impact of LSM on Uncoupled Performance
Compute nodes used: 128
21
Impact of ESMF on Coupled Performance
Weather Research and Forecasting Model (WRF) Goddard Cumulus Ensemble Model (GCE)
18
18
Default WRF
16
Default GCE
16
LIS-GCE
Total elapsed time (hrs)
Total elapsed time (hrs)
LIS-WRF
14
12
10
8
6
4
14
12
10
8
6
4
2
2
0
0
0
20
40
Number of Processors
60
80
0
20
40
60
Number of Processors
Key conclusion: ESMF-compliant coupling adds minimal
computational overhead relative to native models
22
80
WRF/LIS and WRF/GCE
Performance scaling for the coupled systems
9
8
7
Speedup
6
5
4
3
WRF (100x100)
2
WRF (200x200)
GCE (128x128)
1
GCE (256x256)
0
0
10
20
30
40
50
60
70
Number of Processors
23
Scientific Evaluation: June 12 “Real” Case
Overview of LIS Spinup Impact on WRF+LIS Precipitation
24 hour accumulated precipitation with
default soil initialization
24 hour accumulated precipitation with
LIS 7.5 year spinup soil initialization
24
Scientific Evaluation: June 12 Case
Radar Derived Precipitation vs. Modeled Precipitation (mm)
6GMT June13th, 2002
Observed Radar Derived Surface
Modeled WRF+LIS Precipitation using
Precipitation (Source: NOAA/NCEP)
LIS initial surface conditions
25
Scientific Evaluation: June 12 Case
Domain Integrated Precipitation versus Time
Observed: Radar+Gauge
3.00E+05
2.50E+05
2.00E+05
1.50E+05
1.00E+05
5.00E+04
WRF/LIS Modeled using Radar+Gauge
precipitation for LIS uncoupled Spinup
WRF/LIS Modeled using NOAA Model
precipitation (GDAS) for LIS
uncoupled Spinup
WRF/LIS Modeled using WRF
Standard Initialiation (WRFSI) w/o
LIS spinup
10
:0
0
00
8:
00
6:
00
4:
00
2:
00
0:
:0
0
22
:0
0
20
:0
0
18
:0
0
16
14
12
:0
0
0.00E+00
:0
0
Integrated Precipitation (mm)
3.50E+05
Hour Starting on June12
GDAS=Global Data Assimilation System (NOAA/National Centers for Environmental Prediction (NCEP))
WRFSI
GDAS
NLDAS
RADAR
26
Project Highlights and Future Plans
1.
2.
ESMF-Compliant Coupling of WRF-LIS
•
LISv4.0 has been coupled to WRFv.2.0.3.1 using ESMF
v.2.1.0rp2 and a subcomponent coupling design
•
Coupling needs to be generalized to handle: multiple nests,
projections and grids; multiple PBLs; and LSMs other than
Noah and CLM2.
Recognition
•
3.
LIS selected as co-winner of NASA 2005 Software of the
Year Award
Benchmarking on Columbia
•
We have been allocated 150,000 hours on NASA’s SGI
Altix Supercomputer Columbia for benchmarking and
additional scaling studies for the LIS/WRF and LIS/GCE
coupled systems.
27