Transcript Slide 1

AIA Core-Team Meeting
20-22 April 2009
JSOC Stuff
Phil Scherrer
5. Data export and centers, US and beyond; data import
a. Data distribution: system and volume requirements
i. JSOC to world
Scherrer 20min 14:40
ii. JSOC to SAO
Scherrer/Davey 10min 15:00
iii. Elsewhere (ROB, Lancashire, …)
iv. Internal flow: Stanford to LM , and back
Break
b. JSOC (direct) data interfaces
i. DRMS system
ii. Web interface
iii. IDL+… interfaces
iv. VSO
c. Data from other instruments:
i. SDO: HMI and EVE, discussion
ii. Other – Discussion
d. Documentation review and action items
6. Summary: HMI data for AIA science investigation
Fleck/Boyes/Dalla 15min 15:10
Hurlburt/Serafin 10min 15:25
15:35 – 15:55
Scherrer 20min 15:55
Summers [TBD] 10min 16:15
Freeland 20min 16:25
Gurman 15min 16:45
Hurlburt/Hock 15min 17:00
Schrijver 10min 17:15
Scherrer/Green 15min 17:25
Hoeksema/Scherrer 10min 17:40
Data export and centers, US and beyond; data import
a. Data distribution: system and volume
requirements
i. JSOC to world
http://jsoc.stanford.edu/jsocwiki/TeamMeetings
Link to SDO Pre-Ship Review(ppt)
http://hmi.stanford.edu/Presentations/SDO-PSR/25-AIA_HMIInst&ScienceOps-PhilScherrer.ppt
And
Link to JSOC Status shown at 2008 SDO Teams Meeting
http://hmi.stanford.edu/TeamMeetings/Mar_2008/Proceedings/JSOC_Status_March_2008.ppt
First page
HMI and AIA JSOC Architecture
GSFC
White Sands
MOC
DDS
HMI & AIA
Operations
Stanford
HMI JSOC Pipeline
Processing System
Redundant
Data
Capture
System
Housekeeping
Database
Quicklook
Viewing
JSOC-IOC
Primary
Archive
12-Day
Archive
Offsite
Archiv
e
LMSAL
Catalog
Offline
Archiv
e
Data
Export
& Web
Service
JSOC-SDP
AIA
Analysis
System
Local
Archive
High-Level
Data Import
JSOC-AVC
World
Science Team
Forecast Centers
EPO
Public
JSOC – SDP Locations at Stanford
Cedar South
Cypress North
P&A 1st floor
P&A Basement
JSOC Interfaces with SDO Ground System
Instrument Commands
RT HK Telemetry (S-band)
MOC at GSFC
DDS at WSC
Science data files
(Ka-band)
DDS
Handshake
files
Spare
Science Data
Capture
AIA Science
System
Data Capture
HMI Science
System
Data Capture
System
RT HK telemetry
L-0 HK files
FDS products
Planning data
AIA MON
monitoring
HMI MON
Planning
monitoring
Planning
SDP segment
JSOC-SDP Stanford
JSOC-SDP Primary responsibilities:
Capture, archive and process science data
Additional: Instrument H&S monitoring
L-0 HK files
FDS products
Mission support data
Instrument
Commands
AIA OPS
Real-time
HMI OPS
Inst Real-time
monitor
andInst
Control
monitor
and Control
RT HK
Telemetry
AIAQL
Quicklook
HMI QL
Planning
Quicklook
Analysis
Planning
Analysis
T&C-Segment
JSOC-IOC LMSAL
JSOC-IOC Primary responsibilities:
Monitor instruments health and safety in real-time, 24/7
Control instrument operations and generate commands
Support science planning functions
JSOC-SDP Major Components
DDS
Support W/S,
FDS, L0 HK,
Pipeline User
Interface, etc.
Data Capture System
Database – DRMS & SUMS
4 Quad Core X86-64
4 Quad Core X86-64
4 Quad Core X86-64
Processors
Processors
Processors
Web Server
& Export Cache
Web
2 Dual Core X86-64
Processors
2 Dual Core X86-64
2 Dual Core X86-64
2 Dual Core X86-64
Processors
Processors
Processors
8 TB Disk
8 TB Disk
8 TB Disk
10 TB Disk
HMI
LTO-4 Tape Library
SPARE
2 Dual Core X86-64
Processors
1 TB Disk
2 TB Disk
Offsite
10-Gig
ethernet
Link
Pipeline Processor Cluster
SUMS Server
512 cores in 64 nodes
SPARE@MOC
Firewall
Workstations
Export
LMSAL
LTO-4 Tape Library
Local Science
Secondary
AIA
Processors
LTO-4 Tape Library
10 TB Disk
Primary
LTO-4 Tape Library
2 Dual Core X86-64
Link
10 TB Disk
2 Quad Core X86-64
Processors
LTO-4 Tape Library
10 TB Disk
1-Gig
2 Quad Core X86-64
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2 Quad Core X86-64
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
2
Quad Core X86-64
Processors
2 Quad Core X86-64
Processors
Processors
Processors
400 TB Disk
Tapes
12 LTO-4 Drives
150 TB Disk
2200 Cartridge
Per year
Library
Fast interconnect
JSOC Dataflow Rates
LMSAL secure host
0.04
Hk
Joint
Ops
Dataflow (GB/day)
Quick Look
1610
1230
Data Capture
1230
2 processors each
HMI &
AIA
Science
1210
Level 0
(HMI & AIA)
Level 1
(HMI)
HMI High Level
Processing
2 processors
16 processors
c. 200 processors
1210
75
1610
1200
30d cache
40TB each
Online Data
325TB+50TB/yr
rarely
needed
240
1820
Redundant data
capture system
LMSAL Link
(AIA Level 0, HMI
Magnetograms)
Data Exports
1230
Science
Archive
440TB/yr
(Offiste)
HMI Science
Analysis
Archive
650TB/yr
2 processors
SDO Scientist &
User Interface
JSOC Data Volumes from Proposal
this version modified to show the links to the hardware plan
Data Path Assumptions
img size
Processe Volume Combin Online
d at
(GB/day) ed
disk
(GB/day cache
channel cadenc compre
)
days
s
e
ss
In from
DDS
HMI: 55,000,000 bps **
SU
553
AIA: 67,000,000 bps **
SU
674
Level-0
HMI: 4k*4k*2 bytes/2-seconds*(pi/4)
0.39 SU
530
3.4E+07
2
4
AIA: 4k*4k*2 bytes * 8 imgs per 10 seconds
3.4E+07
8
10
0.50 SU
1,080
HMI: V,M,Ic @ 45s & B, ld, ff @ 90s*(pi/4)
3.4E+07
5.5
45
0.39 SU
130
AIA: Level 1.0 same as level-0
3.4E+07
8
10
0.50 tbd
1,080
Higher
level
HMI: See below
7.5E+10
1 86400
1.00 SU
70
AIA (lev1a): movies & extracted regions. @ 20%
6.7E+06
8
10
0.50 LM
216
LMSAL
Link
HMI: Magnetograms (M, B)
3.4E+07
5
90
0.39 na
59
AIA: Full Level-0 data+lev1_extract
3.5E+07
8
10
0.50 na
1,134
Export
HMI: 2 * Higher Level products + 5*10 min B
SU
149
AIA: 3* higher Level products (TRACE < 1)
SU
648
HMI: tlm
SU
553
AIA: tlm
SU
674
Local tape HMI: Lev0, Lev-1, All Higher
SU
730
AIA: Lev0, Lev1a
SU
1,296
Level-1
Offsite
tape
Totals
1,227
1,610
1,210
286
1,193
797
Fixed
Disk
cache
(TB)
Perm
Tape
Tape per
disk per Archive year
year
Fraction (TB)
(TB)
Nearline
retain
days
Nearline
Cache
(TB)
30
16
200%
395
90
49
30
20
200%
482
90
59
100
52
100%
189
180
93
30
32
100%
386
1,900
2,004
0
0
90
95
10%
39
0
0
25
100%
25
0
0
77
100%
77
0
0
46
0
0
0
0
0
100
6
0
0
100
111
0
0
60
1
0
0
60
6
0
0
100%
198
24
100%
241
30
1,227
2,026
412
93
743
2,004
HMI Totals
68
71
610
118
AIA Totals
146
77
984
2,034
Combined (TB)
214
148
1,594
2,151
Tape shelf size (TB)
Tape shelf number of tapes - mixed density
7,968
11,257
JSOC
Data
Volumes from Proposal
Fixed
Perm
Tape
Tape per Data Path Assumptions
this version modified to show the links to the hardware plan
Combin
ed
(GB/day
)
1,227
1,610
1,210
286
1,193
797
Online
disk
cache
days
Disk
cache
(TB)
disk per Archive year
year
Fraction (TB)
(TB)
im
HMI: 55,000,000 bps **
100%
395 In from
DDS
482
189 Level-0
HMI: 4k*4k*2 bytes/2-seconds*(pi/4)
3
100%
386
AIA: 4k*4k*2 bytes * 8 imgs per 10 seconds
3
HMI: V,M,Ic @ 45s & B, ld, ff @ 90s*(pi/4)
3
39
AIA: Level 1.0 same as level-0
3
25 Higher
level
77
0 LMSAL
Link
0
0 Export
HMI: See below
7
AIA (lev1a): movies & extracted regions. @ 20%
6
HMI: Magnetograms (M, B)
3
AIA: Full Level-0 data+lev1_extract
3
30
16
200%
30
20
200%
100
52
30
32
0
0
90
95
0
0
0
25
100%
0
77
100%
100
6
100
111
60
1
60
6
0 Level-1
46
10%
0
100%
1,227
100%
2,026
AIA: 67,000,000 bps **
HMI: 2 * Higher Level products + 5*10 min B
AIA: 3* higher Level products (TRACE < 1)
HMI: tlm
198 Offsite
tape
AIA: tlm
241
412 Local tape HMI: Lev0, Lev-1, All Higher
743
AIA: Lev0, Lev1a
68
71
610 Totals
HMI Totals
146
77
984
AIA Totals
214
148
1,594
Combined (TB)
7,968
Tape shelf size (TB)
11,257
Tape shelf number of tapes - mixed density
JSOC Processing “Levels”
•
Tlm is raw telemetry files as received from SDOGS
•
Level-0 is images extracted from tlm with added meta-data, no change
to pixels
•
Level-1 is cleaned up and calibrated into physical units in
standardized form
•
Level-2 is science data products
•
Level-3 is higher level products or user produced products and are not
JSOC products but may be archived and distributed as desired by
owner
JSOC DCS Science Telemetry Data Archive
•
•
•
•
•
•
Telemetry data is archived twice
The Data Capture System (DCS) archives tlm files for offsite storage
Archive tapes are shipped to the offsite location and verified for
reading
The Data Capture System copies tlm files to the Pipeline Processing
System
The Pipeline Processing System generates Level-0 images and
archives both tlm and Level-0 data to tape
Only when the DCS has received positive acks on both tlm archive
copies does it inform the DDS, which is now free to remove the file
from its tracking logic
HMI and AIA Level-0
•
Level-0 Processing is the same for HMI and AIA
•
Level 0.1 – Immediate – Used for Ops quicklook
–
–
–
–
•
Reformat images
Extract Image Header meta-data
Add “Image Status Packet” high-rate HK Packet (per image)
Export for JSOC IOC Quicklook
Level 0.3 – Few minute lag - Used for quicklook science data products
– Add other RT HK meta-data
– Add FDS S/C info
•
Level 0.5 – Day or more lag – Used for final science data products
– Update FDS data
– Add SDO roll info
– Includes final images
Level 1 HMI and AIA Basic Observable Quantities
•
•
HMI and AIA level-1 “levels” are similar but the details differ (a lot).
HMI – from filtergrams to physical quantities
– 1.0 Flat field applied to enable limb fit and registration
– 1.5 Final product types
• 1.5q – Quicklook available in ~10 minutes, saved ~10 days
• 1.5p – Provisional mix of 1.5q and 1.5 final
• 1.5 – Final best possible product
– Products
• Continuum Intensity
• Doppler Velocity
• Line of Sight Field
• Vector Field
•
AIA – Filtergrams are basic product
– Quicklook and Final both produced
– Planning movies from quicklook
– Full details in development
Newer Processing Flow Diagrams
http://jsoc.stanford.edu/jsocwiki/Lev1Doc
Link to Cmap at diagram to
http://jsoc.stanford.edu/Cmaps/web/JSOC_SDP_Data_Flow.html
On following pages ---
DDS
Level-0
HSB image
Immediate or
Retransmitted,
permanent
JSOC
IOC
Ground Tables
DDS
Level-0 HK
ISP
HK via
MOC
HK 1553
Other APID
Level-0
Immediate
Level 1.0q
Flat fielded and
bad pixel list included,
Temp 1d
Select nearest
or average
Command logs
if needed
MOC
FDS predict data
dayfiles
MOC
FDS final data
dayfiles
FDS series,
temp
MOC
HK 1553 APID
dayfiles
Level-0 HK,
temp
SDO HK dayfiles
From MOC
JSOC-IOC
quicklook,
Temp, 5d
Level-0.1
JSOC
IOC
MOC
JSOC LEVEL-0 Processing
Level-0.3
FDS series,
temp
SDO HK lev0 temp
Few minutes lag
Level 1.5q
Quicklook Observables,
Temp, 5d
Level 1.5p
Provisional Observables,
Links to best avail
Level 1.5
Final Observables,
permanent
Level-0.5
Day lag
Level 1.0
Flat fielded and
bad pixel list included,
Temp 60d
Configuration Management & Control
• Capture System
– Managed by JSOC-SDP CCB after August freeze
– Controlled in CVS
• SUMS, DRMS, PUI, etc. Infrastructure
– Managed by JSOC-SDP CCB after launch
– Controlled in CVS
• PUI Processing Tables
– Managed by HMI and/or AIA Instrument Scientist
– Controlled in CVS
• Level 0,1 Pipeline Modules
– Managed by HMI and/or AIA Instrument Scientist
– Controlled in CVS
• Science Analysis Pipeline Modules
– Managed by program author
– Controlled in CVS
Data export and centers, US and beyond; data import
a. Data distribution: system and volume
requirements
And
b. JSOC (direct) data interfaces
i. DRMS system
ii. Web interface
http://jsoc.stanford.edu/jsocwiki/TeamMeetings
Link to JSOC Status shown at 2008 SDO Teams Meeting
http://hmi.stanford.edu/TeamMeetings/Mar_2008/Proceedings/JSOC_Status_March_2008.ppt
First page
JSOC Export
•
•
•
•
•
ALL HMI and AIA data will be available for export at level-1 through
standard products (level-1 for both and level-2 for HMI)
It would be unwise to expect to export all of the data. It is simply not a
reasonable thing to expect and would be a waste of resources.
Our goal is to make all useful data easily accessible.
This means “we” must develop browse and search tools to help generate
efficient data export requests.
Quicklook Products
– Quicklook raw images to JSOC IOC
– Quicklook Basic Products to Space Weather Users
•
Prime Science Users
–
–
–
–
–
•
JSOC will support Virtual Solar Observatory (VSO) access
JSOC will also have a direct web access
There will be remote DRMS/SUMS systems at key Co-I institutions
JSOC In Situ Delivery and processing
Special Processing at JSOC-SDP as needed and practical
Public Access
– Web access for all data; Special products for E/PO and certain solar events
JSOC DRMS/SUMS Basic Concepts
•
•
•
•
•
•
•
•
•
•
•
Each “image” is stored as a record in a data “series”.
There will be many series: e.g. hmi_ground.lev0 is ground test data
The image metadata is stored in a relational database – our Data Record
Management System (DRMS)
The image data is stored in SUMS (Storage Unit Management System)
which itself has database tables to manage its millions of files.
SUMS owns the disk and tape resources.
Users interact with DRMS via a programming language, e.g. C, FORTRAN,
IDL.
The “name” of a dataset is actually a query in a simplified DRMS naming
language that also allows general SQL clauses.
Users are encouraged to use DRMS for efficient use of system resources
Data may be exported from DRMS as FITS or other protocols for remote
users.
Several Remote DRMS (RDRMS) sites will be established which will
“subscribe” to series of their choice. They will maintain RSUMS containing
their local series and cached JSOC series.
The JSOC may act as an RDRMS to access products made at remote sites.
JSOC data organization
•
Evolved from FITS-based MDI dataset concept to
– Fix known limitations/problems
– Accommodate more complex data models required by higher-level processing
•
Main design features
– Lesson learned from MDI: Separate meta-data (keywords) and image data
• No need to re-write large image files when only keywords change (lev1.8 problem)
• No (fewer) out-of-date keyword values in FITS headers
• Can bind to most recent values on export
– Easy data access through query-like dataset names
• All access in terms of sets of data records, which are the “atomic units” of a data series
• A dataset name is a query specifying a set of data records (possibly from multiple data series):
– Storage and tape management must be transparent to user
• Chunking of data records into “storage units” and tape files done internally
• Completely separate storage and catalog databases: more modular design
• Legacy MDI modules should run on top of new storage service
– Store meta-data (keywords) in relational database (PostgreSQL)
• Can use power of relational database to rapidly find data records
• Easy and fast to create time series of any keyword value (for trending etc.)
• Consequence: Data records for a given series must be well defined (i.e. have a fixed set of
keywords)
DRMS DataSeries
• A Dataseries consists of:
– A SeriesName which consists of
• <projectname>.<productname>
– a sequence of Records which consist of a set of:
• Keywords and
• Segments which consist of:
– structure information and
– storage unit identifier
• Links which provide pointers to associated records in other
series.
– A list of 0 or more PrimeKeys which are keywords sufficient to
identify each record (default to “recnum”)
DRMS DataSeries - cont
• Data is accessed in RecordSets which are collections of
records identified by seriesname and primekeys
• RecordSets are identified by a “name” which is really a
query.
• Records may have versions which have the same set of
primekey values, most recent is current record.
• See: http://jsoc.stanford.edu/jsocwiki/DataSeries
Logical Data Organization
JSOC Data Series
Data records for
series hmi.fd_V
hmi.lev0_cam1_fg
aia.lev0_cont1700
hmi.lev1_fd_M
hmi.lev1_fd_V
aia.lev0_FE171
…
hmi.lev1_fd_V#12345
hmi.lev1_fd_V#12346
hmi.lev1_fd_V#12347
hmi.lev1_fd_V#12348
hmi.lev1_fd_V#12349
hmi.lev1_fd_V#12350
hmi.lev1_fd_V#12351
hmi.lev1_fd_V#12352
Keywords:
RECORDNUM = 12345 # Unique serial number
T_OBS = ‘2009.01.05_23:22:40_TAI’
DATAMIN = -2.537730543544E+03
DATAMAX = 1.935749511719E+03
...
P_ANGLE = LINK:ORBIT,KEYWORD:SOLAR_P
…
Links:
ORBIT = hmi.lev0_orbit, SERIESNUM = 221268160
CALTABLE = hmi.lev0_dopcal, RECORDNUM = 7
L1 = hmi.lev0_cam1_fg, RECORDNUM = 42345232
R1 = hmi.lev0_cam1_fg, RECORDNUM = 42345233
…
Data Segments:
hmi.lev1_fd_V#12353
…
Single hmi.fd_V data record
Storage Unit
= Directory
Velocity =
JSOC Pipeline Processing System Components
Pipeline
Operato
r
Pipeline
processing
plan
JSOC Science
Libraries
Processing
script, “mapfile”
PUI
Pipeline User
Interface
Pipeline Program, “module”
List of pipeline
modules with
needed datasets for
input, output
Utility Libraries
SUMS Disks
DRMS Library
Record
Manage
ment
Keyword
Access
Link
Manage
ment
Record Cache
Data
Access
SUMS
Storage Unit
Management System
DRMS
Processing
History Log
Data Record
Management System
Database Server
SUMS
Tape
Farm
Pipeline batch processing
•
A pipeline “session” is encapsulated in a single database transaction:
– If no module fails all data records are commited and become visible to other clients of the JSOC catalog
at the end of the session
– If failure occurs all data records are deleted and the database rolled back
– It is possible to commit data produced up to intermediate checkpoints during sessions
Pipeline session = atomic transaction
DRMS Server
Initiate session
Analysis pipeline
Module 2.1
…
Module 1
Module N
DRMS Server
Commit Data
&
Deregister
DRMS API
DRMS API
DRMS API
Module 2.2
DRMS API
Input data Output data
records
records
DRMS Service = Session Master
Record & Series
Database
SUMS
Data export and centers, US and beyond; data import
ii. JSOC to SAO
http://jsoc.stanford.edu/netdrms/
First page
Remote DRMS/SUMS
• Cooperating sites run NetDRMS code which is
the JSOC DRMS/SUMS code base.
– They maintain their own PostgreSQL database
– Remote systems can “subscribe” to series created at
other DRMS sites
– Subscribed series DRMS records are synchronized
automatically with a short lag
– SUMS Storage Units (SUs) which contain the file data
are imported on demand to the remote SUMS when a
non-local sunum is requested.
– JSOC will serve all and will receive data from some
remote sites.
Remote DRMS Sites
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Site
Location
Contact
SUMS ID
CFA Cambridge, MA, USA
Alisdair Davey
0x0004
CORA Boulder, CO, USA
Aaron Birch
0x0005
GSFC Greenbelt, MD, USA
Joe Hourclé
0x0002
IAS Toulouse, France
Frederic Auchere
0x0018
IIAP Bangalore, India
Paul Rajaguru
0x000c
JSOC Stanford, CA, USA
Art Amezcua
0x0000
JILA Boulder, CO, USA
Deborah Haber
0x0008
LMSAL Palo Alto, CA, USA
John Serafin
0x0023
MPI
Katlenburg-Lindau, Germany Raymond Burston 0x0001
MSSL Dorking, UK
Elizabeth Auden
0x0020
NSO Tucson, AZ, USA
Igor Suarez-Sola
0x0003
ROB Brussels, Belgium
Benjamin Mampaey 0x001d
Yale New Haven, CT, USA
Charles Baldner
0x0010
Web Access
• JSOC page at http://jsoc.stanford.edu
– Semantics see: Jsocwiki at
http://jsoc.stanford.edu/jsocwiki
– Syntax for code see: Man Pages
– Access for data see: e.g.
http://jsoc.stanford.edu/ajax/lookdata.html
– Also links for CVS repository and trouble reports
First page
Work Remaining
•
•
•
•
•
•
•
•
•
•
Oh, gee…
Web browsable catalog
Better “user experience”
Links to HKB
VSO provided SU availability catalog
…
Testing
Not to mention HMI analysis code…
Testing
Data from the Sky
6. Summary: HMI data for AIA science investigation
http://hmi.stanford.edu/Presentations/LWS-2007TeamsDay/HMI_Dataproducts_Sept_2007.ppt
First page
Primary goal: origin of solar variability
• The primary goal of the Helioseismic and Magnetic Imager (HMI)
investigation is to study the origin of solar variability and to
characterize and understand the Sun’s interior and the various
components of magnetic activity.
• HMI produces data to determine the interior sources and
mechanisms of solar variability and how the physical processes
inside the Sun are related to surface and coronal magnetic fields
and activity.
LWS Teams Day JSOC Overview
Page 40
Key Features of HMI Science Plan
•
•
•
•
•
•
Data analysis pipeline: standard helioseismology and magnetic field
analyses
Development of new approaches to data analysis
Targeted theoretical and numerical modeling
Focused data analysis and science working groups
Joint investigations with AIA and EVE
Cooperation with other space- and ground-based projects (SOHO, Hinode,
PICARD, STEREO, RHESSI, GONG+, SOLIS, HELAS)
LWS Teams Day JSOC Overview
Page 41
HMI Major Science Objectives
1.B – Solar Dynamo
1.J – Sunspot Dynamics
1.I – Magnetic Connectivity
1.C – Global Circulation
1.A – Interior Structure
1.D – Irradiance Sources
1.H – Far-side Imaging
1.E – Coronal Magnetic Field
NOAA
9393
Farside
1.G – Magnetic Stresses
LWS Teams Day JSOC Overview
1.F – Solar Subsurface Weather
Page 42
Primary Science Objectives
1.
2.
3.
4.
5.
Convection-zone dynamics and solar dynamo
–
Structure and dynamics of the tachocline
–
Variations in differential rotation.
–
Evolution of meridional circulation.
–
Dynamics in the near-surface shear layer.
Origin and evolution of sunspots, active regions and complexes of activity
–
Formation and deep structure of magnetic complexes.
–
Active region source and evolution.
–
Magnetic flux concentration in sunspots.
–
Sources and mechanisms of solar irradiance variations.
Sources and drivers of solar activity and disturbances
–
Origin and dynamics of magnetic sheared structures and delta-type sunspots.
–
Magnetic configuration and mechanisms of solar flares and CME.
–
Emergence of magnetic flux and solar transient events.
–
Evolution of small-scale structures and magnetic carpet.
Links between the internal processes and dynamics of the corona and heliosphere
–
Complexity and energetics of solar corona.
–
Large-scale coronal field estimates.
–
Coronal magnetic structure and solar wind
Precursors of solar disturbances for space-weather forecasts
–
Far-side imaging and activity index.
–
Predicting emergence of active regions by helioseismic imaging.
–
Determination of magnetic cloud Bs events.
LWS Teams Day JSOC Overview
Page 43
HMI Science Analysis Plan
HMI Data
Processing
Data Product
Science Objective
Global
Helioseismology
Processing
Internal rotation Ω(r,Θ)
(0<r<R)
Tachocline
Internal sound speed,
cs(r,Θ) (0<r<R)
Differential Rotation
Local
Helioseismology
Processing
Full-disk velocity, v(r,Θ,Φ),
And sound speed, cs(r,Θ,Φ),
Maps (0-30Mm)
Activity Complexes
Filtergrams
Carrington synoptic v and cs
maps (0-30Mm)
Observables
Doppler
Velocity
High-resolution v and cs
maps (0-30Mm)
Deep-focus v and cs
maps (0-200Mm)
Far-side activity index
Line-of-sight
Magnetograms
Vector
Magnetograms
Continuum
Brightness
Line-of-Sight
Magnetic Field Maps
Near-Surface Shear Layer
Active Regions
Sunspots
Irradiance Variations
Magnetic Shear
Flare Magnetic Configuration
Flux Emergence
Magnetic Carpet
Coronal energetics
Vector Magnetic
Field Maps
Large-scale Coronal Fields
Coronal magnetic
Field Extrapolations
Far-side Activity Evolution
Coronal and
Solar wind models
Brightness Images
LWS Teams Day JSOC Overview
Meridional Circulation
Solar Wind
Predicting A-R Emergence
IMF Bs Events
Version 1.0w
Page 44
HMI module status and MDI heritage
Intermediate and high level data products
Primary
observables
Heliographic
Doppler velocity
maps
Mode frequencies
And splitting
Ring diagrams
Local wave
frequency shifts
Doppler
Velocity
Tracked Tiles
Of Dopplergrams
Internal rotation
Spherical
Harmonic
Time series
Time-distance
Cross-covariance
function
Wave travel times
Egression and
Ingression maps
Wave phase
shift maps
Internal sound speed
Full-disk velocity,
sound speed,
Maps (0-30Mm)
Carrington synoptic v and
cs maps (0-30Mm)
High-resolution v and cs
maps (0-30Mm)
Far-side activity index
Line-of-sight
Magnetograms
Stokes
I,Q,U,V
Full-disk 10-min
Averaged maps
Vector Magnetograms
Fast algorithm
Tracked Tiles
Vector Magnetograms
Inversion algorithm
Coronal magnetic
Field Extrapolations
Solar limb parameters
Coronal and
Solar wind models
Brightness feature
maps
Brightness Images
LWS Teams Day JSOC Overview
Tracked full-disk
1-hour averaged
Continuum maps
Standalone
production codes
in use at Stanford
Research codes in use
by team
Deep-focus v and cs
maps (0-200Mm)
Stokes
I,V
Continuum
Brightness
MDI pipeline
modules exist
Line-of-Sight
Magnetic Field Maps
Vector Magnetic
Field Maps
Codes being
developed in the
community
Codes to be
developed at HAO
Codes to be
developed at
Stanford
Page 45
JSOC - HMI Pipeline
Processing
HMI Data
Heliographic
Doppler velocity
maps
Filtergrams
Level-0
Doppler
Velocity
Spherical
Harmonic
Time series
To l=1000
Mode frequencies
And splitting
Ring diagrams
Local wave
frequency shifts
Time-distance
Tracked Tiles
Cross-covariance
Of Dopplergrams
function
Egression and
Ingression maps
Level-1
Data Product
Wave travel times
Wave phase
shift maps
Internal rotation Ω(r,Θ)
(0<r<R)
Internal sound speed,
cs(r,Θ) (0<r<R)
Full-disk velocity, v(r,Θ,Φ),
And sound speed, cs(r,Θ,Φ),
Maps (0-30Mm)
Carrington synoptic v and cs
maps (0-30Mm)
High-resolution v and cs
maps (0-30Mm)
Deep-focus v and cs
maps (0-200Mm)
Far-side activity index
Stokes
I,V
Line-of-sight
Magnetograms
Stokes
I,Q,U,V
Full-disk 10-min
Averaged maps
Vector Magnetograms
Fast algorithm
Tracked Tiles
Vector Magnetograms
Inversion algorithm
Coronal magnetic
Field Extrapolations
Tracked full-disk
1-hour averaged
Continuum maps
Solar limb parameters
Coronal and
Solar wind models
Brightness feature
maps
Brightness Images
Continuum
Brightness
HMI Data Analysis Pipeline
LWS Teams Day JSOC Overview
Line-of-Sight
Magnetic Field Maps
Vector Magnetic
Field Maps
Page 46
Magnetic Fields
Filtergrams
Stokes
I,V
Line-of-sight
Magnetograms
Stokes
I,Q,U,V
Full-disk 10-min
Averaged maps
Vector Magnetograms
Fast algorithm
Tracked Tiles
Vector Magnetograms
Inversion algorithm
Line-of-Sight
Magnetic Field Maps
Vector Magnetic
Field Maps
Coronal magnetic
Field Extrapolations
Coronal and
Solar wind models
Code: Stokes I,V,
Lev0.5 V & LOS
field
J. Schou
S. Tomzcyk
Code: Stokes
I,Q,U,V
J. Schou
S. Tomzcyk
Status: in
development
Status: in
development
LWS Teams Day JSOC Overview
Page 47
Line-of Sight Magnetic Field
Filtergrams
Stokes
I,V
Line-of-sight
Magnetograms
Line-of-Sight
Magnetic Field Maps
Synoptic
Magnetic Field Maps
Magnetic Footpoint
Velocity Maps
Code: LOS
magnetograms
J. Schou
S. Tomzcyk
R. Ulrich (cross
calib)
Status: in
development
LWS Teams Day JSOC Overview
Code: LOS
magnetic maps
(project?)
T. Hoeksema
R. Bogart
Status: in
development
Code: Synoptic
Magnetic Field
Maps
T. Hoeksema
X. Zhao
R. Ulrich
Status: in
development
Code: Velocity
Maps of Magnetic
Footpoints
Y. Liu
G. Fisher
Status: in
development
Page 48
Vector Magnetic Field
Vector Magnetic
Field Maps
Filtergrams
Stokes
I,Q,U,V
Full-disk 10-min
Averaged maps
Tracked Tiles
Vector Magnetograms
Fast algorithm
Vector Magnetograms
Inversion algorithm
Coronal magnetic
Field Extrapolations
Coronal and
Solar wind models
Code: fastrack
R. Bogart
Status: needs
modifications
for fields
Code: Vector
Field Fast and
Inversion
Algorithms
J. Schou
S. Tomzcyk
Code: Vector
Field Maps
T. Hoeksema
Y.Liu
Status: in
development
Status: in
development
Code: Coronal
Field
Extrapolations &
Ambiguity issue
T.Hoeksema
Y.Liu, X.Zhao
C. Schrijver
P.Goode T.Metcalf
K.D.Leka
Status: in
development
Code: Solar
Wind Models
X.Zhao
K.Hayshi
J.Linker
P.Goode
V.Yurchishin
Code: Coronal
Magnetic Field
Topological
Properties
J.Linker
V. Titov
Status: in
development
Status: needs
implementation
Need $$
LWS Teams Day JSOC Overview
Page 49
Intensity
Brightness Images
Filtergrams
Tracked full-disk
1-hour averaged
Continuum maps
Continuum
Brightness
Brightness Synoptic Maps
Solar limb parameters
Solar limb parameters
per image for Lev0.5
Code: Continuum
Maps
Schou
Code: Averaged
Continuum Maps
Bush
Code: Solar Limb
Parameters,
Lev0.5 used to make
other Lev1 products,
Lev2 for science goals
Brightness feature
maps
Code: Brightness
Feature Maps
(European
contribution)
Code: Brightness
Synoptic Maps
Scherrer
Status: in
development
Status: in
development
R. Bush
J. Kuhn
Status: in
development
LWS Teams Day JSOC Overview
Status: in
development
Status: in
development
Page 50