Transcript 講演資料 - JVO
バーチャル天文台を実現する 分散データベース・計算資源の 国際連携機構 大石雅寿 (国立天文台) Mar. 01, 2005 DEWS2005 1 JVO collaborators Project Scientists NAOJ • Mizumoto • Oe • Shirasaki • Tanaka • Honda • Kawanomoto ICRR • Yasuda Ochanomizu U. • Masunaga Mar. 01, 2005 System Engineers Fujitsu Ltd. • Monzen • Kawarai • Ishihara • Yamazaki SEC Ltd. • Morita • Nakamoto • Kobayashi • Yoshida Supporter NII •Miura DEWS2005 2 Data Resources in NAOJ • Subaru 8.2m Optical-Infrared Telescope • Kiso 105cm Schmidt Camera Nobeyama 45m • Okayama 188cm Optical Telescope • Nobeyama 45m Radio Telescope • Nobeyama Millimeter Array • Nobeyama Radioheliograph • VSOP Subaru • VERA • ALMA Mar. 01, 2005 DEWS2005 3 Data Resources in JAXA/ISAS • ASCA X-ray astronomy satellite • YOHKO solar physics satellite • Ginga X-ray astronomy satellite • HALCA VLBI satellite • Geotail geomagnetosphere satellite ASCA • Akebono aurora observation satellite • ASTRO-F Infrared satellite • ASTRO-E2 X-ray satellite YOHKO • SOLAR-B Mar. 01, 2005 DEWS2005 HALCA 4 Catalogues, published tables Whole sky view shows heterogeneity Mar. 01, 2005 DEWS2005 5 Simple Spectrum Access Image / Spectrum / Catalog interoperability Mar. 01, 2005 DEWS2005 6 Mar. 01, 2005 DEWS2005 7 VO Projects in the world • 15の国と地域で進行中 (参加希望国増加中） • International Virtual Observatory Alliance (IVOA) VO間の連携のための標準を策定 • 日本は分散連携DBへの統一 検索言語のとりまとめ役 Mar. 01, 2005 DEWS2005 8 Standardization in IVOA • 分散DBへの連携検索言語(VOQL) • メタデータ：内容，交換プロトコル based on OAI-PMH • 画像，スペクトル等の取得プロトコル SkyNode, SIAP, SSAP, STC, etc. • DBの各属性値の呼称の統一 UCD (Unified Contents Descriptions) • 出力形式：VOTable (XML) 天文標準のFITSフォーマットを包含 • 他 Mar. 01, 2005 DEWS2005 9 Mar. 01, 2005 DEWS2005 10 国際機関における認知 • 国際天文学連合—2003年7月に決議採択 • OECD—2004年8月に勧告採択 天文学におけるデータベース活用の重要性が 国際的に認知された Mar. 01, 2005 DEWS2005 11 JVOの概念図 Mar. 01, 2005 DEWS2005 14 Design toward Prototype 3 • Support IVOA Standard Protocols – SIAP, SSAP, SkyNode – implement ADQL • improve registry, employing OAI-PMH architecture • flexible workflow architecture • introduce User management – LDAP – User Storage Area (support VOStore?) • API to control JVO with SOAP Mar. 01, 2005 DEWS2005 15 利便性の確保：検索文の自動生成 Mar. 01, 2005 DEWS2005 16 JVO Query Language (JVOQL) JVOQL is designed as a prototype of VO Query Language. Characteristics of the JVOQL: • SQL based Query Language • Query Language for the distributed astronomical DB. • Can search and retrieve observational data as well as catalog data • Upward compatible with the ADQL and SIAP syntax. • Scalable syntax structure. Very simple core syntax and extension syntax packages. Mar. 01, 2005 DEWS2005 17 Image Search FITS file management table Parameter based Image Query Search Parameters FITS_ID Image Search Engine XML XML,FITS, Jpeg Column metadata request Mar. 01, 2005 Filename File name, Metadata Image FITS file Image cutout SQL based Image Query Data Search on a virtual table. SQL SQL Coord. FITS_ID region Other search parameters region１ … region２ … region３ … DEWS2005 Image SQL image cutout Coord. Filename File name, Metadata 19 JVOQL Syntax Requirement 1 • Unified query language for both the catalog and observation data such as image data, spectrum, 3D-cube, photon list … – Parameter query (SIAP) can be replaced by SQL thinking that the parameters are columns of a relational table. – Observational data or pointer (URL) to retrieve the data is also a column of the relation table. http://jvo.nao.ac.jp/imageData?Pos=24,5&Size=0.2&format=VOTable Select From Where imageURL, … naoj:imageData pos=Point(24,5) and size=0.2 and format=‘VOTable’ “pos” , “size”, “imaegURL” DEWS2005are virtual columns. Mar. 01, 2005 20 Parsing JVOQL and Generating Workflow • “JVOQL Parser” generates query for each host • “Scheduler” generates: – count query job for host1 – count query job for host2 • “Executer” executes jobs on remote hosts • “Scheduler” generates based on the result of execution – query job for host1 – xmatch job for host2 – image query for host1 and host2 • “Executer” executes jobs on remote hosts Mar. 01, 2005 DEWS2005 24 メタデータ • ２種類のメタデータ 1. どのサーバに天文データがあるか？ • 分散アーカイブ連携検索のためにシステムが必要な情報 2. どのような内容の天文データか？ • FITSヘッダに書かれるような情報 • 観測装置、波長帯、観測日時、天域、… • 実装方法の改良 – プロトタイプ１： UDDI • サーバ 情報のみ検索。データ内容の検索は不可 – プロトタイプ２： XMLDB • IVOAで検討されたメタデータ仕様 （XML形式で定義） に準拠 • XMLDB（カレアレア）を用いて検索 Mar. 01, 2005 DEWS2005 25 メタデータ交換:OAI-PMH Mar. 01, 2005 DEWS2005 26 Resource Metadata Identity metadata service metadata curation metadata content metadata title short_name identifier publisher publisher_id creator creator_logo contributer date version contact_name contact_email service_interface_url service_base_url service_http_result service_standard_uri service_standard_url service_msr string string URI string URI string URL string string string string e-mail address URL URL MIME type URI URL float,decimal degrees ucd unit datatype width precision arraysize string string string int string string catalog table column identity ○ ○ ○ Mar. 01, 2005 curation ○ ○ ○ service ○ ○ ○ content ○ ○ ○ column × × ○ column metadata subject description source reference_url type coverage_spatial coverage_region_of_regard coverage_spectral coverage_spectral_bandpass coverage_spectral_central_wavelength coverage_spectral_minimum_wavelength coverage_spectral_maximum_wavelength coverage_temporal_start_time coverage_temporal_stop_time coverage_depth coverage_depth_unit coverage_object_density coverage_object_count coverage_sky_fraction resolution_spatial resolution_spectral resolution_temporal content_level facility instrument format right DEWS2005 string,list string,free text string URL string,list string float,decimal degrees string,list string,list float float float string string float string float int float float float float string,list string,list string,list string,list string 27 JVO SkyNode Architecture JVO Portal Internet JVOQL JVOQL ADQL-x • Apache Axis and Tomcat are used for implementing the VO standard interface, and Globus Toolkit is used for an optional Grid interface. • Four kinds of query languages are converted to java class SelectSQL and query is executed through JDBC-like interface. • Query results is obtained as ResultSet, and table data is formatted to VOTable or CSV file. JVO SkyNode Grid GTK 3 (optional) HTTP/SOAP AXIS SIAP HTTP/Parameter SSAP Tomcat Mar. 01, 2005 Translator Query Executer Formatter DEWS2005 SelectSQL JDBC for SkyNode ResultSet getMetadata() getString() getFits() getJpeg() JVO SkyNode DBMS 29 JVO : Japanese Virtual Observatory • Purpose: – Easy access to federated Astronomical databases – Interoperability through IVOA JVO Interoperability through IVOA Researcher Ground-base telescopes Satellite-borne telescopes Nobeyama Subaru Mar. 01, 2005 ASTRO-EII DEWS2005 31 JVO is seen from the UK VO Mar. 01, 2005 DEWS2005 32 Access time to the US VO 十分実用になるアクセス速度 （ファイル本体の転送時間は含まない） Mar. 01, 2005 DEWS2005 33 Search for Gravitational Lenses produced (?) by Cosmic Strings • SXDS data observed by Subaru • Query results were obtained less than 5 min, displaying SEDs • It has been proven that VO can accelerate researches. Mar. 01, 2005 DEWS2005 35 Mar. 01, 2005 DEWS2005 36 今後の課題 データベースからデータが取れるだけでは 天文学はできない →データ解析ツール群をJava-wrappingなどに よりJVOに追加 →解析ツールの相互利用に向けてwrapping 方式の国際標準化を推進 Mar. 01, 2005 DEWS2005 37 JVOによる仮想データ検索システム Observation System Observation System ASTRO-E2 Remote Observation DATA かに星雲の時刻と波長での違い Slide courtesy of Robert Brunner @ CalTech. Real Time Observation DATA Observation System 「超」仮想ＤＢ すばる望遠鏡 Archived DATA Query Mass storage archiver インターネット Archived DATA DATA ユーザ／研究者 解析／加工処理 Observation System Simulated DATA 公開天文台 On-Demand DATA 解析データ Mar. 01, 2005 DEWS2005 国立天文台４次元デジタル 宇宙プロジェクト ご提供 Simulation System 38 データベースは使って（使えて?） なんぼのもん Mar. 01, 2005 DEWS2005 39