Transcript United Nations Economic Commission for Europe Statistical Division The Importance of Databases in the Dissemination Process – The UNECE Approach UNECE Training Workshop on Dissemination.

United Nations Economic Commission for Europe
Statistical Division
The Importance of Databases
in the Dissemination Process
– The UNECE Approach
UNECE Training Workshop on Dissemination of
MDG Indicators and Statistical Information
Astana, Kazakhstan 23 – 25 November 2009
Steven Vale, UNECE
Contents





UNECE system overview
Introduction to data cubes
Input systems
Data processing
Dissemination systems
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 2
What is a Data Cube?
A multi-dimensional structure
containing data points that represent
unique combinations of several
classifications
 A flexible way of storing and
disseminating data

06 November 2015
Steven Vale - UNECE Statistical Division
Slide 4
Two-dimensional Cube
Year
Country 2000
2001
2002
2003
AAA
123 456 124 567 125 678 126 789
BBB
987 654 988 654 989 654 999 654
CCC
35 789
06 November 2015
36 789
37 789
Steven Vale - UNECE Statistical Division
38 789
Slide 5
Threedimensional
Cube
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 6
More dimensions are possible,
but not easy to display!
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 7
Why Data Cubes are Important




Many statistical data management models
and systems are based on cubes
Users can select just those data that are of
interest
Cubes can easily be expanded, e.g. for
extra years, countries, or other categories
At least in theory, cubes can have an
infinite number of dimensions
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 8
Input Systems

Functionality needed:
•
•
•
•
•
•
•
Bulk input of large data files
Automatic data collection routines
Data format conversion
Metadata capture and “translation”
Manual entry of data values
Link to electronic questionnaires
Data validation
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 9
UNECE Approach



Automatic data collection each night
from some important sources
File transfers in standard formats for
other bulk updates
Questionnaires for some types of data
•

Automatic updates under development
Manual input / editing interface
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 10
Data Processing

Functionality needed:
•
•
•
•
•
Data validation
Imputation of missing values
Calculation of derived variables
Calculation of regional aggregates, e.g. for
CIS countries
Definition of data outputs
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 11
UNECE Approach




Create a “super cube” containing all data
Use applications developed ourselves for
validation, imputation and calculation
High level programming language allows
statisticians to develop and manage their
own calculation routines
Smaller output cubes are defined using
metadata, and updated every night
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 12
Dissemination Systems

Functionality needed:
•
•
•
•
•
Internet enabled
Easy access to key data
User-friendly interface
Multiple languages
Possibility to manipulate and download data
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 13
Why UNECE adopted PC-Axis


Lack of resources for system development
PC-Axis advantages:
•
Rich in features
• User-friendly
• Flexible structure
• Strong support network of users – over 40
other statistical organizations
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 14
PC-Axis Around the World
Americas
Licenses (3)
Brasil
Bolivia
Guatemala
Prospects
Canada
Guyana
Argentina
El Salvador
Costa Rica
IMF
Bahamas
UNSD
US Dep.Agric.
Ecuador
Africa
Licenses (14)
Algeria
Mocambique
Namibia
South Africa
Tanzania
Uganda
East Africa Commission
West Africa (ECOWAS)
UEMOAS (FAO)
Kenya
Senegal(FAO)
Mali(FAO)
Togo(FAO)
Cap Verde
CountrySTAT in Projects
(2006-2007) (2008-2009)
Bhutan
Ethiopia
Haiti
Iraq
Malawi
Mali
Mozambique
Palestine O.T.
Philippines
Sudan
Tanzania
Angola
Benin
Burkina Faso
Cameroon
Ethiopia
Ghana
Ivory Coast
Kenya
Malawi
Mali
Mozambique
Nigeria
Rwanda
Senegal
Tanzania
Uganda
Zambia
Asia and Pacific
Licenses (5)
Philippines (2)
Taiwan(R.O.C.)
Bhutan(FAO)
Iraq(FAO)
New Zealand
Prospects
Hong Kong
Tadjikistan
Europe Licenses (68)
Basque (5)
Croatia
Denmark (9)
Estonia
Faroe Islands
Finland (15)
Åland
Greece
Greenland
Iceland
Ireland (2)
Latvia
Lithuania
Macedonia F.Y.R.
Norway
Slovakia
Slovenia (2)
Spain (3)
Ukraine, Lviv
UNECE
Sweden (18)
Prospects
UK ONS
Cyprus
Moldova
Montenegro
North Ireland
Romania
Serbia
Kirgizistan (FAO)
Ukraine
Albania
Switzerland
UK Dep. Work&pens.
FAO Forest Stat.
What We Have Added





Metadata input application
Data cube management application
Time Series Computation Language
PX-Web update server
Russian interface
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 16
Metadata Input Application
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 17
Open-source Components
Visual HTML Designer
Spell checker
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 18
The User Interface

Uses “PX-Web” a component of the
PC-Axis software suite produced by
Statistics Sweden
•



Currently being upgraded to latest version
English and Russian interfaces
“Tree structure” to help users find data
Possibility to manipulate data and
download in several formats
Steven Vale - UNECE Statistical Division
Slide 19
Plans for the Future

Develop End-to-End UNECE applications:
•
•
•
•
•
•

Data import
Validation
Processing
Calculation
Imputation
Dissemination
Develop online analytical tool
06 November 2015
Steven Vale - UNECE Statistical Division
Slide 26
New UNECE
Database System



Under
construction
Calculations and
“Supercube”
implemented
Expected to be
fully operational
end 2010
Technical Assistance



UNECE is happy to share software /
experience
Russian speaking database coordinator
Technical assistance missions 2008/09
•
Kazakhstan
• Kyrgyzstan
• Tajikistan
Steven Vale - UNECE Statistical Division
Slide 28
Questions?
Steven Vale - UNECE Statistical Division
Slide 29