Transcript CREX software Milan Dragosavac Slide 1 WMO BUFR training course Langen 17-20 April 2007 Slide 1

CREX software
Milan Dragosavac
Slide 1
WMO BUFR training course
Langen
17-20 April 2007
Slide 1
Requirements for CREX software
 Suitable for small volume of observations
 Building blocks for other applications
 Encoding
 Decoding
 Conversion to BUFR
 Low maintenance costs
 Portable
Slide 2
WMO BUFR training course
Langen
17-20 April 2007
Slide 2
Programming languages used
 Fortran 77
- All CREX subroutines and functions
C
- Reading/writing CREX data
- gbyte, sbyte routines unpack/pack bytes
Slide 3
WMO BUFR training course
Langen
17-20 April 2007
Slide 3
Ecmwf CREX software
 How to get the software?
The software can be downloaded from Ecmwf web site
http://www.ecmwf.int/products/data/software/
 Go to Free download and get crex_000320.tar.gz file
 gunzip crex_000320.tar.gz
 cd crex_000320
 Read README file for explanations
 Execute make command which will create libcrex.a
 cd example
Slide 4
 Running make will create number of executables
WMO BUFR training course
Langen
17-20 April 2007
Slide 4
Ecmwf CREX software
 Content of crex_000310 directory
-rw-r----- 1 maa ma
450 Jul 5 2006 Makefile
-rw-r----- 1 maa ma 10758 Jul 5 2006 README
-rwxr----- 1 maa ma
3662 Jul 5 2006 build_library
drwxr-x--- 2 maa ma
4096 Jul 5 2006 config
drwxr-x--- 2 maa ma
4096 Feb 12 07:53 crexdc
drwxr-x--- 2 maa ma
4096 Feb 12 11:07 crextables
drwxr-x--- 2 maa ma
4096 Jul 5 2006 doc
drwxr-x--- 3 maa ma
4096 Feb 12 09:27 example
-
Slide 5
WMO BUFR training course
Langen
17-20 April 2007
Slide 5
Ecmwf CREX software
rwxr----- 1 maa ma
3107 Jul 5 2006 install
-rw-r----- 1 maa ma
1196 Jul 5 2006 licence
drwxr-x--- 2 maa ma
4096 Jul 5 2006 options
drwxr-x--- 2 maa ma
4096 Feb 12 07:53 pbio
Slide 6
WMO BUFR training course
Langen
17-20 April 2007
Slide 6
Ecmwf CREX software
 config directory contains various configuration files
for different platforms and compilers
 options directory contains various options for different
platforms
 You can make additional configurations and options if
needed
Slide 7
WMO BUFR training course
Langen
17-20 April 2007
Slide 7
Ecmwf CREX software
 pbio
pbio directory contain all c and fortran routines needed
for BUFR and CREX IO
all routines are part of bufr and crex libraries
 crexdc
crexdc directory contain all crex software source code
routines which are written in fortran 77
Slide 8
WMO BUFR training course
Langen
17-20 April 2007
Slide 8
Ecmwf CREX software
 example
bufr2crex.F
create_crex.F
crex2bufr.F
crextdexp.F
decode_crex.F
 doc
doc directory contain CREX User’s Guide and Reference
Slide 9
Manual in pdf format
WMO BUFR training course
Langen
17-20 April 2007
Slide 9
Ecmwf CREX software
 Crex is table driven code form
- no need to transfer element names, units …
- no need to change CREX software for new
observations
 Crex table B – classification elements
- reference number
- element name
- element unit
- scale
- data width in bytes
Slide 10
WMO BUFR training course
Langen
17-20 April 2007
Slide 10
Ecmwf CREX software
 Crex table D – list of common sequences to describe the
data
- table B elements
- operators
- other table D descriptors
 Crex table C – text and meaning of code/flag tables
Slide 11
WMO BUFR training course
Langen
17-20 April 2007
Slide 11
Ecmwf CREX software
 CREX table naming convention
BXXYYZZ
DXXYYZZ
B - Crex table B
D - Crex table D
XX - Crex Master table used “00”
YY - Crex edition number used “01”
ZZ - Crex table version number “03”
Slide 12
WMO BUFR training course
Langen
17-20 April 2007
Slide 12
Ecmwf CREX software
 PATH for CREX tables
export CREX_TABLES=/xxx/
setenv CREX_TABLES /xxx/
The path must end with back slash “/”
 Check digit indicator
export USE_E=true
setenv USE_E true
Slide 13
WMO BUFR training course
Langen
17-20 April 2007
Slide 13
Ecmwf CREX software
 Defaults
Integer missing value indicator
NVIND=2147483647
Real*8 missing value indicator
RVIND= 1.7E38
Slide 14
WMO BUFR training course
Langen
17-20 April 2007
Slide 14
Decoding CREX data
There are 4 steps in CREX data processing
 Open crex input file
 Read in one crex message at a time
 Expand crex message
 Use the data
Repeat last three steps until the end of file is reached
Slide 15
WMO BUFR training course
Langen
17-20 April 2007
Slide 15
PBIO routines
FORTRAN routines to handle products in an unblocked binary file
subroutine PBOPEN ( kunit, filename, mode, kerr )
Input arguments:
filename - character variable
mode
- character variable ‘r’ read, ‘w’ write, ‘a’ append, ‘r+’ read and
write
Output arguments:
kunit - integer unit number
kerr - integer status return code
Slide 16
WMO BUFR training course
Langen
17-20 April 2007
Slide 16
PBIO routines
kerr - 0, successful
-1, file does not exist
-2, invalid file name
-3, invalid open mode specified
Slide 17
WMO BUFR training course
Langen
17-20 April 2007
Slide 17
PBIO routines
subroutine PBCLOSE (kunit, kret)
Input argument is an integer:
kunit - unit number from PBOPEN
Output argument is an integer:
kret – status return code
0 - OK
-1 - error in handling the file
Slide 18
WMO BUFR training course
Langen
17-20 April 2007
Slide 18
PBIO routines
subroutine PBWRITE( kunit, karray, kount, kerr)
Input arguments are integers:
kunit – unit number from PBOPEN
karray- an integer array holding bytes for write
kount – number of bytes to write to the file
Output arguments:
kret – status return code
>= 0 number of bytes written to theSlide
file
19
-1
Error in writing to the file
WMO BUFR training course
Langen
17-20 April 2007
Slide 19
PBIO routines
 Subroutine PBCREX( KUNIT, KARRAY, KINLEN,
KOUTLEN, KRET)
Input arguments:
KUNIT
- unit number returned by PBOPEN
KARRAY - integer array big enough to hold crex message
KINLEN - size in bytes of KARRAY
Output arguments:
KOUTLEN
- size in bytes of theSlide
CREX
product read
20
KRET
- 0 if OK
WMO BUFR training course
Langen
17-20 April 2007
Slide 20
PBIO routines
KRET
-1 if EOF is hit before CREX product is read
-2 if file read error is generated
-3 if size of KARRAY is not sufficient for the
CREX product
Slide 21
WMO BUFR training course
Langen
17-20 April 2007
Slide 21
CREX software FORTRAN routines
 CREX expansion
subroutine crexex( kbufl, ybuff, ksup, ksec0,ksec1,
ksec3, kelem, cnames, cunits,
kvals, values, cvals, kerr)
Input arguments:
kbufl
- length of bufr message in words
ybuff
- character string containing crex message
kelem - expected number of expanded elements
kvals - the size of values array in words
Slide 22
WMO BUFR training course
Langen
17-20 April 2007
Slide 22
CREX software FORTRAN routines
Output arguments:
ksup
- an array containing supplementary information
ksec0
- integer array of 3 words containing CREX
section 0 information
ksec1
- integer array of at least 40 words containing
CREX section 1 information
ksec3
- integer array of 4 words
cnames – character*64 array of kelem containing
element names
cunits
- character*24 array of kelem containing element
Slide 23
units
WMO BUFR training course
Langen
17-20 April 2007
Slide 23
CREX software FORTRAN routines
values
- real*8 array of kvals containing expanded data
cvals
- character*80 array of kvals containing code
table or CCITTIA5 CREX element entries
kerr
- return error code
Slide 24
WMO BUFR training course
Langen
17-20 April 2007
Slide 24
KSUP array of 9 words
Array
index
Word content
1
IDIM1, dimension of ksec1
2
Reserved
3
IDIM3, dimension of KSEC3
4
Reserved
5
M (number of elements in in values array, first index)
6
N (number of subsets)
7
JVC number of elements in CVALS array
8
Total length of the message in bytes
9
IDIM0, dimension of KSEC0 Slide 25
WMO BUFR training course
Langen
17-20 April 2007
Slide 25
KSEC0 array
Array
index
Word content
1
Length of section 0 in bytes
2
Total length of CREX message in bytes
3
Reserved
Slide 26
WMO BUFR training course
Langen
17-20 April 2007
Slide 26
KSEC1 array
Array
index
Word content
1
Reserved
2
CREX edition number
3
Originating centre (oooo)
4
Update sequence number (uu)
5
Number of subsets (sss)
6
CREX data category (nnn)
7
International sub-category (mmm)
8
Version number of local table used
9
Year (yyyy)
WMO BUFR training course
Langen
Slide 27
17-20 April 2007
Slide 27
KSEC1 array
Array
index
Word content
10
Month (mm)
11
Day (dd)
12
Hour (hh)
13
Minute (nn)
14
CREX master table (tt)
15
Version number of master table used (vv)
16
Originating sub-centre (ppp)
17
Bufr master table version number
18
Bufr version number of local tables
Slide 28 used
19-40
Reserved
WMO BUFR training course
Langen
17-20 April 2007
Slide 28
KSEC3 array
Array
index
Word content
1
Reserved
2
Reserved
3
Number of subsets
4
Reserved
Slide 29
WMO BUFR training course
Langen
17-20 April 2007
Slide 29
CREX software FORTRAN routines
 Values array is of type real*8
 If crex table B element is CCITTIA5, corresponding
element contain a real number, when truncated to an
integer represents
index*1000+length
where
index – subscript of the element in CVALS array where
character strings are stored
length – number of characters in the string
Slide 30
WMO BUFR training course
Langen
17-20 April 2007
Slide 30
CREX software FORTRAN routines
In the case of multi-subset data, one dimensional array
VALUES contains all subsets. Index to values array of
i-th element of observation is:
index=i+(nsub-1)*kelem
Slide 31
WMO BUFR training course
Langen
17-20 April 2007
Slide 31
CREX SOFTWARE fortran ROUTINES
 CREX encoding
subroutine crexen (ksec0, ksec1, ksec3, ktdlen,
crexktdlst, kdlen, kdata,
kelem, kvals, values, cvals, kbufl,
kerr)
Input arguments:
ksec0
- an integer array containing section 0 information
ksec1
- an integer array containing section 1 information
ksec3
- an integer array of size 4 containing number of
subsets
Slide 32
WMO BUFR training course
Langen
17-20 April 2007
Slide 32
CREX software FORTRAN routines
ktdlen - an integer, number of data descriptors to be
packed in section 1 of crex message
crexktdlst
- an integer array containing ktdlen descriptors
kdlen
- an integer, dimension of kdata array
kdata
- an integer array containing delayed replication
values
kelem - an integer containing expected number of
expanded elements
kvals
- an integer containing expected number of data
values
Slide 33
WMO BUFR training course
Langen
17-20 April 2007
Slide 33
CREX software FORTRAN routines
values - real*8 array of kvals words containing element
values
cvals
- character*80 array of kvals words containing
CCITTIA5 elements
Output arguments:
kbufl - an integer containing the length of bufr message
in words
kbuff - an integer array containing packed bufr message
kerr
- integer, status return code
Slide 34
WMO BUFR training course
Langen
17-20 April 2007
Slide 34
KSEC0 array
Array
index
Word content
1
Length of section 0 bytes
2
Total length of crex message
3
Reserved
Slide 35
WMO BUFR training course
Langen
17-20 April 2007
Slide 35
KSEC1 array
Array
index
Word content
1
Reserved
2
CREX edition number
3
Originating centre (oooo)
4
Update sequence number (uu)
5
Number of subsets (sss)
6
CREX data category (nnn)
7
International sub-category (mmm)
8
Version number of local table used
9
Year (yyyy)
WMO BUFR training course
Langen
Slide 36
17-20 April 2007
Slide 36
KSEC1 array
Array
index
Word content
10
Month (mm)
11
Day (dd)
12
Hour (hh)
13
Minute (nn)
14
CREX master table (tt)
15
Version number of master table used (vv)
16
Originating sub-centre (ppp)
17
Bufr master table version number
18
Bufr version number of local tables
Slide 37 used
19-40
Reserved
WMO BUFR training course
Langen
17-20 April 2007
Slide 37
KSEC3 array
Array
index
Word content
1
Reserved
2
Reserved
3
Number of subsets
4
Reserved
Slide 38
WMO BUFR training course
Langen
17-20 April 2007
Slide 38
CREX software FORTRAN routines
 Printing routines
subroutine CREXPRS0 (ksec0)
Input arguments:
ksec0 – an array containing section 0 information
Array
index
Word content
1
Length of section 0
2
Total length of crex message in bytes
3
Crex edition number
WMO BUFR training course
Langen
17-20 April 2007
Slide 39
Slide 39
CREX software FORTRAN routines
 subroutine crexprs1 ( ksec1,ksec3, ktdlen, ktdlst, ktdexl,
ktdexp, kelem, cnames)
Input arguments:
ksec1
- an integer array containing CREX section 1
ksec3
- an integer array containing number of subsets
ktdlen
- number of data descriptors in section 1
ktdlst
- an array containing data descriptors in section
1
ktdexl
- number of entries in the list of expanded data
Slide 40
descriptors
ktdexp
- an array containing expanded data descriptors
WMO BUFR training course
Langen
17-20 April 2007
Slide 40
CREX software FORTRAN routines
kelem
- expected number of expanded elements
cnames
- character*64 array of kelem containing CREX
section 1 information
KSEC1
Array
index
Word content
1
Reserved
2
CREX edition number
3
Originating centre (oooo)
4
Update sequence number (uu)
Slide 41
5
Number of subsets (sss)
WMO BUFR training course
Langen
17-20 April 2007
Slide 41
CREX software FORTAN routines
6
CREX data category (nnn)
7
International data sub-category (mmm)
8
Version number of local tables used
9
Year (yyyy)
10
Month (mm)
11
Day (dd)
12
Hour (hh)
13
Minute (nn)
14
CREX Master table (tt)
15
Version number of Master table used (vv)
16
Originating sub-centre (ppp)
17
Bufr master table version number
WMO BUFR training course
Langen
17-20 April 2007
Slide 42
Slide 42
CREX software FORTRAN routines
18
Bufr version number of local tables used
19-40 Reserved
Slide 43
WMO BUFR training course
Langen
17-20 April 2007
Slide 43
CREX software FORTRAN routines
ksec3
Array
index
Word content
1
Reserved
2
Reserved
3
Number of subsets
4
Reserved
Slide 44
WMO BUFR training course
Langen
17-20 April 2007
Slide 44
CREX software FORTRAN routines
 Subroutine crexsel2 (kbubset, kelem, ktdlen, ktdlst,
ktdexl, ktdexp, cnames, cunits,
kerr)
Input arguments:
ksubset
- subset number
kelem
- number of expected elements
Output arguments:
ktdlen
- number of data descriptors in section 1
Slide 45
ktdlst- list of data descriptors in section 1
WMO BUFR training course
Langen
17-20 April 2007
Slide 45
CREX software FORTRAN routines
ktdexl
- number of expanded data descriptors
ktdexp
- array containing expanded data descriptors
cnames
- array containing element names
cunits
- array containing element units
kerr
- return error code
Slide 46
WMO BUFR training course
Langen
17-20 April 2007
Slide 46
CREX software FORTRAN routines
 Subroutine crexprt ( k, ksub1, ksub2, kelem, cnames,
cunits, cvals, kvals, values, ksup,
ksec1, kerr)
Input arguments:
k
- switch to print with/without code tables
content
0 – no code table content, 1 – code table
content
ksub1 – starting subset
ksub2 – ending subset
kelem – dimension of cnames, cunits arrays
Slide 47
cnames- character*64 array containing element names
WMO BUFR training course
Langen
17-20 April 2007
Slide 47
CREX software FORTRAN routines
cunits – character*24 array containing element units
kvals – dimension of values array
values – real*8 array ( expanded data values)
ksup – an array containing supplementary information
ksec1 – an integer array of 40 containing section 1
Slide 48
WMO BUFR training course
Langen
17-20 April 2007
Slide 48
CREX software FORTRAN routines
KSUP
Array
index
Word content
1
IDIM1, dimension of ksec1
2
Reserved
3
IDIM3, dimension of ksec3
4
Reserved
5
M ( number of elements in values array, first index)
6
N (number of subsets)
7
JVC (number of elements in CVALS array)
8
Total CREX message length in bytes
9
IDIM0, dimension of ksec0
Slide 49
WMO BUFR training course
Langen
17-20 April 2007
Slide 49
CREX software FORTRAN routines
KSEC1
Array
index
Word content
1
Reserved
2
CREX edition number
3
Originating centre (oooo)
4
Update sequence number (uu)
5
Number of subsets (sss)
Slide 50
6
CREX data category (nnn)
WMO BUFR training course
Langen
17-20 April 2007
Slide 50
CREX software FORTRAN routines
Array
index
Word content
7
International sub-category (mmm)
8
Version number of local table used
9
Year (yyyy)
10
Month (mm)
11
Day (dd)
12
Hour (hh)
13
Minute (nn)
14
CREX Master table (tt)
15
Version number of master table used (vv)
16
Originating sub-centre (ppp)
17
Bufr master table number
WMO BUFR training course
Langen
17-20 April 2007
Slide 51
Slide 51
CREX software FORTRAN routines
Array
index
Word content
18
Bufr version of local tables used
19-40
Reserved
Slide 52
WMO BUFR training course
Langen
17-20 April 2007
Slide 52
CREX software FORTRAN routines
 CREX template design
Basic idea is to know one to one correspondence
between data descriptors expanded and the data values
To achieve that use:
subroutine CREXDES ( k, ksec1, ktdlen, crexktdlst,
kdlen, kdata, kelem, ktdexl,
crexktdexp, cnames, cunits, kerr)
Input arguments:
k
- an integer, 0 no print; 1 – print Slide 53
WMO BUFR training course
Langen
17-20 April 2007
Slide 53
CREX software FORTRAN routines
ksec1 - an integer array containing section 1 information.
Set the following words
ksec1(2) – Bufr edition number
ksec1(3) – Originating centre
ksec1(8) – Version number of local tables used
ksec1(15) – version number of master table used
ktdlen – number of data descriptors
crexktdlst- list of data descriptors in section 1
kdlen – dimension of kdata array
kdata - an integer array containing delayed
replications
Slide 54
kelem – expected number of expanded elements
WMO BUFR training course
Langen
17-20 April 2007
Slide 54
CREX software FORTRAN routines
Output arguments:
ktdexl – number of expanded elements
crexktdexp – integer array containing list of expanded
elements
cnames – character *64 array containing element names
cunits – character*24 array containing element units
kerr – status return code
Slide 55
WMO BUFR training course
Langen
17-20 April 2007
Slide 55
CREX software FORTRAN routines
 Program to use for template design
…/crex_000320/example/crextdexp.F
 Conversion CREX to BUFR
…/crex_000320/example/crex2bufr.F
 Conversion BUFR to CREX
…/crex_000320/example/bufr2crex.F
Slide 56
WMO BUFR training course
Langen
17-20 April 2007
Slide 56
CREX software and pre-processing
 What happens when crex data reach DPC
 How to organize observation
 Pre-process observations
- unpacking
- re-organization of data
- possible quality control
- grouping of observations which belong to
particular analysis cycle
- creating RDB key for further more efficient processing
- packing the data into BUFR format
Slide 57
- writing data into RDB
WMO BUFR training course
Langen
17-20 April 2007
Slide 57
CREX software and pre-processing
 Alternatively
- unpack crex message
- write unpacked values into DB
- make decisions about archiving
Slide 58
WMO BUFR training course
Langen
17-20 April 2007
Slide 58