Difference between revisions of "How to read GRIB files!"

From mn/geo/geoit
Jump to: navigation, search
m (COMMAND-LINE GRIB TOOLS)
 
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Back to ECMWF overview[http://muspelheim.nilu.no/w/index.php/ECMWF]
+
Back to [[ECMWF]] overview
  
 +
''Updated by jfb: 15 March 2014''
  
''Updated by nik: 13 March 2013''
+
<br/>'''GRIB'''
 
 
 
 
'''GRIB'''
 
  
 
'''GRI'''dded '''B'''inary is a compressed format for meterological data used since 1985.
 
'''GRI'''dded '''B'''inary is a compressed format for meterological data used since 1985.
Line 20: Line 18:
  
 
One GRIB1 file may include many messages (blocks of sections).
 
One GRIB1 file may include many messages (blocks of sections).
 +
  
  
Line 25: Line 24:
  
 
== GRIB2 ==
 
== GRIB2 ==
was introduced for ECMWF operational ''model level'' data on '''18 May 2011'''.  
+
 
 +
was introduced for ECMWF operational ''model level'' data on '''18 May 2011'''.
  
 
Migration from GRIB1 to GRIB2 was necessary because GRIB1 data could only hold 126 model levels.
 
Migration from GRIB1 to GRIB2 was necessary because GRIB1 data could only hold 126 model levels.
Line 31: Line 31:
 
Number of model levels in the high resolution forecast will be increased from 91 to 137 in June 2013.
 
Number of model levels in the high resolution forecast will be increased from 91 to 137 in June 2013.
  
Pressure levels will still have GRIB1 but eventually this will also be changed.  
+
Pressure levels will still have GRIB1 but eventually this will also be changed.
  
 
"Old" model data will still have GRIB1 format in the archive.
 
"Old" model data will still have GRIB1 format in the archive.
  
GRIB2 also allows more flexibility and is more complex than GRIB1.  
+
GRIB2 also allows more flexibility and is more complex than GRIB1.
  
 
For example higher precision of variables.
 
For example higher precision of variables.
Line 42: Line 42:
 
  GRIB2: Lon/lat is in micro-degree
 
  GRIB2: Lon/lat is in micro-degree
  
 +
<br/>GRIB2 data are stored in 8 ''sections'' where section 7 includes the compressed data and the other sections are data specifications.
  
GRIB2 data are stored in 8 ''sections'' where section 7 includes the compressed data and the other sections are data specifications.
 
 
Thus, the structure of GRIB1 and GRIB2 messages is different, the order of the sections are different and there are additional sections for GRIB2.
 
Thus, the structure of GRIB1 and GRIB2 messages is different, the order of the sections are different and there are additional sections for GRIB2.
  
 +
== GRIB-decoding ==
  
 
== GRIB-decoding ==
 
 
 
To decode GRIB data there are software available from ECMWF.
 
To decode GRIB data there are software available from ECMWF.
  
Line 56: Line 54:
 
This software hides the binary layer of the message and uses a key/value approach to access the information in a GRIB message.
 
This software hides the binary layer of the message and uses a key/value approach to access the information in a GRIB message.
  
GRIB1 and GRIB2 uses the '''same''' function call.  
+
GRIB1 and GRIB2 uses the '''same''' function call.
  
 
'''Many keys to identify a parameter are edition independent, use these where possible!'''
 
'''Many keys to identify a parameter are edition independent, use these where possible!'''
  
 +
<br/>'''GRIB_API''' can be used either through
  
'''GRIB_API''' can be used either through
 
 
  Command line tools
 
  Command line tools
 
  Fortran90/C/Python interfaces
 
  Fortran90/C/Python interfaces
  
  
== COMMAND-LINE GRIB TOOLS==
 
  
 +
== COMMAND-LINE GRIB TOOLS ==
  
 
General syntax for use of the grib tools
 
General syntax for use of the grib tools
  
''' grib_<tool> [options] grib_file1 grib_file2... [output file]'''
+
'''grib_&lt;tool&gt; [options] grib_file1 grib_file2... [output file]'''
  
 +
<br/>Manuals for each tool are available by running the tool without any options or input file.
  
Manuals for each tool are available by running the tool without any options or input file.
+
<br/>'''Tools to inspect the content of and compare GRIB messages'''
  
 +
'''grib_ls'''            summary of the content of a GRIB file. (Do not use in scripts - only interactive mode!)
 +
                   
 +
                    A default list of keys in printed without using options:
 +
                    edition, centre, typeOfLevel, level, dataDate, stepRange, dataType, shortName, packingType, gridType
 +
                   
 +
                    Useful options
 +
                                      -p        keys to print (other than the default list)
 +
                                      -P        additional keys to print (additional to the default list)
 +
                                      -m        print MARS keys. Very useful to see the keys as they would be specified in a MARS request!
 +
                                                E.g. domain, levtype, levelist, date, time, step, param, class, type, stream, expver
 +
                                      -w        "where", prints only the content for specified key=value. (Can be used with all grib tools).
 +
                                      -l        Find the lat/long value in the model close to a given lat/long
 +
                    Example:
 +
                    '''grib_ls -w shortName=t,level=1 /flex_wrk/WIND_FIELDS/ECMWF/GLOBAL/OPER_fields/2013/03/EN13030100'''
 +
                    prints the information about the temperature field at level 1:
 +
                    edition      centre      date        dataType    gridType    stepRange    typeOfLevel  level        shortName    packingType
 +
                    2            ecmf        20130301    an          regular_ll  0            hybrid      1            t            grid_jpeg
 +
                   
 +
                    '''grib_ls -w shortName=t,level=1 /flex_wrk/WIND_FIELDS/ECMWF/GLOBAL/OPER_fields/2013/03/EN13030103'''
 +
                    prints the information about the temperature field at level 1:
 +
                    edition      centre      date        dataType    gridType    stepRange    typeOfLevel  level        shortName    packingType
 +
                    2            ecmf        20130301    fc          regular_ll  3            hybrid      1            t            grid_jpeg
 +
                   
 +
                    You can see that this is GRIB edition 2, and data for midnight (00) is analysis data while 03 is forecast data 3 hours from midnight (stepRange).
 +
                    The data is on a regular lat-long grid.
  
'''Tools to inspect the content of and compare GRIB messages'''
 
  
'''grib_ls'''            summary of the content of a GRIB file. (Do not use in scripts - only interactive mode!)
 
                   
 
                    A default list of keys in printed without using options:
 
                    edition, centre, typeOfLevel, level, dataDate, stepRange, dataType, shortName, packingType, gridType
 
                   
 
                    Useful options
 
                                      -p        keys to print (other than the default list)
 
                                      -P        additional keys to print (additional to the default list)
 
                                      -m        print MARS keys. Very useful to see the keys as they would be specified in a MARS request!
 
                                                E.g. domain, levtype, levelist, date, time, step, param, class, type, stream, expver
 
                                      -w        "where", prints only the content for specified key=value
 
                                      -l        Find the lat/long value in the model close to a given lat/long
 
                    Example:
 
                    grib_ls -m -w shortName=t,level=1 EN13030100
 
                    prints the information
 
                    domain      date        time        expver      class      type        stream      step        levtype    levelist    param
 
                    g          20130301    0000        0001        od          an          oper        0          ml          1          130
 
  
  
  
 
  '''grib_dump'''          provides a more detailed view and listing of the file.  
 
  '''grib_dump'''          provides a more detailed view and listing of the file.  
                   
+
                   
                    Useful options
+
                    Useful options
                                      -O to show the content in WMO style (easier to read)
+
                                      -O to show the content in WMO style (easier to read)
 +
                                      -w        "where", prints only the content for specified key=value
 +
 
 
                     Example:
 
                     Example:
 +
                    '''grib_dump -O -w shortName=t,level=1 /flex_wrk/WIND_FIELDS/ECMWF/GLOBAL/OPER_fields/2013/03/EN13030103'''
 +
  
                   
 
  
 
  '''grib_get'''
 
  '''grib_get'''
Line 111: Line 120:
 
  '''grib_get_data'''
 
  '''grib_get_data'''
  
 
+
<br/>'''Tools for counting and copying some messages'''
'''Tools for counting and copying some messages'''
 
  
 
  '''grib_copy'''
 
  '''grib_copy'''
Line 118: Line 126:
 
  '''grib_count'''
 
  '''grib_count'''
  
 
+
<br/>'''Tools for making changes to the content of a GRIB message'''
'''Tools for making changes to the content of a GRIB message'''
 
  
 
  '''grib_set'''
 
  '''grib_set'''
Line 125: Line 132:
 
  '''grib_filter'''
 
  '''grib_filter'''
  
== FORTRAN GRIB TOOLS==
+
== FORTRAN GRIB TOOLS ==
 +
 
 +
Grib API
 +
 
 +
[[Category:Tools]][[Category:Models]][[Category:ECMWF]]

Latest revision as of 16:23, 30 January 2015

Back to ECMWF overview

Updated by jfb: 15 March 2014


GRIB

GRIdded Binary is a compressed format for meterological data used since 1985.

Requires software for encoding and decoding.


GRIB1

Currently used for ECMWF operational surface and pressure level data.

GRIB1 data is stored in 5 sections where section 4 includes the compressed data and the other sections are data specifications.

One GRIB1 file may include many messages (blocks of sections).



GRIB2

was introduced for ECMWF operational model level data on 18 May 2011.

Migration from GRIB1 to GRIB2 was necessary because GRIB1 data could only hold 126 model levels.

Number of model levels in the high resolution forecast will be increased from 91 to 137 in June 2013.

Pressure levels will still have GRIB1 but eventually this will also be changed.

"Old" model data will still have GRIB1 format in the archive.

GRIB2 also allows more flexibility and is more complex than GRIB1.

For example higher precision of variables.

GRIB1: Lon/lat is in milli-degree
GRIB2: Lon/lat is in micro-degree


GRIB2 data are stored in 8 sections where section 7 includes the compressed data and the other sections are data specifications.

Thus, the structure of GRIB1 and GRIB2 messages is different, the order of the sections are different and there are additional sections for GRIB2.

GRIB-decoding

To decode GRIB data there are software available from ECMWF.

This software is the GRIB_API (Application Programming Interface) library.

This software hides the binary layer of the message and uses a key/value approach to access the information in a GRIB message.

GRIB1 and GRIB2 uses the same function call.

Many keys to identify a parameter are edition independent, use these where possible!


GRIB_API can be used either through

Command line tools
Fortran90/C/Python interfaces


COMMAND-LINE GRIB TOOLS

General syntax for use of the grib tools

grib_<tool> [options] grib_file1 grib_file2... [output file]


Manuals for each tool are available by running the tool without any options or input file.


Tools to inspect the content of and compare GRIB messages

grib_ls             summary of the content of a GRIB file. (Do not use in scripts - only interactive mode!)
                   
                   A default list of keys in printed without using options: 
                   edition, centre, typeOfLevel, level, dataDate, stepRange, dataType, shortName, packingType, gridType
                   
                   Useful options
                                     -p        keys to print (other than the default list)
                                     -P        additional keys to print (additional to the default list)
                                     -m        print MARS keys. Very useful to see the keys as they would be specified in a MARS request!
                                               E.g. domain, levtype, levelist, date, time, step, param, class, type, stream, expver
                                     -w        "where", prints only the content for specified key=value. (Can be used with all grib tools).
                                     -l        Find the lat/long value in the model close to a given lat/long
                   Example:
                   grib_ls -w shortName=t,level=1 /flex_wrk/WIND_FIELDS/ECMWF/GLOBAL/OPER_fields/2013/03/EN13030100
                   prints the information about the temperature field at level 1: 
                   edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType
                   2            ecmf         20130301     an           regular_ll   0            hybrid       1            t            grid_jpeg
                   
                   grib_ls -w shortName=t,level=1 /flex_wrk/WIND_FIELDS/ECMWF/GLOBAL/OPER_fields/2013/03/EN13030103
                   prints the information about the temperature field at level 1: 
                   edition      centre       date         dataType     gridType     stepRange    typeOfLevel  level        shortName    packingType
                   2            ecmf         20130301     fc           regular_ll   3            hybrid       1            t            grid_jpeg
                   
                   You can see that this is GRIB edition 2, and data for midnight (00) is analysis data while 03 is forecast data 3 hours from midnight (stepRange).
                   The data is on a regular lat-long grid.



grib_dump           provides a more detailed view and listing of the file. 
                   
                   Useful options
                                     -O to show the content in WMO style (easier to read)
                                     -w        "where", prints only the content for specified key=value
                    Example:
                   grib_dump -O -w shortName=t,level=1 /flex_wrk/WIND_FIELDS/ECMWF/GLOBAL/OPER_fields/2013/03/EN13030103


grib_get

grib_get_data


Tools for counting and copying some messages

grib_copy
grib_count


Tools for making changes to the content of a GRIB message

grib_set
grib_filter

FORTRAN GRIB TOOLS

Grib API