Retrieve large data sets from MARS with ecmwfapi
ecmwfapi is a python interface which allows you to retrieve data from ECMWF directly on your computer (no need to be run at ECMWF). To use it you need:
- to set-up your environment for python
- to install your ECMWFAPI key
Contents
The easiest to set-up your environment is to "load" flexpart modulefile:
module load flexpart
Note: if you are willing to run WRF instead, you should load wrf instead of flexpart.
To install your API key:
To access ECMWF you will need an API key. For that you first need to login at https://apps.ecmwf.int/auth/login/ and then retrieve you key at https://api.ecmwf.int/v1/key/. For this, you will need to have an account on ECMWF web site. If you don't have an account, please self register at https://apps.ecmwf.int/registration/
Copy the information in this page and paste it in the file $HOME/.ecmwfapirc.
{
"key"
:
"XXXXXXXXXXXXXXXXXXXXXX"
,
"email"
:
"john.smith@example.com"
}
If you need to retrieve data to run your model on abel,
you should copy this file on abel too.Example
#!/usr/bin/env python # # (C) Copyright 2012-2013 ECMWF. # # This software is licensed under the terms of the Apache Licence Version 2.0 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. # In applying this licence, ECMWF does not waive the privileges and immunities # granted to it by virtue of its status as an intergovernmental organisation nor # does it submit to any jurisdiction. # from ecmwfapi import ECMWFDataServer # To run this example, you need an API key # available from https://api.ecmwf.int/v1/key/ server = ECMWFDataServer() current_date=20120801 server.retrieve({ 'dataset' : "interim", 'date' : "%s"%(current_date), 'time' : "00", 'step' : "0", 'stream' : "oper", 'levtype' : "sfc", 'levelist' : "all", 'type' : "an", 'class' : "ei", 'grid' : "128", 'param' : "148", 'target' : "ERA_148_%s.grb"%(current_date), })
Specifying class is not enough and you also need to set "dataset".
Choosing a dataset
The "dataset
" parameter is one of:
Dataset | Description | Licence |
---|---|---|
era15 | ECMWF Global Reanalysis Data - ERA-15 (Jan 1979 - Dec 1993) | general |
era20cmv0 | ERA-20CM: Ensemble of climate model integrations (Experimental version) | general |
era40 | ECMWF Global Reanalysis Data - ERA-40 (Sep 1957 - Aug 2002) | general |
eraclim | ERA-20CM: Ensemble of climate model integrations | general |
icoads | ICOADS v2.5.1 with interpolated 20CR feedback | general |
interim | ECMWF Global Reanalysis Data - ERA Interim (Jan 1979 - present) | general |
ispd | ISPD v2.2 | general |
macc | MACC | macc |
macc_ghg_inversions | N/A | macc_ghg_inversions |
macc_nrealtime | MACC Near Real-time | macc_nrealtime |
tigge | TIGGE (THORPEX Interactive Grand Global Ensemble) | tigge |
yotc | YOTC (Year of Tropical Convection) | general |
To access these dataset, you need to agree on the the corresponding terms and conditions that can be found under the "Licence" link in the table above. See http://apps.ecmwf.int/datasets/ for the content of the datasets. The other parameters are described at: http://www.ecmwf.int/publications/manuals/mars/guide/index.html