8. Retrieve high-resolution nested ECMWF fields for FLEXPART

From mn/geo/geoit
Revision as of 15:13, 18 February 2013 by Nik (talk | contribs)

Jump to: navigation, search

Back to ECMWF overview[1]


Updated by nik: 18 Feb. 2013


The retrieval procedure uses a number of fortran scripts which are run on MARS. In short:

- Retrieves the meteorological model data output from ECMWF

- De-accumulates atmospheric flux data (FLXACC2 fortran code under source.tar)

- Calculates the vertical velocity from the continuity equation (CONVERT2 fortran code under source.tar)

- Prepares the met.files (ENyyyymmddhh) that can be used for FLEXPART.


More details are given in the documents on the webpage:

http://transport.nilu.no/flexpart
under Routines for retrieval of FLEXTRA/FLEXPART input data from ECMWF


LIST OF PROCEDURES TO FOLLOW TO RETRIEVE DATA:


Retrieving high resolution data (0.25 degree or finer) should be performed on the supercomputer HPCE due to the computational demand and accuracy dependency in the vertical velocity calculations.



1. Log in to the HPCE (c2a) and copy the following directory and files to your home directory:

cp -r /home/ms/sbc/flex_extract_ecgate_V4/ YourDestination


List of files needed:

flex_ecmwf_91_finegrid_*
extract_ecmwf_ftpbase 
seq_extract.V4* 
seq_control.V4*
submit* 
source.tar (contains Fortran90 source files and makefiles for the data processing to FLEXPART grid etc)


2. Edit the times in the file: seq_control.V4*

./seq_extract.V4 1990 1990 06 06 29 29 1

This extract the files from 1990 06 29 to 1990 06 29 - for 1 day!


3. In file flex_ecmwf_91_finegrid_* modify the italic parameters:

(for an overview over the parameters and what they mean - see bottom of page)

-----------------------------------
BEGIN: modification of config file
-----------------------------------
cat <<EOF >CONTROL_ERA
DAY1 YYYYMMDD1
DAY2 YYYYMMDD2
DTIME 3
M_GRID 20 (resolution 0.2 deg, for resolution of e.g. 10 deg use 1000)
M_LEFT -3600 (36 W)
M_LOWER 4000 (40 N)
M_UPPER 7600 (76 N)
M_RIGHT 3600 (36 E)  
M_LEVEL 91
M_RESOL 799
M_GAUSS 1
M_ACCURACY 24
M_OMEGA 0
M_OMEGADIFF 0
M_ETA 0
M_ETADIFF 0
M_SMOOTH 0
M_FORMAT GRIB2 (GRIB1 or GRIB2)
GATEWAY ecaccess.ecmwf.int
DESTINATION nina (or another MS association set up on the web interface, e.g flexpart)
ECSTORAGE 1
ECTRANS 1
ECFSDIR ectmp:/${USER}/GRIM/ (directory name)
MAILOPS ${USER}
MAILFAIL ${USER}
EXEDIR .
SOURCECODE /home/ms/no/sb9/flex_extract_ecgate_V4 (your username)
EOF 
-----------------------------------
- END: modification of config file
-----------------------------------


Notice:

M_GRID/90 must be a whole number

M_LOWER/M_GRID must be a whole number, same with the three other coordinates

(M_UPPER-M_LOWER) / M_GRID must equal a whole number

(M_LEFT-M_RIGHT) / M_GRID must equal a whole number


4. In the scripts, reference to the right ECMWF userID must be made.

Use the command

grep $USER *

to list all lines in the files containing a reference to the ECMWF userId and that should be set to your own user.

You will need to change in: submit, seq_control.V4, seq_extract.V4 and flex_ecmwf_91_global_1.0_ecgate


5. Copy the template file flex_ecmwf_91_finegrid_* to extract_ecmwf_ftpbase

cp flex_ecmwf_91_finegrid_* extract_ecmwf_ftpbase


6. Run the job:

./submit


7. Check your job:

llq         to see your job in the queue 
llq -u sb9  to see only jobs for user sb9
eoj <jobid> provides details on the performance of a running job. (also automatically provided at end of job)


You can also follow your execution on the Web access facilities on ECMWF

You will receive an email notifying the start of the script and the successful completion of the script.

You will also get a file finished.* in your directory indicating that the script successfully completed.

The scripts can take from a few minutes up to a few hours to complete, depending on the load of the MARS system.


8. Possible error messages:

LoadL_starter: The program, extract_ecmwf_ftp01.5046746, exited normally and returned an exit code of 1.

In the file *.out search for "exit" to find the line where exit code 1 is given. Here might be hints to find the error message.


9. Get your files:

The retrieved files are automatically transferred to your ecgate account (via ECTRANS) and then to NILU by ftp. (NB: cannot ftp directly from HPCE, but must go via ecgate).

If you can’t find your files in your defined NILU-directory, check the directory you specified in flex_ecmwf_91_finegrid_* (ECFSDIR) for where it is temporarily stored on the ECWMF system. Then try ftp it over again using the ftp commands specified previously.



OVERVIEW OVER PARAMETERS THAT MAY BE CHANGED BY THE USER (default values in brackets):

FORCE_E40  - (0)  -  Retrieve data from class E4 (ERA-40)
FORCE_EI   - (0)  -  Retrieve data from class EI (ERA-Interim)
FORCE_EPS  - (0)  -  Retrieve data from ensemble prediction system
M_STREAM   -(OPER)-  Allows to retrieve data from IFS experiment
M_EXPVER   - (1)  -  Experiment number, necessary for ERA-Interim
M_NUMBER   - (1)  -  Forecast leg, necessary for EPS. 1=T399 forecasts, 2=T255
M_GRID     -(100) -  Horizontal resolution of Lat/Lon grid in 1/100 degree
M_RESOL    -(159) -  Horizontal resolution of spectral fields
M_LEVEL    - (91) -  Number of vertical levels
M_LEVELIST -(ALL) -  List of vertical levels for MARS request
M_ADDPAR          -  Additional surface parameters  (/27/28/173/186/187/188/235/139/39)
M_UPPER           -  Latitude of upper right corner of grid area to be retrieved in Hundredth of a degree (e.g. 3750 for 37.5N)
M _LOWER          -  Latitude of lower left corner of grid area in 1/100 degree
M_LEFT            -  Longitude of lower left corner of grid area in 1/100 degree
M_RIGHT           -  Longitude of upper right corner of grid area in1/100 degree.
M_ACCURACY - (24) -  Accuracy of GRIB data in bits
M_GAUSS    -  (1) -  Calculation of etadot on Lat/Lon grid (0) or on Gaussian grid (1)
M_OMEGA    -  (0) -  Retrieve Omega from MARS and put it to file OMEGAyymmddhh
M_OMEGADIFF-  (0) -  Calculate Omega and Dps/Dt from continuity equation for diagnostic purposes and include it in file OMEGAyymmddhh
M_SMOOTH   -  (0) -  Spectral truncation of etadot after calculation on Gaussian grid
GATEWAY           -  Name of ECaccess gateway server
DESTINATION       -  Name of destination at ECaccess gateway server
ECSTORAGE  -  (1) -  Store EN-files in ECFS file system 0/1 (1)
ECTRANS    -  (1) -  Transfer EN-files to gateway server 0/1 (1)  
ECFSDIR           -  Destination directory on ECFS file system
MAILOPS           -  List of email addresses for operational log-files
MAILFAIL          -  List of email addresses for failure log-files
EXEDIR     - (./) -  Location of FORTRAN executables
SOURCECODE        -  Location of file source.tar containing source code and makefiles ($HOME/flex_extract_ecgate_V2)