Difference between revisions of "11. File systems and data structure at ECMWF - where to store and get data!"

From mn/geo/geoit
Jump to: navigation, search
Line 1: Line 1:
'''ECgate is a Unix server (soon to be changed to a Linux server) where you have home directories:'''
+
'''ECgate'''
 +
'''..is a Unix server (soon to be changed to a Linux server) where you have home directories:'''
  
 
  - HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To check quota: ecquota
 
  - HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To check quota: ecquota
Line 5: Line 6:
 
  - SCRATCHDIR: Files are autodeleted after end of job. Part of SCRATCH and quota.
 
  - SCRATCHDIR: Files are autodeleted after end of job. Part of SCRATCH and quota.
  
'''HPCF (High performance computing facility) is the supercomputer (c2a) where you have home directories:'''
+
 
 +
'''HPCF'''
 +
''' (High performance computing facility) is the supercomputer (c2a) where you have home directories:'''
 
   
 
   
 
- HOME: Small permanent files, daily backup/snapshot. Quota: 320 MB. To check quota: quota
 
- HOME: Small permanent files, daily backup/snapshot. Quota: 320 MB. To check quota: quota

Revision as of 18:12, 11 March 2013

ECgate ..is a Unix server (soon to be changed to a Linux server) where you have home directories:

- HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To check quota: ecquota
- SCRATCH: Temporary data, removed regularly. Quota: 100 GB
- SCRATCHDIR: Files are autodeleted after end of job. Part of SCRATCH and quota.


HPCF (High performance computing facility) is the supercomputer (c2a) where you have home directories:

- HOME: Small permanent files, daily backup/snapshot. Quota: 320 MB. To check quota: quota

- SCRATCH: Temporary data, removed regularly. Quota: 25 TB.
- SCRATCHDIR: Autodeled after end of job. 
- PERM: Large paermanent files, no backup. Quota: 26 GB. Accessible from ECgate as /c2a/perm

The two systems use the same name convension - but different file systems.


On both ECgate and HPCF you can access 2 different archives.

1) MARS which contains the meteorological data.

2) ECFS (ECMWF File storage system): This is a User Archive where you can store long-term data on tape.

  On ECFS you have two domains:
  - ec: Permanent domain where files are stored indefinitively.
  - ectmp: temporary domain where files are stored for 90 days after which they are automatically deleted (and cannot be recovered).
  To store/access fiels in ECFS you must issue explicit commands which is set-up to use a UNIX like interface. Commands are same as for unix but starting with "e":
  els, ecd, ecat.. 
  For a manual of the commands type "man els".
  els lists your data under ec:
  ls lists your data under $HOME
  ECFS can be accessed both from ECgate and HPCF.