Difference between revisions of "11. File systems and data structure at ECMWF - where to store and get data!"
From mn/geo/geoit
(Created page with "'''ECgate is a Unix server (soon to be changed to a Linux server) where you have home directories:''' - HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To chec...") |
|||
Line 1: | Line 1: | ||
'''ECgate is a Unix server (soon to be changed to a Linux server) where you have home directories:''' | '''ECgate is a Unix server (soon to be changed to a Linux server) where you have home directories:''' | ||
− | - HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To check quota: ecquota | + | |
− | - SCRATCH: Temporary data, removed regularly. Quota: 100 GB | + | - HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To check quota: ecquota |
− | - SCRATCHDIR: Files are autodeleted after end of job. Part of SCRATCH and quota. | + | - SCRATCH: Temporary data, removed regularly. Quota: 100 GB |
+ | - SCRATCHDIR: Files are autodeleted after end of job. Part of SCRATCH and quota. | ||
'''HPCF (High performance computing facility) is the supercomputer (c2a) where you have home directories:''' | '''HPCF (High performance computing facility) is the supercomputer (c2a) where you have home directories:''' | ||
+ | |||
- HOME: Small permanent files, daily backup/snapshot. Quota: 320 MB. To check quota: quota | - HOME: Small permanent files, daily backup/snapshot. Quota: 320 MB. To check quota: quota | ||
− | - SCRATCH: Temporary data, removed regularly. Quota: 25 TB. | + | - SCRATCH: Temporary data, removed regularly. Quota: 25 TB. |
− | - SCRATCHDIR: Autodeled after end of job. | + | - SCRATCHDIR: Autodeled after end of job. |
− | - PERM: Large paermanent files, no backup. Quota: 26 GB. Accessible from ECgate as /c2a/perm | + | - PERM: Large paermanent files, no backup. Quota: 26 GB. Accessible from ECgate as /c2a/perm |
The two systems use the same name convension - but different file systems. | The two systems use the same name convension - but different file systems. |
Revision as of 18:11, 11 March 2013
ECgate is a Unix server (soon to be changed to a Linux server) where you have home directories:
- HOME: Small permanent files, daily backup/snapshot. Quota: 3.6 GB. To check quota: ecquota - SCRATCH: Temporary data, removed regularly. Quota: 100 GB - SCRATCHDIR: Files are autodeleted after end of job. Part of SCRATCH and quota.
HPCF (High performance computing facility) is the supercomputer (c2a) where you have home directories:
- HOME: Small permanent files, daily backup/snapshot. Quota: 320 MB. To check quota: quota
- SCRATCH: Temporary data, removed regularly. Quota: 25 TB. - SCRATCHDIR: Autodeled after end of job. - PERM: Large paermanent files, no backup. Quota: 26 GB. Accessible from ECgate as /c2a/perm
The two systems use the same name convension - but different file systems.
On both ECgate and HPCF you can access 2 different archives.
1) MARS which contains the meteorological data.
2) ECFS (ECMWF File storage system): This is a User Archive where you can store long-term data on tape.
On ECFS you have two domains: - ec: Permanent domain where files are stored indefinitively. - ectmp: temporary domain where files are stored for 90 days after which they are automatically deleted (and cannot be recovered).
To store/access fiels in ECFS you must issue explicit commands which is set-up to use a UNIX like interface. Commands are same as for unix but starting with "e": els, ecd, ecat.. For a manual of the commands type "man els".
els lists your data under ec: ls lists your data under $HOME
ECFS can be accessed both from ECgate and HPCF.