Difference between revisions of "Getting started"
m |
m |
||
Line 227: | Line 227: | ||
<code> /cluster/work/jobs/[slurm-id] </code> | <code> /cluster/work/jobs/[slurm-id] </code> | ||
+ | |||
+ | '''Example job script''' | ||
#!/bin/bash | #!/bin/bash | ||
Revision as of 09:35, 29 September 2021
Contents
Getting started
User access
Go to www.metacenter.no > User Login (Passwords) > 1. Request a user account > HPC
Fill in the form and choose the appropriate project. It is recommended to use your UiO username.
The project manager can give you access to the VASP license through the VASP portal and thereafter informing sigma2@uninett.no to give your user the necessary group affiliations (xvasp and vasp5).
Project | Manager | Group | Resources |
---|---|---|---|
NN4604K | Jonathan M. Polfus | Electrochemistry | Saga |
NN9136K | Lasse Vines | LENS | Saga |
NN9180K | Clas Persson | Structure Physics | Saga |
Login
ssh and scp
ssh can be used to login to the cluster with a terminal app on UNIX, macOS or Windows Subsystem for Linux
ssh [username]@saga.sigma2.no
Hostnames and usernames can be saved by creating or editing the config file using for instance the vim text editor
vim ~/.ssh/config
Press i
to insert text
Host saga HostName = saga.sigma2.no user [username]
Press escape
to exit insert mode, and then :wq
to 'write' and 'quit'.
Login is then simplified
ssh saga
Copy files from cluster to current folder (.
)
scp saga:[path]/[filename] .
Copy files to the home directory on the cluster
scp [filename] saga:
PuTTY and WinSCP
PuTTY is an ssh client for Windows that can be downloaded from www.putty.org.
WinSCP is an scp client for Windows that can be downloaded from winscp.net.
Working in UNIX
Basic commands
Learn to use the TAB key to autocomplete commands, paths and filenames.
Learn to view the manual for any command: man [command]
Command | Description |
---|---|
ls
|
List files and directories in current directory |
mkdir [directoryname]
|
Make a directory |
cd [directoryname]
|
Change current path to another directory |
pwd
|
Display current path |
cd ..
|
Change path to parent directory |
cp [path]/[filename] .
|
Copy file to current directory |
cp [filename] [directoryname]
|
Copy file to another directory |
mv [filename] [path]
|
Move file to another directory |
mv [filename] [newfilename]
|
Change the name of a file |
cp -r [directory] [path]
|
Copy directory and its contents to new location |
rm [filename]
|
Delete file |
rm -r [directoryname]
|
Delete directory and its contents |
Symbol | Description |
---|---|
.
|
Current directory |
..
|
Parent directory |
~
|
Home directory |
/
|
Root directory |
*
|
Any text pattern |
?
|
Any symbol |
Command | Description |
---|---|
less
|
View text file. View live updates to end of file: Shift-F
|
head
|
Display top 10 lines of text file. Specify number of lines: -n [count]
|
tail
|
Display bottom 10 lines of text file. Specify number of lines: -n [count]
|
grep [pattern]
|
Display each line in text file containing text specific text/symbol pattern. |
diff [file1] [file2] | Compare files line by line and display the difference between them. |
cat
|
Concatenate text files sequentially. Write the output to a file: > [filename]
|
Filesystem on Saga
Home directory: mainly for personal files, scripts etc. due to limited storage space
/cluster/home/[username]
Project folder: Run jobs and store output files in a subfolder called your username
/cluster/projects/[project number]/[username]
Check disk usage and storage quota for home directory and project folder
dusage
Check disk usage of current folder and subfolders
du -hs
Find and remove files matching [filename] in current folder and subfolders
find . -name "[filename]" -exec rm {} \;
Job script
Sigma2 NRIS HPC clusters use Slurm as workload manager and job scheduler. Documentation is available on documentation.sigma2.no/
Computations are run by submitting a job script to the queue
sbatch jobfile
View the queue for a user or project
squeue -u [username]
squeue -p [project number]
Job scripts for Saga are available in the project folder
/cluster/projects/nn4604k/jobfile
The job script below contains several SBATCH options
Saga has 40 CPUs per node and it is recommended to use all CPUs on a node and a 1-2 nodes for optimal performance documentation.sigma2.no/hpc_machines/saga
The module system is restored to avoid potential conflicts. There exist several compiled versions of VASP and specific versions are compiled for increased performance with gamma K-point grid
module restore system module load VASP/5.4.4-intel-2019a-std #module load VASP/5.4.4-intel-2019a-gam
Available modules on the cluster can be listed with
module avail
Files are copied to a temporary job folder ($SCRATCH). The WAVECAR file is copied if it is available.
cp [filenames] $SCRATCH if [ -f WAVECAR ] then cp WAVECAR $SCRATCH fi
Output files are copied from the job folder to the submit folder when job completes. Savefile will also copy files if the job crashes.
savefile [filenames]
The job is started from the $SCRATCH folder and the elapsed time is recorded to slurm output
cd $SCRATCH time srun vasp
The progression of the job can be viewed in the slurm output file slurm-[slurm-id] in the submit folder once the job is running.
The working folder can be accessed while the job is running
/cluster/work/jobs/[slurm-id]
Example job script
#!/bin/bash #SBATCH --job-name=[jobname] #SBATCH --account=[project number] #SBATCH --time=10:00:00 #SBATCH --mem-per-cpu=4600M #SBATCH --nodes=2 #SBATCH --ntasks-per-node=40 set -o errexit set -o nounset module restore system module load VASP/5.4.4-intel-2019a-std #module load VASP/5.4.4-intel-2019a-gam cp INCAR POSCAR POTCAR KPOINTS $SCRATCH if [ -f WAVECAR ] then cp WAVECAR $SCRATCH fi savefile CONTCAR OUTCAR DOSCAR CHGCAR WAVECAR cd $SCRATCH time srun vasp
Aliases and settings
Define useful shortcuts by adding aliases to ~/.bash_profile
alias l='ls -l' alias la='ls -l -a' alias ..='cd ..' alias nn='cd /cluster/projects/[project number]' alias q='squeue -u [username]'
You can also add commands such as automatically changing directory to the project folder upon login.
Changes will take effect on next login or manually with source ~/.bash_profile
.
vim can be configured in ~/.vimrc
. Turn on line numbers to easily jump to a specific line using :[line number]
set number
Shell scripting
Shell scripting can be used to submit several jobs as exemplified below
#!/bin/bash for i in $(seq 0.0 0.2 1.4); do for j in $(seq 0.0 0.2 1.4); do dir="$i-$j" echo $dir mkdir $dir cp INCAR POTCAR KPOINTS jobfile $dir cp "prefix_$dir.POSCAR.vasp" $dir/POSCAR cd $dir sbatch jobfile cd .. done done
First VASP calculation
File | Description |
---|---|
POSCAR | Structure file that specifies the unit cell vectors, types of atoms and their positions |
POTCAR | Pseudopotentials for the atoms (same order as in POSCAR) |
INCAR | Input parameters for the calculation including relaxation algorithms, functionals, convergence criteria, etc. |
KPOINTS | k-point grid |
jobfile | job script |
Cell volume relaxation of BaZrO3 unit cell
Go to the project folder: cd /cluster/projects/nn4604k
Or if you have defined an alias: nn
Create a subfolder with your username (if you have not done so already)
mkdir [username]
cd [username]
mkdir test
cd test
Copy POSCAR, KPOINTS and INCAR from the 'vasptest' folder to your current folder (reminder: use TAB to autocomplete the path)
cp ../../vasptest/POSCAR .
Press the UP arrow key to bring up your previous command
cp ../../vasptest/INCAR .
cp ../../vasptest/KPOINTS .
cp ../../vasptest/jobfile .
Copy the necessary POTCAR files
cp ../../potpaw_PBE.54/Ba_sv/POTCAR POTCAR_Ba_sv
cp ../../potpaw_PBE.54/Zr_sv/POTCAR POTCAR_Zr_sv
cp ../../potpaw_PBE.54/O/POTCAR POTCAR_O
Check the order of the atoms in POSCAR file
head POSCAR
Concatenate the POTCAR files in the same order to a file named POTCAR
cat POTCAR_Ba_sv POTCAR_Zr_sv POTCAR_O > POTCAR
Check the order of the POTCAR
grep TITEL POTCAR
Copy job script to current folder
cp ../../jobfile .
Submit the job
sbatch jobfile
Check the queue
squeue -u [username]
Or if you have defined an alias: q
When the job starts you can follow the progress in the slurm log file
less slurm-[slurm-id]
After three ionic relaxation steps, the required accuracy has been reached and the output files defined job script are copied to the current folder.
3 F= -.41724097E+02 E0= -.41724097E+02 d E =-.431449E-02 curvature: -0.24 expect dE=-0.515E-06 dE for cont linesearch -0.515E-06 trial: gam= 0.00000 g(F)= 0.940E-64 g(S)= 0.213E-05 ort =-0.198E-03 (trialstep = 0.895E+00) search vector abs. value= 0.213E-05 reached required accuracy - stopping structural energy minimisation writing wavefunctions
Check the lattice parameter in the POSCAR (input) and output (CONTCAR)
head POSCAR
BaZrO3 1.00000000000000 4.2200000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 4.2200000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 4.2200000000000000
head POSCAR
BaZrO3 1.00000000000000 4.2353879522908331 -0.0000000000000000 -0.0000000000000000 0.0000000000000000 4.2353879522908331 -0.0000000000000000 0.0000000000000000 -0.0000000000000000 4.2353879522908331
The optimized cubic lattice parameter is 4.2354 Å.