Difference between revisions of "Getting started"
m |
m (→Job script) |
||
(11 intermediate revisions by 2 users not shown) | |||
Line 3: | Line 3: | ||
Go to [https://www.metacenter.no www.metacenter.no] > User Login (Passwords) > 1. Request a user account > HPC | Go to [https://www.metacenter.no www.metacenter.no] > User Login (Passwords) > 1. Request a user account > HPC | ||
− | Fill in the form and choose the appropriate project. | + | Fill in the form with your UiO username and choose the appropriate project. If you already have an account through for instance FYS-MENA41111, the project manager can in principle directly add you to the project without requesting a user account. |
− | The project manager can give you access to the VASP license through the [https://www.vasp.at/ VASP portal] and thereafter informing [mailto: | + | The project manager can give you access to the VASP license through the [https://www.vasp.at/ VASP portal] and thereafter informing [mailto:sigma@uninett.no sigma2@uninett.no] to give your user the necessary group affiliations (vasp5). |
{| class="wikitable" | {| class="wikitable" | ||
Line 180: | Line 180: | ||
<code> find . -name "[filename]" -exec rm {} \; </code> | <code> find . -name "[filename]" -exec rm {} \; </code> | ||
+ | |||
+ | ===Logfile=== | ||
+ | Excel sheet for tracking jobs can be downloaded here: [[Media:Logfile.xlsx]] | ||
+ | It also includes conversion between supercell size, number of atoms and k-point grid for a given k-point density. | ||
===Job script=== | ===Job script=== | ||
Line 202: | Line 206: | ||
The module system is restored to avoid potential conflicts. There exist several compiled versions of VASP and specific versions are compiled for increased performance with gamma K-point grid | The module system is restored to avoid potential conflicts. There exist several compiled versions of VASP and specific versions are compiled for increased performance with gamma K-point grid | ||
module restore system | module restore system | ||
− | module load VASP/ | + | module load VASPModules |
− | #module load VASP/ | + | module load VASP/6.4.1-intel-2022b-std-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74 |
+ | # module load VASP/6.4.1-intel-2022b-gam-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74 | ||
Available modules on the cluster can be listed with | Available modules on the cluster can be listed with | ||
Line 228: | Line 233: | ||
+ | '''Example job script''' | ||
#!/bin/bash | #!/bin/bash | ||
− | #SBATCH --job-name= | + | #SBATCH --job-name=jobname |
− | #SBATCH --account= | + | #SBATCH --account=nn4604k |
#SBATCH --time=10:00:00 | #SBATCH --time=10:00:00 | ||
− | #SBATCH --mem-per-cpu= | + | #SBATCH --mem-per-cpu=4000M |
− | #SBATCH --nodes= | + | #SBATCH --nodes=1 |
#SBATCH --ntasks-per-node=40 | #SBATCH --ntasks-per-node=40 | ||
Line 240: | Line 246: | ||
set -o nounset | set -o nounset | ||
− | + | module restore system | |
− | + | module load VASPModules | |
− | + | module load VASP/6.4.1-intel-2022b-std-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74 | |
− | + | # module load VASP/6.4.1-intel-2022b-gam-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74 | |
+ | |||
cp INCAR POSCAR POTCAR KPOINTS $SCRATCH | cp INCAR POSCAR POTCAR KPOINTS $SCRATCH | ||
if [ -f WAVECAR ] | if [ -f WAVECAR ] | ||
Line 328: | Line 335: | ||
<code> cd test </code> | <code> cd test </code> | ||
− | Copy | + | Copy all the files from the 'vasptest' folder (POSCAR, KPOINTS and INCAR) to your current folder (remember to use TAB to autocomplete the path) |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | <code> cp ../../vasptest/ | + | <code> cp ../../vasptest/* . </code> |
− | + | Copy the necessary POTCAR files (press the UP arrow key to bring up your previous command) | |
− | |||
− | Copy the necessary POTCAR files | ||
<code> cp ../../potpaw_PBE.54/Ba_sv/POTCAR POTCAR_Ba_sv </code> | <code> cp ../../potpaw_PBE.54/Ba_sv/POTCAR POTCAR_Ba_sv </code> | ||
Line 396: | Line 395: | ||
0.0000000000000000 4.2200000000000000 0.0000000000000000 | 0.0000000000000000 4.2200000000000000 0.0000000000000000 | ||
0.0000000000000000 0.0000000000000000 4.2200000000000000 | 0.0000000000000000 0.0000000000000000 4.2200000000000000 | ||
− | + | ... | |
− | <code> head | + | <code> head CONTCAR </code> |
BaZrO3 | BaZrO3 | ||
Line 404: | Line 403: | ||
0.0000000000000000 4.2353879522908331 -0.0000000000000000 | 0.0000000000000000 4.2353879522908331 -0.0000000000000000 | ||
0.0000000000000000 -0.0000000000000000 4.2353879522908331 | 0.0000000000000000 -0.0000000000000000 4.2353879522908331 | ||
− | + | ... | |
The optimized cubic lattice parameter is 4.2354 Å. | The optimized cubic lattice parameter is 4.2354 Å. |
Latest revision as of 13:38, 3 November 2023
Contents
Getting started
User access
Go to www.metacenter.no > User Login (Passwords) > 1. Request a user account > HPC
Fill in the form with your UiO username and choose the appropriate project. If you already have an account through for instance FYS-MENA41111, the project manager can in principle directly add you to the project without requesting a user account.
The project manager can give you access to the VASP license through the VASP portal and thereafter informing sigma2@uninett.no to give your user the necessary group affiliations (vasp5).
Project | Manager | Group | Resources |
---|---|---|---|
NN4604K | Jonathan M. Polfus | Electrochemistry | Saga |
NN9136K | Lasse Vines | LENS | Saga |
NN9180K | Clas Persson | Structure Physics | Saga |
Login
ssh and scp
ssh can be used to login to the cluster with a terminal app on UNIX, macOS or Windows Subsystem for Linux
ssh [username]@saga.sigma2.no
Hostnames and usernames can be saved by creating or editing the config file using for instance the vim text editor
vim ~/.ssh/config
Press i
to insert text
Host saga HostName = saga.sigma2.no user [username]
Press escape
to exit insert mode, and then :wq
to 'write' and 'quit'.
Login is then simplified
ssh saga
Copy files from cluster to current folder (.
)
scp saga:[path]/[filename] .
Copy files to the home directory on the cluster
scp [filename] saga:
PuTTY and WinSCP
PuTTY is an ssh client for Windows that can be downloaded from www.putty.org.
WinSCP is an scp client for Windows that can be downloaded from winscp.net.
Working in UNIX
Basic commands
Learn to use the TAB key to autocomplete commands, paths and filenames.
Learn to view the manual for any command: man [command]
Command | Description |
---|---|
ls
|
List files and directories in current directory |
mkdir [directoryname]
|
Make a directory |
cd [directoryname]
|
Change current path to another directory |
pwd
|
Display current path |
cd ..
|
Change path to parent directory |
cp [path]/[filename] .
|
Copy file to current directory |
cp [filename] [directoryname]
|
Copy file to another directory |
mv [filename] [path]
|
Move file to another directory |
mv [filename] [newfilename]
|
Change the name of a file |
cp -r [directory] [path]
|
Copy directory and its contents to new location |
rm [filename]
|
Delete file |
rm -r [directoryname]
|
Delete directory and its contents |
Symbol | Description |
---|---|
.
|
Current directory |
..
|
Parent directory |
~
|
Home directory |
/
|
Root directory |
*
|
Any text pattern |
?
|
Any symbol |
Command | Description |
---|---|
less
|
View text file. View live updates to end of file: Shift-F
|
head
|
Display top 10 lines of text file. Specify number of lines: -n [count]
|
tail
|
Display bottom 10 lines of text file. Specify number of lines: -n [count]
|
grep [pattern]
|
Display each line in text file containing text specific text/symbol pattern. |
diff [file1] [file2] | Compare files line by line and display the difference between them. |
cat
|
Concatenate text files sequentially. Write the output to a file: > [filename]
|
Filesystem on Saga
Home directory: mainly for personal files, scripts etc. due to limited storage space
/cluster/home/[username]
Project folder: Run jobs and store output files in a subfolder called your username
/cluster/projects/[project number]/[username]
Check disk usage and storage quota for home directory and project folder
dusage
Check disk usage of current folder and subfolders
du -hs
Find and remove files matching [filename] in current folder and subfolders
find . -name "[filename]" -exec rm {} \;
Logfile
Excel sheet for tracking jobs can be downloaded here: Media:Logfile.xlsx It also includes conversion between supercell size, number of atoms and k-point grid for a given k-point density.
Job script
Sigma2 NRIS HPC clusters use Slurm as workload manager and job scheduler. Documentation is available on documentation.sigma2.no/
Computations are run by submitting a job script to the queue
sbatch jobfile
View the queue for a user or project
squeue -u [username]
squeue -p [project number]
Job scripts for Saga are available in the project folder
/cluster/projects/nn4604k/jobfile
The job script below contains several SBATCH options
Saga has 40 CPUs per node and it is recommended to use all CPUs on a node and a 1-2 nodes for optimal performance documentation.sigma2.no/hpc_machines/saga
The module system is restored to avoid potential conflicts. There exist several compiled versions of VASP and specific versions are compiled for increased performance with gamma K-point grid
module restore system module load VASPModules module load VASP/6.4.1-intel-2022b-std-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74 # module load VASP/6.4.1-intel-2022b-gam-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74
Available modules on the cluster can be listed with
module avail
Files are copied to a temporary job folder ($SCRATCH). The WAVECAR file is copied if it is available.
cp [filenames] $SCRATCH if [ -f WAVECAR ] then cp WAVECAR $SCRATCH fi
Output files are copied from the job folder to the submit folder when job completes. Savefile will also copy files if the job crashes.
savefile [filenames]
The job is started from the $SCRATCH folder and the elapsed time is recorded to slurm output
cd $SCRATCH time srun vasp
The progression of the job can be viewed in the slurm output file slurm-[slurm-id] in the submit folder once the job is running.
The working folder can be accessed while the job is running
/cluster/work/jobs/[slurm-id]
Example job script
#!/bin/bash #SBATCH --job-name=jobname #SBATCH --account=nn4604k #SBATCH --time=10:00:00 #SBATCH --mem-per-cpu=4000M #SBATCH --nodes=1 #SBATCH --ntasks-per-node=40 set -o errexit set -o nounset
module restore system module load VASPModules module load VASP/6.4.1-intel-2022b-std-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74
- module load VASP/6.4.1-intel-2022b-gam-wannier90-libxc-hdf5-beef-0a928426e459cf2aeab3d0bf8f441c74
cp INCAR POSCAR POTCAR KPOINTS $SCRATCH if [ -f WAVECAR ] then cp WAVECAR $SCRATCH fi savefile CONTCAR OUTCAR DOSCAR CHGCAR WAVECAR cd $SCRATCH time srun vasp
Aliases and settings
Define useful shortcuts by adding aliases to ~/.bash_profile
alias l='ls -l' alias la='ls -l -a' alias ..='cd ..' alias nn='cd /cluster/projects/[project number]' alias q='squeue -u [username]'
You can also add commands such as automatically changing directory to the project folder upon login.
Changes will take effect on next login or manually with source ~/.bash_profile
.
vim can be configured in ~/.vimrc
. Turn on line numbers to easily jump to a specific line using :[line number]
set number
Shell scripting
Shell scripting can be used to submit several jobs as exemplified below
#!/bin/bash for i in $(seq 0.0 0.2 1.4); do for j in $(seq 0.0 0.2 1.4); do dir="$i-$j" echo $dir mkdir $dir cp INCAR POTCAR KPOINTS jobfile $dir cp "prefix_$dir.POSCAR.vasp" $dir/POSCAR cd $dir sbatch jobfile cd .. done done
First VASP calculation
File | Description |
---|---|
POSCAR | Structure file that specifies the unit cell vectors, types of atoms and their positions |
POTCAR | Pseudopotentials for the atoms (same order as in POSCAR) |
INCAR | Input parameters for the calculation including relaxation algorithms, functionals, convergence criteria, etc. |
KPOINTS | k-point grid |
jobfile | job script |
Cell volume relaxation of BaZrO3 unit cell
Go to the project folder: cd /cluster/projects/nn4604k
Or if you have defined an alias: nn
Create a subfolder with your username (if you have not done so already)
mkdir [username]
cd [username]
mkdir test
cd test
Copy all the files from the 'vasptest' folder (POSCAR, KPOINTS and INCAR) to your current folder (remember to use TAB to autocomplete the path)
cp ../../vasptest/* .
Copy the necessary POTCAR files (press the UP arrow key to bring up your previous command)
cp ../../potpaw_PBE.54/Ba_sv/POTCAR POTCAR_Ba_sv
cp ../../potpaw_PBE.54/Zr_sv/POTCAR POTCAR_Zr_sv
cp ../../potpaw_PBE.54/O/POTCAR POTCAR_O
Check the order of the atoms in POSCAR file
head POSCAR
Concatenate the POTCAR files in the same order to a file named POTCAR
cat POTCAR_Ba_sv POTCAR_Zr_sv POTCAR_O > POTCAR
Check the order of the POTCAR
grep TITEL POTCAR
Copy job script to current folder
cp ../../jobfile .
Submit the job
sbatch jobfile
Check the queue
squeue -u [username]
Or if you have defined an alias: q
When the job starts you can follow the progress in the slurm log file
less slurm-[slurm-id]
After three ionic relaxation steps, the required accuracy has been reached and the output files defined job script are copied to the current folder.
3 F= -.41724097E+02 E0= -.41724097E+02 d E =-.431449E-02 curvature: -0.24 expect dE=-0.515E-06 dE for cont linesearch -0.515E-06 trial: gam= 0.00000 g(F)= 0.940E-64 g(S)= 0.213E-05 ort =-0.198E-03 (trialstep = 0.895E+00) search vector abs. value= 0.213E-05 reached required accuracy - stopping structural energy minimisation writing wavefunctions
Check the lattice parameter in the POSCAR (input) and output (CONTCAR)
head POSCAR
BaZrO3 1.00000000000000 4.2200000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 4.2200000000000000 0.0000000000000000 0.0000000000000000 0.0000000000000000 4.2200000000000000 ...
head CONTCAR
BaZrO3 1.00000000000000 4.2353879522908331 -0.0000000000000000 -0.0000000000000000 0.0000000000000000 4.2353879522908331 -0.0000000000000000 0.0000000000000000 -0.0000000000000000 4.2353879522908331 ...
The optimized cubic lattice parameter is 4.2354 Å.