Difference between revisions of "Getting started"
m |
m |
||
Line 112: | Line 112: | ||
{| class="wikitable" | {| class="wikitable" | ||
− | |+ | + | |+Special paths and symbols |
!Symbol | !Symbol | ||
!Description | !Description | ||
Line 174: | Line 174: | ||
===Jobscript=== | ===Jobscript=== | ||
− | + | Sigma2 NRIS HPC clusters use Slurm as workload manager and job scheduler. Documentation is available on [https://documentation.sigma2.no/ documentation.sigma2.no/] | |
+ | |||
+ | Computations are run by submitting a job script to the queue | ||
+ | |||
+ | <code> sbatch jobfile </code> | ||
+ | |||
+ | View the queue for a user or project | ||
+ | |||
+ | <code> squeue -u [username] </code> | ||
+ | <code> squeue -p [project number] </code> | ||
+ | |||
+ | Job scripts for Saga are available in the project folders | ||
+ | /cluster/projects/nn4604k/jobfile | ||
+ | |||
+ | The job script contains several SBATCH options | ||
#SBATCH --job-name=[jobname] | #SBATCH --job-name=[jobname] | ||
Line 183: | Line 197: | ||
#SBATCH --ntasks-per-node=40 | #SBATCH --ntasks-per-node=40 | ||
− | Copy files to temporary job folder | + | Saga has 40 CPUs per node and it is recommended to use all CPUs on a node and an 1-2 nodes for optimal performance [https://documentation.sigma2.no/hpc_machines/saga.html documentation.sigma2.no/hpc_machines/saga] |
+ | |||
+ | Copy files to temporary job folder ($SCRATCH) | ||
cp [filenames] $SCRATCH | cp [filenames] $SCRATCH | ||
− | Copy files from job folder when job completes. Savefile will also copy files if the job crashes. | + | Copy output files from the job folder to the submit folder when job completes. Savefile will also copy files if the job crashes. |
savefile [filenames] | savefile [filenames] | ||
The working folder can be accessed while a job is running | The working folder can be accessed while a job is running | ||
<code> /cluster/work/jobs/[slurm-id] </code> | <code> /cluster/work/jobs/[slurm-id] </code> | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
===Aliases and settings === | ===Aliases and settings === |
Revision as of 19:56, 26 September 2021
Contents
User access
Go to www.metacenter.no > User Login (Passwords) > 1. Request a user account > HPC
Fill in the form and choose the appropriate project. It is recommended to use your UiO username.
The project manager can give you access to the VASP license through the VASP portal and thereafter informing sigma2@uninett.no to give your user the necessary group affiliations (xvasp and vasp5).
Project | Manager | Group | Resources |
---|---|---|---|
NN4604K | Jonathan M. Polfus | Electrochemistry | Saga |
NN9136K | Lasse Vines | LENS | Saga |
NN9180K | Clas Persson | Structure Physics | Saga |
Login
ssh and scp
ssh can be used to login to the cluster with a terminal app on UNIX, macOS or Windows Subsystem for Linux
ssh [username]@saga.sigma2.no
Hostnames and usernames can be saved by creating or editing the config file using for instance the vim text editor
vim ~/.ssh/config
Press i
to insert text
Host saga HostName = saga.sigma2.no user [username]
Press escape
to exit insert mode, and then :wq
to 'write' and 'quit'.
Login is then simplified
ssh saga
Copy files from cluster to current folder (.
)
scp saga:[path]/[filename] .
Copy files to the home directory on the cluster
scp [filename] saga:
PuTTY and WinSCP
PuTTY is an ssh client for Windows that can be downloaded from www.putty.org.
WinSCP is an scp client for Windows that can be downloaded from winscp.net.
Working in UNIX
Basic commands
Learn to use the TAB key to autocomplete commands, paths and filenames.
Learn to view the manual for any command: man [command]
Command | Description |
---|---|
ls
|
List files and directories in current directory |
mkdir [directoryname]
|
Make a directory |
cd [directoryname]
|
Change current path to another directory |
pwd
|
Display current path |
cd ..
|
Change path to parent directory |
cp [path]/[filename] .
|
Copy file to current directory |
cp [filename] [directoryname]
|
Copy file to another directory |
mv [filename] [path]
|
Move file to another directory |
mv [filename] [newfilename]
|
Change the name of a file |
cp -r [directory] [path]
|
Copy directory and its contents to new location |
rm [filename]
|
Delete file |
rm -r [directoryname]
|
Delete directory and its contents |
Symbol | Description |
---|---|
.
|
Current directory |
..
|
Parent directory |
~
|
Home directory |
/
|
Root directory |
*
|
Any text pattern |
?
|
Any symbol |
Command | Description |
---|---|
less
|
View text file. View live updates to end of file: Shift-F
|
head
|
Display top 10 lines of text file. Specify number of lines: -n [count]
|
tail
|
Display bottom 10 lines of text file. Specify number of lines: -n [count]
|
grep [pattern]
|
Display each line in text file containing text specific text/symbol pattern. |
cat
|
Concatenate text files sequentially. Write the output to a file: > [filename]
|
Filesystem on Saga
Home directory: mainly for personal files, scripts etc. due to limited storage space
/cluster/home/[username]
Project folder: Run jobs and store output files in a subfolder called your username
/cluster/projects/[project number]/[username]
Check disk usage and storage quota for home directory and project folder
dusage
Find and remove files matching [filename] in current folder and subfolders
find . -name "[filename]" -exec rm {} \;
Jobscript
Sigma2 NRIS HPC clusters use Slurm as workload manager and job scheduler. Documentation is available on documentation.sigma2.no/
Computations are run by submitting a job script to the queue
sbatch jobfile
View the queue for a user or project
squeue -u [username]
squeue -p [project number]
Job scripts for Saga are available in the project folders
/cluster/projects/nn4604k/jobfile
The job script contains several SBATCH options
#SBATCH --job-name=[jobname] #SBATCH --account=[project number] #SBATCH --time=hh:mm:ss #SBATCH --mem-per-cpu=4600M #SBATCH --nodes=2 #SBATCH --ntasks-per-node=40
Saga has 40 CPUs per node and it is recommended to use all CPUs on a node and an 1-2 nodes for optimal performance documentation.sigma2.no/hpc_machines/saga
Copy files to temporary job folder ($SCRATCH)
cp [filenames] $SCRATCH
Copy output files from the job folder to the submit folder when job completes. Savefile will also copy files if the job crashes.
savefile [filenames]
The working folder can be accessed while a job is running
/cluster/work/jobs/[slurm-id]
Aliases and settings
Define shortcuts by adding aliases to ~/.bash_profile
alias l='ls -l' alias la='ls -l -a' alias ..='cd ..' alias nn='cd /cluster/projects/[project number]' alias q='squeue -u [username]'
You can also add commands such as automatically changing directory to the project folder upon login.
Changes will take effect on next login or manually with source ~/.bash_profile
.
vim can be configured in ~/.vimrc
. Turn on line numbers to easily jump to a specific line using :[line number]
set number