Difference between revisions of "WorkingOnServers"

From mn/geo/geoit
Jump to: navigation, search
(Connecting to the Servers)
 
(13 intermediate revisions by 3 users not shown)
Line 2: Line 2:
  
 
At UiO we have a powerful resource of computers available for running various meteorological codes. The documentation here is provided to help you work on the servers efficiently from your own desktop. These guides will hopefully also be useful for running code on any other cluster computers including the Norwegian Super Computing Resources and ECMWF.
 
At UiO we have a powerful resource of computers available for running various meteorological codes. The documentation here is provided to help you work on the servers efficiently from your own desktop. These guides will hopefully also be useful for running code on any other cluster computers including the Norwegian Super Computing Resources and ECMWF.
 +
 +
The computing facilities at the Department of Geosciences are consisting of several sub-systems.
 +
<p style="text-align: center">[[File:Overview.jpg|center|638x498px]]</p>
 +
<br />Your home directory (accessible via an environment variable called $HOME) should not be used to store large datasets. Data in home partition are backed-up once a day (incremental back-up).
 +
 +
If you need to store large datasets, please contact us and we will provide you with storage space on our data handling server (vann) or advice you on [https://www.norstore.no/ NorStore].
 +
 +
The policy at the Geosciences Department is to provide users with "standard" PCs that are not necessarily very powerful but contains all the software/libraries necessary for your work. Once your code is in production, you may use our general purpose servers (sverdrup, etc.) and if insufficient you may migrate your code on our [http://www.uio.no/english/services/it/research/hpc/abel/ High-Performance Computing server].
 +
 +
  
 
For some information on the Abel cluster resources available at UiO see [http://www.uio.no/english/services/it/research/hpc/abel/ abel].
 
For some information on the Abel cluster resources available at UiO see [http://www.uio.no/english/services/it/research/hpc/abel/ abel].
Line 10: Line 20:
  
 
=== Connecting to the Servers ===
 
=== Connecting to the Servers ===
 +
 +
First things first, get comfortable with ssh. Hans Peter created an [http://www.mn.uio.no/geo/english/services/it/help/using-linux/ssh-tips-and-tricks.html Excellent Tutorial].
  
 
There are many tools that make connecting to the servers easier, from '''passwordless login''' (which is more secure), to making sure you have your '''display settings'''
 
There are many tools that make connecting to the servers easier, from '''passwordless login''' (which is more secure), to making sure you have your '''display settings'''
  
 +
*Professor Burkhart's [[https://wiki.uio.no/mn/geo/geoit/index.php/WorkingOnServers/JFBsWorkFlow WorkFlow]] -- this is a 'cookbook' solution to use Mac or Linux clients to work on the servers from anywhere and always start where you left off. The key tools are&nbsp;''ssh, screen,&nbsp;''and a few&nbsp;''scripts.''
 +
*If you have a linux machine, maybe you want to [[keep a link]] to the UIO servers in your $HOME directory.
 
*Return right to where you left off by following some DisplayEnvironment&nbsp;[https://wiki.uio.no/mn/geo/geoit/index.php/WorkingOnServers/DisplayEnvironmentTricks Tricks]
 
*Return right to where you left off by following some DisplayEnvironment&nbsp;[https://wiki.uio.no/mn/geo/geoit/index.php/WorkingOnServers/DisplayEnvironmentTricks Tricks]
*Professor Burkhart's [[https://wiki.uio.no/mn/geo/geoit/index.php/WorkingOnServers/JFBsWorkFlow WorkFlow]] -- this is a 'cookbook' solution to use Mac or Linux clients to work on the servers from anywhere and always start where you left off. The key tools are&nbsp;''ssh, screen,&nbsp;''and a few&nbsp;''scripts.''
+
 
 +
 
 +
 
 +
=== [https://wiki.uio.no/mn/geo/geoit/index.php/WorkingOnServers/ModulePackage Set-up your environment at UIO] (usage of module package) ===
 +
 
 +
On most of our linux machines, you can use the module package to set up your user environment (PATH, LD_LIBRARY_PATH, etc.).
 +
 
 +
 
 +
 
 +
=== UNIX & SHELL (not updated by UIO) ===
 +
 
 +
*Software on LINUX systems [https://wiki.uio.no/mn/geo/geoit/Reference%2FRequired%20Software Reference/Required Software]
 +
*Better Bashing - Tips & Tricks for the Bash Shell [https://wiki.uio.no/mn/geo/geoit/Shell%2FBetterBashing Shell/BetterBashing]
 +
*How to use your daily UNIX shell [https://wiki.uio.no/mn/geo/geoit/Shell%2FUNIXHints Shell/UNIXHints]
 +
*Using the SVN versioning system [https://wiki.uio.no/mn/geo/geoit/Shell%2FUsingSVN Shell/UsingSVN]  [[Category:Tools]] <br /> [[Category:UioNetwork]] <br /> [[Category:Svedrup]] <br /> [[Category:geo-hiperf]]

Latest revision as of 15:12, 2 December 2016

Overview

At UiO we have a powerful resource of computers available for running various meteorological codes. The documentation here is provided to help you work on the servers efficiently from your own desktop. These guides will hopefully also be useful for running code on any other cluster computers including the Norwegian Super Computing Resources and ECMWF.

The computing facilities at the Department of Geosciences are consisting of several sub-systems.

Overview.jpg


Your home directory (accessible via an environment variable called $HOME) should not be used to store large datasets. Data in home partition are backed-up once a day (incremental back-up).

If you need to store large datasets, please contact us and we will provide you with storage space on our data handling server (vann) or advice you on NorStore.

The policy at the Geosciences Department is to provide users with "standard" PCs that are not necessarily very powerful but contains all the software/libraries necessary for your work. Once your code is in production, you may use our general purpose servers (sverdrup, etc.) and if insufficient you may migrate your code on our High-Performance Computing server.


For some information on the Abel cluster resources available at UiO see abel.

To see the status of the servers check here: AbelLog.

To see the CPU and Memory usage of the cluster check here: AbelCPU

Connecting to the Servers

First things first, get comfortable with ssh. Hans Peter created an Excellent Tutorial.

There are many tools that make connecting to the servers easier, from passwordless login (which is more secure), to making sure you have your display settings

  • Professor Burkhart's [WorkFlow] -- this is a 'cookbook' solution to use Mac or Linux clients to work on the servers from anywhere and always start where you left off. The key tools are ssh, screen, and a few scripts.
  • If you have a linux machine, maybe you want to keep a link to the UIO servers in your $HOME directory.
  • Return right to where you left off by following some DisplayEnvironment Tricks


Set-up your environment at UIO (usage of module package)

On most of our linux machines, you can use the module package to set up your user environment (PATH, LD_LIBRARY_PATH, etc.).


UNIX & SHELL (not updated by UIO)