Difference between revisions of "Welcome to Geosciences Modeling Wiki"

From mn/geo/geoit
Jump to: navigation, search
(Added SURFEX/Crocus section where we will compile information on how to run SURFEX/Crocus on our machines)
 
(67 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Welcome to the Geosciences Modeling and Data Science Wiki.</span></span>
+
This Wiki is meant to provide a resource for those at UiO interested in working with computer modeling and data analysis in Earth Sciences.
  
== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Purpose:</span></span> ==
+
Content herein is generally specific to UiO, as otherwise it is recommended to make posts to [http://stackexchange.com/sites Stack Exchange] and the sites therein.
  
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Provide information on software tools available for data analysis and scientific program for projects within the field of Geosciences&nbsp;''in general''&nbsp;and to provide users a cenral resource for keeping notes on '''UIO specific''' aspects of running various software routines.&nbsp;Ideally, this site will become overtime a 'go to' location where incoming students who will be working with modeling and heavy data analysis on servers can find tips for getting started.</span></span>
+
This wiki uses 'category' tags, and below are some of the categories available. Note that in some cases, the content does not yet exist, in which case you are encouraged to edit and add information, or [https://wiki.uio.no/mn/geo/geoit/index.php/Maintainers contact one of the site maintainers] to request further information.
  
== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Make a contribution</span></span> ==
+
=== <span style="font-family:arial,helvetica,sans-serif">Purpose:</span> ===
 +
<span style="font-family:arial,helvetica,sans-serif">Provide information on software tools available for data analysis and scientific program for projects within the field of Geosciences&nbsp;''in general''&nbsp;and to provide users a cenral resource for keeping notes on '''UIO specific''' aspects of running various software routines.&nbsp;Ideally, this site will become overtime a 'go to' location where incoming students who will be working with modeling and heavy data analysis on servers can find tips for getting started.</span>
  
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Feel free to tackle something from the list of [[NeededUpdates]]. Essentially, the usefullness of this site is up to '''you'''. We are building this up as we go, but&nbsp;don't be shy. Your&nbsp;contributions are valuable. Please, if you are working on something and stumble... it's likely someone else may have the same problem eventually. Make a note here of your solution.&nbsp;</span></span>
+
Enjoy!
  
== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Tools and Topics</span></span> ==
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Editing Editing pages]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:How_to How to]
=== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Tools for [[WorkingOnServers]] at UiO.</span></span> ===
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Style_guide Style guide]
 
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Data Data]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Processing large datasets and running models for geosciences will likely require that you work on the command line of a server and not on your laptop. It's important to learn how to do this efficiently so that you don't feel that it restricts where you can work from. In fact, it's generally the opposite. Once you learn how to use a server well, you will find it much easier to work from anywhere! All you need is an internet connection.</span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Abbreviations_and_terminology Abbreviations_and_terminology]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Data_formats Formats]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">In this section, we cover some tips for connecting via ssh, setting up your own laptop so it is 'as if' you are working locally, and other nice tricks for staying productive in a server-based environment.</span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Measurements Measurements]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Ordering Ordering]
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[Python]]</span></span> ====
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Satellites Satellite imagery]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Re_analysis Re-analysis]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">A starting point for all-things Python. Provides information on best practices, different packages that are frequently used, and other information to make using Python at NILU easier.</span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:GIS GIS]
 
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Tools Tools]
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">General [[Fortran]] Tips</span></span> ====
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/WorkingOnServers WorkingOnServers]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Command_line Command_line]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Some general usage tips for coding in FORTRAN.</span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Software Software]
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Code_Coffee Code Coffee]<br />
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">GNU and intel compilers are both available on our servers. We have a very limited number of licenses for intel compilers so we suggest you to use GNU compilers for all your development.</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:R R]<br />
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Fortran Fortran]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">There is a default GNU compiler but it may be a quite old version. If you wish to use a newer version:</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Micmac Micmac]
<pre><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">          module avail gcc
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Python Python]
</span></span></pre>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Matlab Matlab]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">it returns the list of available versions.</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Excel Excel]
<pre><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">          module load gcc/4.9.0
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:UioNetwork Computing Facilities<br />]
</span></span></pre>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Login How to login?]<br />
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">It loads GNU compilers 4.9.0 (both Fortran, C and C++ compilers).</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Wessel Wessel]<br />
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Sverdrup Sverdrup]<br />
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><br/></span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Moulin Moulin]
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:geo-hiperf geo-hiperf]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Intel compilers are available by default on our UIO servers but on abel, you would need to load it:</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Abel Abel HPC]<br />
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Nortur Notur HPC]<br />
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><br/></span></span>
+
:::[[Set-up your own server (IaaS)]]
<pre><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">          module load intel</span></span></pre>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Norstore Data storage (lagrinshotell, Norstore & storage for researchers)]<br />
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><br/></span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Models Models]
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:WRF WRF, WRF-CHEM and WRF-Hydro]
===== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Fortran materials</span></span> =====
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:OpenIFS OpenIFS]
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:FLEXPART Flexpart]
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Gunnar Wollan has compiled a set of documents at [http://folk.uio.no/gunnarw/GEO4060/ http://folk.uio.no/gunnarw/GEO4060/] and you can also find some of his documents at [http://www.uio.no/studier/emner/matnat/geofag/GEF4510/h12/index.html http://www.uio.no/studier/emner/matnat/geofag/GEF4510/h12/index.html]</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:MITgcm MITgcm]
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Cesm cesm]
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">An interesting free tutorial at [http://www.fortrantutorial.com/ http://www.fortrantutorial.com/] You can skip the installation of silverfrost compilers if you already have a Fortran compiler installed on your machine or if you are using sverdrup.uio.no</span></span>
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Noresm noresm]<br />
 
+
:::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:ESysParticle ESys-Particle]
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">The National Computational Infrastructure in Australia has class notes online for a Basic Fortran course ([http://nf.nci.org.au/training/FortranBasic/ http://nf.nci.org.au/training/FortranBasic/]) and an Advanced Fortran course ([http://nf.nci.org.au/training/FortranAdvanced/ http://nf.nci.org.au/training/FortranAdvanced/]).</span></span>
+
:::[[SURFEX/Crocus]]<br />
 
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Field_of_study Field of study]
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Fortran Wiki [http://fortranwiki.org/fortran/show/HomePage http://fortranwiki.org/fortran/show/HomePage]</span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Hydrology Hydrology]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Meteorology Meteorology]
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Getting data from [[ECMWF]]</span></span> ====
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Glaciology Glaciology]
 
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Communication Communication]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Decriptions of how to get data in various ways from the ECMWF server.</span></span>
+
::[[Mail and Mailing lists|Mail and Mailing lists]]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Presentations Presentations]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><br/></span></span>
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Templates Templates]
 
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Logo Logo]
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[FLEXPART Model]]</span></span> ====
+
::[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Climabyte Climabyte]
 
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Administration Administration]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Information and resources for running and setting up FLEXPART. Mostly geared toward the ATMOS users.</span></span>
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Category:Teaching Teaching]
 
+
:[https://wiki.uio.no/mn/geo/geoit/index.php/Summerschools Summerschools]
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><br/></span></span>
 
 
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[FLEXINVERT]]</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Bayesian inversion code for use with FLEXPART</span></span>
 
 
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[WRFand WRF-CHEM|WRF and WRF-CHEM Models]]</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Information and resources for running and setting up WRF and WRF-CHEM.</span></span>
 
 
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[OpenIFS|OpenIFS ECMWF Model]]</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Information and resources for running and setting up OpenIFS (ECMWF forecast model).</span></span>
 
 
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[DataFormats|Data formats ]](netCDF, HDF4, HDF5, GRIB-1, GRIB-2, BUFR, etc.)</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Descriptions on how to manipulate and visualize data in these different formats.</span></span>
 
 
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[[Matlab]] (Not open source, but extensively used)</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Contributions from MATLAB users</span></span>
 
 
 
== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Editing pages on the Geo-IT MediaWiki</span></span> ==
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">A good starting place for markup reference can be found here: [http://meta.wikimedia.org/wiki/Help:Wikitext_examples http://meta.wikimedia.org/wiki/Help:Wikitext_examples]</span></span>
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Another point is that this wiki is using Subpages, so in places you'll see reference to a <nowiki><subpages .></nowiki> tag. This just means that the list of subpages will be dynamically generated in that location.</span></span>
 
 
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Editing Source Code</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Note that for the purposes of documenting source code there are two important features:</span></span>
 
 
 
#<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">use the <nowiki><code> ... </code></nowiki> to show some simple code snippets: <code>def foo(): return</code></span></span>
 
#<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">The syntax highlighting extension has been installed, so you can wrap source code examples in <nowiki><syntaxhighlight></nowiki> tags. This is helpful for documenting a code example and supports many languages. Usage is simple. Here's an example of how to display some 'Hello World' source code examples:</span></span>
 
<pre><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><syntaxhighlight lang='python'>
 
# Hello World in Python
 
print("Hello World")
 
</syntaxhighlight>
 
<syntaxhighlight lang='csharp'>
 
//Hello World in C#
 
class HelloWorld
 
{
 
    static void Main()
 
    {
 
        System.Console.WriteLine("Hello, World!");
 
    }
 
}
 
 
 
</syntaxhighlight>
 
 
 
 
 
<syntaxhighlight lang='cpp'>
 
// Hello World in C++ (pre-ISO)
 
 
 
#include <iostream.h>
 
 
 
main()
 
{
 
    cout << "Hello World!" << endl;
 
    return 0;
 
}
 
 
 
 
 
 
 
</syntaxhighlight>
 
 
 
</span></span></pre>
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">The above code block results in:</span></span>
 
<div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="python source-python"><pre class="de1"><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><span class="co1"># Hello World in Python</span>
 
<span class="kw1">print</span><span class="br0">(</span><span class="st0">"Hello World"</span><span class="br0">)</span></span></span></pre></div></div><div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="csharp source-csharp"><pre class="de1"><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><span class="co1">//Hello World in C#</span>
 
</span></span></pre>
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">&lt;span class="kw4"&gt;class&lt;/span&gt; HelloWorld &lt;span class="br0"&gt;{&lt;/span&gt;</span></span>
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">  <span class="kw1">static</span> <span class="kw4">void</span> Main<span class="br0">(</span><span class="br0">)</span>
 
<span class="br0">{</span>
 
<span class="kw5">System</span><span class="sy0">.</span><span class="me1">Console</span><span class="sy0">.</span><span class="me1">WriteLine</span><span class="br0">(</span><span class="st0">"Hello, World!"</span><span class="br0">)</span><span class="sy0">;</span>
 
<span class="br0">}</span>
 
</span></span>
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">&lt;span class="br0"&gt;}&lt;/span&gt; &nbsp; C&lt;span class="sy0"&gt;++&lt;/span&gt;</span></span>
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Back to index</span></span></div></div><div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="cpp source-cpp"><pre class="de1"><span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><span class="co1">// Hello World in C++ (pre-ISO)</span>
 
</span></span></pre>
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">&nbsp; &lt;span class="co2"&gt;#include &lt;iostream.h&gt;&lt;/span&gt; &nbsp; main&lt;span class="br0"&gt;(&lt;/span&gt;&lt;span class="br0"&gt;)&lt;/span&gt; &lt;span class="br0"&gt;{&lt;/span&gt;</span></span>
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">  <span class="kw3">cout</span> <span class="sy1"><<</span> <span class="st0">"Hello World!"</span> <span class="sy1"><<</span> endl<span class="sy4">;</span>
 
<span class="kw1">return</span> <span class="nu0">0</span><span class="sy4">;</span>
 
</span></span>
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">&lt;span class="br0"&gt;}&lt;/span&gt;</span></span></div></div>
 
==== <span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Other Resources</span></span> ====
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">Consult the [http://meta.wikimedia.org/wiki/Help:Contents User's Guide] for information on using the wiki software. See Also:</span></span>
 
 
 
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[http://www.mediawiki.org/wiki/Manual:Configuration_settings Configuration settings list]</span></span>
 
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[http://www.mediawiki.org/wiki/Manual:FAQ MediaWiki FAQ]</span></span>
 
*<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;">[https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]</span></span>
 
 
 
<span style="font-family:trebuchet ms,helvetica,sans-serif;"><span style="font-size:small;"><br/>[[/Reference]]</span></span>
 

Latest revision as of 15:48, 5 December 2017

This Wiki is meant to provide a resource for those at UiO interested in working with computer modeling and data analysis in Earth Sciences.

Content herein is generally specific to UiO, as otherwise it is recommended to make posts to Stack Exchange and the sites therein.

This wiki uses 'category' tags, and below are some of the categories available. Note that in some cases, the content does not yet exist, in which case you are encouraged to edit and add information, or contact one of the site maintainers to request further information.

Purpose:

Provide information on software tools available for data analysis and scientific program for projects within the field of Geosciences in general and to provide users a cenral resource for keeping notes on UIO specific aspects of running various software routines. Ideally, this site will become overtime a 'go to' location where incoming students who will be working with modeling and heavy data analysis on servers can find tips for getting started.

Enjoy!

Editing pages
How to
Style guide
Data
Abbreviations_and_terminology
Formats
Measurements
Ordering
Satellite imagery
Re-analysis
GIS
Tools
WorkingOnServers
Command_line
Software
Code Coffee
R
Fortran
Micmac
Python
Matlab
Excel
Computing Facilities
How to login?
Wessel
Sverdrup
Moulin
geo-hiperf
Abel HPC
Notur HPC
Set-up your own server (IaaS)
Data storage (lagrinshotell, Norstore & storage for researchers)
Models
WRF, WRF-CHEM and WRF-Hydro
OpenIFS
Flexpart
MITgcm
cesm
noresm
ESys-Particle
SURFEX/Crocus
Field of study
Hydrology
Meteorology
Glaciology
Communication
Mail and Mailing lists
Presentations
Templates
Logo
Climabyte
Administration
Teaching
Summerschools