Difference between revisions of "Python"

From mn/geo/geoit
Jump to: navigation, search
(Import Statements & Namespaces)
Line 1: Line 1:
 
The Python programming language is a powerful language which has gained popularity over the past several years. It is widely used in the scientific community, and there are numerous tools readily available for common data analysis tasks. At UiO there is a repository for ''Modules'' developed internally. The pages herein provide important information on working with Python on the UiO servers and also with your own PC.
 
The Python programming language is a powerful language which has gained popularity over the past several years. It is widely used in the scientific community, and there are numerous tools readily available for common data analysis tasks. At UiO there is a repository for ''Modules'' developed internally. The pages herein provide important information on working with Python on the UiO servers and also with your own PC.
  
==Key Topics / Tips for Python at UiO==
+
== Key Topics / Tips for Python at UiO ==
  
====Python on the Servers====
+
==== Python on the Servers ====
By Anne Fouilloux
+
 
Published May 22, 2014 09:59 AM - Last modified May 22, 2014 10:17 AM
+
By Anne Fouilloux Published May 22, 2014 09:59 AM - Last modified May 22, 2014 10:17 AM
  
 
Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++.
 
Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++.
Line 11: Line 11:
 
UiO Geosciences IT provides users with two different python distributions to work with:
 
UiO Geosciences IT provides users with two different python distributions to work with:
  
* [https://www.enthought.com/products/epd/ '''Enthought Python Distribution''']
+
*[https://www.enthought.com/products/epd/ <b>Enthought Python Distribution</b>]
* [http://docs.continuum.io/anaconda/index.html '''Anaconda - Continuum Analytics''']
+
*[http://docs.continuum.io/anaconda/index.html <b>Anaconda - Continuum Analytics</b>]
  
Both are available on:
+
Both are available on: Red Hat 6 linux servers and desktops, use e.g sverdrup.uio.no
Red Hat 6 linux servers and desktops, use e.g sverdrup.uio.no
 
  
 
How to start:
 
How to start:
Line 23: Line 22:
 
   module load python/canopy
 
   module load python/canopy
 
   
 
   
 +
 
''or for Anaconda Python Distribution:''
 
''or for Anaconda Python Distribution:''
  
 
Before using Anaconda Python Distribution, you would need to get your own academic license. Academic licenses are free and can be generated for/by each user from:
 
Before using Anaconda Python Distribution, you would need to get your own academic license. Academic licenses are free and can be generated for/by each user from:
* [https://store.continuum.io/cshop/academicanaconda '''Anaconda''']  
+
 
+
*[https://store.continuum.io/cshop/academicanaconda <b>Anaconda</b>]
Once you receive the license file just place it in the $HOME/.continuum folder. Don't forget "." in front of the folder name (this folder is hidden); if it does not exist, you can create it:
+
 
+
Once you receive the license file just place it in the $HOME/.continuum folder. Don't forget "." in front of the folder name (this folder is hidden); if it does not exist, you can create it:
 +
 
 
   mkdir $HOME/.continuum
 
   mkdir $HOME/.continuum
 
   
 
   
The email accompanying the license will have more detailed instructions. You may also reference the [http://docs.continuum.io/anaconda/install.html ''online anaconda documentation''] for further detail.  
+
 
+
The email accompanying the license will have more detailed instructions. You may also reference the [http://docs.continuum.io/anaconda/install.html <i>online anaconda documentation</i>] for further detail.
 +
 
 
Then to set-up your environment:
 
Then to set-up your environment:
+
 
 
   module load python/anaconda
 
   module load python/anaconda
 
   
 
   
If you have problems with the module command [http://www.mn.uio.no/geo/english/services/it/help/software/user-environment.html#Troubleshooting ''please follow this link''].
+
 
 +
If you have problems with the module command [http://www.mn.uio.no/geo/english/services/it/help/software/user-environment.html#Troubleshooting <i>please follow this link</i>].
  
 
Then for any of these two distributions:
 
Then for any of these two distributions:
Line 44: Line 47:
 
   python
 
   python
 
   
 
   
At the command line.
+
 
By Anne Fouilloux
+
At the command line. By Anne Fouilloux Published May 22, 2014 09:59 AM - Last modified May 22, 2014 10:17 AM
Published May 22, 2014 09:59 AM - Last modified May 22, 2014 10:17 AM
 
  
 
This link: [[http://www.mn.uio.no/geo/english/services/it/help/software/meteorology/python.html PYTHON]] is the source guide from the Geoscience IT for the above documentation.
 
This link: [[http://www.mn.uio.no/geo/english/services/it/help/software/meteorology/python.html PYTHON]] is the source guide from the Geoscience IT for the above documentation.
Line 58: Line 60:
 
The '-XC' tells ssh to forward 'X' connections and to use 'C'ompression. For more details see [[WorkingOnServers/JFBsWorkFlow|JFBsWorkFlow]]
 
The '-XC' tells ssh to forward 'X' connections and to use 'C'ompression. For more details see [[WorkingOnServers/JFBsWorkFlow|JFBsWorkFlow]]
  
+
<br/>'''***IMPORTANT***'''
'''***IMPORTANT***'''
 
  
 
Once you have connected to the servers, in order to use Python it is important to know where the modules are, and how to set up your environment. Many of the commonly used modules are available already, and installed in a directory that you need to add to your <code>PYTHONPATH</code> in order to use.
 
Once you have connected to the servers, in order to use Python it is important to know where the modules are, and how to set up your environment. Many of the commonly used modules are available already, and installed in a directory that you need to add to your <code>PYTHONPATH</code> in order to use.
  
 
For more information see:
 
For more information see:
* [[/SettingPaths]]
 
  
==Resources==
+
*[[/SettingPaths]]
 +
 
 +
== Resources ==
  
 
[[/Resources]]
 
[[/Resources]]
  
  
==Best Practices==
 
  
Let's start with some 'best practices' for programming with Python.  
+
== Best Practices ==
 +
 
 +
Let's start with some 'best practices' for programming with Python.
 +
 
 +
=== Code Style ===
  
===Code Style===
 
 
First and foremost, one should become familiar with the [http://www.python.org/dev/peps/pep-0008/ PEP8]. This is a 'Python Enhancement Proposal' (PEP). In Python this is the equivalent of a detailed featured request. [http://www.python.org/dev/peps/pep-0008/ PEP8] lays the foundation for how you should format your code.
 
First and foremost, one should become familiar with the [http://www.python.org/dev/peps/pep-0008/ PEP8]. This is a 'Python Enhancement Proposal' (PEP). In Python this is the equivalent of a detailed featured request. [http://www.python.org/dev/peps/pep-0008/ PEP8] lays the foundation for how you should format your code.
  
 
One important feature of python to highlight are '''Documentation Strings''':
 
One important feature of python to highlight are '''Documentation Strings''':
<syntaxhighlight lang='text'>
+
<div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="text source-text"><pre class="de1">   Conventions for writing good documentation strings (a.k.a. "docstrings")
 
 
    Conventions for writing good documentation strings (a.k.a. "docstrings")
 
 
     are immortalized in PEP 257 [3].
 
     are immortalized in PEP 257 [3].
 
+
&nbsp;
 
     - Write docstrings for all public modules, functions, classes, and
 
     - Write docstrings for all public modules, functions, classes, and
 
       methods.  Docstrings are not necessary for non-public methods, but you
 
       methods.  Docstrings are not necessary for non-public methods, but you
 
       should have a comment that describes what the method does.  This comment
 
       should have a comment that describes what the method does.  This comment
 
       should appear after the "def" line.
 
       should appear after the "def" line.
 
+
&nbsp;
 
     - PEP 257 describes good docstring conventions.  Note that most
 
     - PEP 257 describes good docstring conventions.  Note that most
 
       importantly, the """ that ends a multiline docstring should be on a line
 
       importantly, the """ that ends a multiline docstring should be on a line
 
       by itself, and preferably preceded by a blank line, e.g.:
 
       by itself, and preferably preceded by a blank line, e.g.:
 
+
&nbsp;
 
       """Return a foobang
 
       """Return a foobang
 
+
&nbsp;
 
       Optional plotz says to frobnicate the bizbaz first.
 
       Optional plotz says to frobnicate the bizbaz first.
 
+
&nbsp;
 
       """
 
       """
 
+
&nbsp;
 
     - For one liner docstrings, it's okay to keep the closing """ on the same
 
     - For one liner docstrings, it's okay to keep the closing """ on the same
       line.
+
       line.</pre></div></div>
 
+
=== Import Statements & Namespaces ===
</syntaxhighlight>
 
 
 
===Import Statements & Namespaces===
 
  
 
In python, almost all code will be preceded with some statements such as: <code>import this</code> (try that, by the way). These are 'import statements' which pull in other modules to use within your own code.
 
In python, almost all code will be preceded with some statements such as: <code>import this</code> (try that, by the way). These are 'import statements' which pull in other modules to use within your own code.
  
 
You may find some examples, particularly when learning [http://matplotlib.sourceforge.net/ matplotlib] and working with 'pylab', where the tutorial uses:
 
You may find some examples, particularly when learning [http://matplotlib.sourceforge.net/ matplotlib] and working with 'pylab', where the tutorial uses:
 
+
<div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="python source-python"><pre class="de1"><span class="kw1">from</span> pylab <span class="kw1">import</span> *</pre></div></div>
<syntaxhighlight lang='python'>
+
.
from pylab import *
 
</syntaxhighlight >.
 
  
 
This is '''NOT''' recommended. This will ''contaminate'' your local namespace, meaning that the module from which you import ''everything'' indicated by the '*', may overwrite some of your own functions or even python builtins.
 
This is '''NOT''' recommended. This will ''contaminate'' your local namespace, meaning that the module from which you import ''everything'' indicated by the '*', may overwrite some of your own functions or even python builtins.
  
 
The '''RECOMMENDED''' way to import modules is to:
 
The '''RECOMMENDED''' way to import modules is to:
<syntaxhighlight lang='python'>
+
<div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="python source-python"><pre class="de1"><span class="kw1">import</span> pylab <span class="kw1">as</span> plb</pre></div></div>
import pylab as plb
 
</syntaxhighlight >
 
 
Then you can still use the example, but you may simply need to prefix some of the function calls with 'plb.', which refers to the 'pylab' namespace.
 
Then you can still use the example, but you may simply need to prefix some of the function calls with 'plb.', which refers to the 'pylab' namespace.
  
====Specific for Matplotlib====
+
==== Specific for Matplotlib ====
 +
 
 
There will be some pages dedicated to Matplotlib, but in the meantime, the 'best practice' for working with Matplotlib is to use the 'object oriented' approach:
 
There will be some pages dedicated to Matplotlib, but in the meantime, the 'best practice' for working with Matplotlib is to use the 'object oriented' approach:
 +
<div dir="ltr" class="mw-geshi mw-code mw-content-ltr"><div class="python source-python"><pre class="de1"><span class="kw1">import</span> matplotlib.<span class="me1">pyplot</span> <span class="kw1">as</span> plt
 +
<span class="kw1">import</span> numpy <span class="kw1">as</span> np
 +
&nbsp;
 +
x <span class="sy0">=</span> np.<span class="me1">arange</span><span class="br0">(</span><span class="nu0">100</span><span class="br0">)</span>
 +
y <span class="sy0">=</span> np.<span class="me1">sin</span><span class="br0">(</span>x<span class="br0">)</span>
 +
&nbsp;
 +
fig <span class="sy0">=</span> plt.<span class="me1">figure</span><span class="br0">(</span><span class="br0">)</span>
 +
ax <span class="sy0">=</span> fig.<span class="me1">add_subplot</span><span class="br0">(</span><span class="nu0">111</span><span class="br0">)</span>
 +
ax.<span class="me1">plot</span><span class="br0">(</span>x<span class="sy0">,</span> y<span class="br0">)</span>
 +
&nbsp;
 +
plt.<span class="me1">show</span><span class="br0">(</span><span class="br0">)</span></pre></div></div>
 +
Here we have shown an example of both the import of matplotlib.pyplot into the 'plt.' namespace, and numpy into the 'np.' namespace. For code at NILU this is the recommended import style.
  
<syntaxhighlight lang='python'>
+
<br/>Particularly for modules you intend to develop at NILU, this is important.
import matplotlib.pyplot as plt
 
import numpy as np
 
 
 
x = np.arange(100)
 
y = np.sin(x)
 
 
 
fig = plt.figure()
 
ax = fig.add_subplot(111)
 
ax.plot(x, y)
 
 
 
plt.show()
 
</syntaxhighlight>
 
 
 
Here we have shown an example of both the import of matplotlib.pyplot into the 'plt.' namespace, and numpy into the 'np.' namespace. For code at NILU this is the recommended import style.
 
  
 +
== Useful Software / Modules ==
  
Particularly for modules you intend to develop at NILU, this is important.
+
*[[/Sphinx]]
 +
*[[/Numpy]]
 +
*[[/Scipy]]
 +
*[[/Matplotlib]]
 +
*[[/Ipython]]
 +
*[[/netcdf4-python]]
 +
*[[/pygrib]]
 +
*[[/pflexpart]]
 +
*[[/netrc]]
 +
*[[Plone]] -- Old notes for reference
  
==Useful Software / Modules==
+
== Sub Pages ==
  
* [[/Sphinx]]
+
&lt;subpages /&gt;
* [[/Numpy]]
 
* [[/Scipy]]
 
* [[/Matplotlib]]
 
* [[/Ipython]]
 
* [[/netcdf4-python]]
 
* [[/pygrib]]
 
* [[/pflexpart]]
 
* [[/netrc]]
 
* [[ Plone ]] -- Old notes for reference
 
  
==Sub Pages==
+
[[Category:Python]]
<subpages />
 

Revision as of 16:02, 30 January 2015

The Python programming language is a powerful language which has gained popularity over the past several years. It is widely used in the scientific community, and there are numerous tools readily available for common data analysis tasks. At UiO there is a repository for Modules developed internally. The pages herein provide important information on working with Python on the UiO servers and also with your own PC.

Key Topics / Tips for Python at UiO

Python on the Servers

By Anne Fouilloux Published May 22, 2014 09:59 AM - Last modified May 22, 2014 10:17 AM

Python is an interpreted, interactive, object-oriented programming language. It incorporates modules, exceptions, dynamic typing, very high level dynamic data types, and classes. Python combines remarkable power with very clear syntax. It has interfaces to many system calls and libraries, as well as to various window systems, and is extensible in C or C++.

UiO Geosciences IT provides users with two different python distributions to work with:

Both are available on: Red Hat 6 linux servers and desktops, use e.g sverdrup.uio.no

How to start:

If you wish to use Enthought Python Distribution:

 module load python/canopy

or for Anaconda Python Distribution:

Before using Anaconda Python Distribution, you would need to get your own academic license. Academic licenses are free and can be generated for/by each user from:

Once you receive the license file just place it in the $HOME/.continuum folder. Don't forget "." in front of the folder name (this folder is hidden); if it does not exist, you can create it:

 mkdir $HOME/.continuum

The email accompanying the license will have more detailed instructions. You may also reference the online anaconda documentation for further detail.

Then to set-up your environment:

 module load python/anaconda

If you have problems with the module command please follow this link.

Then for any of these two distributions:

 python

At the command line. By Anne Fouilloux Published May 22, 2014 09:59 AM - Last modified May 22, 2014 10:17 AM

This link: [PYTHON] is the source guide from the Geoscience IT for the above documentation.

***IMPORTANT***

It is critical that you're display variable gets set properly when you ssh to the servers. In order to do this you need to make sure you connect with an ssh program that will forward your 'X' connections, on windows this means you'll need to have a X-server also running locally. XmingPortablePutty will take care of this for you. From linux it is simply a matter of:

  ssh -XC user@remote

The '-XC' tells ssh to forward 'X' connections and to use 'C'ompression. For more details see JFBsWorkFlow


***IMPORTANT***

Once you have connected to the servers, in order to use Python it is important to know where the modules are, and how to set up your environment. Many of the commonly used modules are available already, and installed in a directory that you need to add to your PYTHONPATH in order to use.

For more information see:

Resources

/Resources


Best Practices

Let's start with some 'best practices' for programming with Python.

Code Style

First and foremost, one should become familiar with the PEP8. This is a 'Python Enhancement Proposal' (PEP). In Python this is the equivalent of a detailed featured request. PEP8 lays the foundation for how you should format your code.

One important feature of python to highlight are Documentation Strings:

    Conventions for writing good documentation strings (a.k.a. "docstrings")
    are immortalized in PEP 257 [3].
 
    - Write docstrings for all public modules, functions, classes, and
      methods.  Docstrings are not necessary for non-public methods, but you
      should have a comment that describes what the method does.  This comment
      should appear after the "def" line.
 
    - PEP 257 describes good docstring conventions.  Note that most
      importantly, the """ that ends a multiline docstring should be on a line
      by itself, and preferably preceded by a blank line, e.g.:
 
      """Return a foobang
 
      Optional plotz says to frobnicate the bizbaz first.
 
      """
 
    - For one liner docstrings, it's okay to keep the closing """ on the same
      line.

Import Statements & Namespaces

In python, almost all code will be preceded with some statements such as: import this (try that, by the way). These are 'import statements' which pull in other modules to use within your own code.

You may find some examples, particularly when learning matplotlib and working with 'pylab', where the tutorial uses:

<span class="kw1">from</span> pylab <span class="kw1">import</span> *

.

This is NOT recommended. This will contaminate your local namespace, meaning that the module from which you import everything indicated by the '*', may overwrite some of your own functions or even python builtins.

The RECOMMENDED way to import modules is to:

<span class="kw1">import</span> pylab <span class="kw1">as</span> plb

Then you can still use the example, but you may simply need to prefix some of the function calls with 'plb.', which refers to the 'pylab' namespace.

Specific for Matplotlib

There will be some pages dedicated to Matplotlib, but in the meantime, the 'best practice' for working with Matplotlib is to use the 'object oriented' approach:

<span class="kw1">import</span> matplotlib.<span class="me1">pyplot</span> <span class="kw1">as</span> plt
<span class="kw1">import</span> numpy <span class="kw1">as</span> np
 
x <span class="sy0">=</span> np.<span class="me1">arange</span><span class="br0">(</span><span class="nu0">100</span><span class="br0">)</span>
y <span class="sy0">=</span> np.<span class="me1">sin</span><span class="br0">(</span>x<span class="br0">)</span>
 
fig <span class="sy0">=</span> plt.<span class="me1">figure</span><span class="br0">(</span><span class="br0">)</span>
ax <span class="sy0">=</span> fig.<span class="me1">add_subplot</span><span class="br0">(</span><span class="nu0">111</span><span class="br0">)</span>
ax.<span class="me1">plot</span><span class="br0">(</span>x<span class="sy0">,</span> y<span class="br0">)</span>
 
plt.<span class="me1">show</span><span class="br0">(</span><span class="br0">)</span>

Here we have shown an example of both the import of matplotlib.pyplot into the 'plt.' namespace, and numpy into the 'np.' namespace. For code at NILU this is the recommended import style.


Particularly for modules you intend to develop at NILU, this is important.

Useful Software / Modules

Sub Pages

<subpages />