WorkingOnServers/DisplayEnvironmentTricks
Contents
Introduction
Using IPython via ssh with X-forwarding is a really great way to work, but it can be a pain with the DISPLAY settings. The tricks outlined below have been really helpful for me and are now a part of my standard 'workflow'. For the most part, I assume Ubuntu (or some 'deb' package OS variant) and bash in my descriptions below. YMMV.
A way to preserve your DISPLAY environment
When working via ssh (and X forwarding) on IPython, sometimes your $DISPLAY environment variable gets mangled. One approach to deal with this is outlined below, but recently I found a new approach.
There is a program called xpra which creates a virtual monitor (x server) to which you can point your DISPLAY variable.
You can start a screen session with the DISPLAY setting pointing to this virtual display. Then when you run IPython (inside screen) anything sent to X-display is sent to the virtual environment. You can attach and detach to this environment and your GUIs / matplotlib windows, etc will remain alive.
To install xpra:
sudo apt-get install xpra
Then a session:
First, start a screen session pointing to a virtual display
xpra start :100 DISPLAY=:100 screen
Now you've started screen and should be "inside" screen with all your X display sent to :100. You can start Ipython in the screen session:
ipython
In another terminal, attach to the xpra display to 'see' the X programs:
xpra attach :100
You can detach from both these sessions, and they'll keep running, get home or wherever... reattach to screen and also reattach to your xpra virtual display. Good luck!
Another approach for working with SSH and GNU screen (helpful for Ipython)
Often you'll want to ssh to a server, start a job and leave. Knowing that you want the job to stay alive after you log out, you can just use:
nohup myjob.sh > nohup_output.nh &
But sometimes you'd rather have an actual term or shell session stay alive. This is often the case when working with IPython. The problem is that your DISPLAY environment variable may change, so that when you log into the machine later, what screen thinks is the present DISPLAY variable will no longer be current and you'll get errors.
My workaround for this has been the following:
1 On the host machine, that is the one you are logging into, in your .bash_profile (which gets read when you log in via ssh) you should add the line:
echo "export DISPLAY=$DISPLAY" > .display.`whoami`.`hostname`
2 Again, on your host machine,make sure the following is in your .bashrc:
## set the prompt command to read the .disply file export PROMPT_COMMAND=". ~/.display.`whoami`.`hostname`"
What is happening here is that every time you log in, the .bash_profile file creates a new file in your home directory, specific to your user, and the hostname. Whenever you log out and log back in, that file is updated with the appropriate DISPLAY information. In the .bashrc file, we've created a PROMPT_COMMAND that will read that file every time you hit return in your shell. This can cause some annoyances if the file does not exist, for example if you use the 'su' command to become another user. However, overall it works quite well, such that when you return to a running screen session, the DISPLAY variable will be updated and you'll be able to send X-display windows back to your client machine.
A script for keeping track of screens
Often you may have several screen sessions running. It is recommended to start screen with the following command:
screen -S my_description
Where my_description is some string that has meaning to you as a 'session' identifier. If only one screen session is running then simply screen -r will reconnect. However, if you have more, things can get complicated. The following script is helpful in such a case:
<span class="sy0">!/</span>bin<span class="sy0">/</span><span class="kw2">bash</span>
<span class="co0">#filters the screen -ls output to show the sesssions sessions=`screen -ls | sed -ne 's/:space://' -ne 's/\((Attached)\|(Detached)\|(Multi, detached)\|(Multi, attached)\)// p'`</span>
<span class="kw3">echo</span> <span class="re1">$sessions</span> <span class="kw3">echo</span> <span class="re1">$sessions</span> <span class="sy0">|</span> <span class="kw2">wc</span> <span class="re5">-w</span> <span class="re2">res</span>=<span class="sy0">`</span><span class="kw3">echo</span> <span class="st0">"<span class="es2">$sessions</span>"</span> <span class="sy0">|</span> <span class="kw2">wc</span> -w<span class="sy0">`</span>
<span class="kw1">if</span> <span class="st0">"<span class="es2">$res</span>"</span><span class="sy0">&</span>nbsp;<span class="sy0">!</span>= <span class="st0">"0"</span> <span class="kw1">then</span>
<span class="kw3">echo</span>
<span class="kw3">echo</span> <span class="st0">" CURRENT SESSIONS"</span>
<span class="kw3">echo</span> <span class="st0">" ------------------------"</span>
<span class="co0">#screen -ls | sed -ne 's/:space://' -ne 's/\((Attached)\|(Detached)\)// p' | cat -n</span>
<span class="kw3">echo</span> <span class="st0">"<span class="es2">$sessions</span>"</span> <span class="sy0">|</span> <span class="kw2">cat</span> <span class="re5">-n</span>
<span class="kw3">echo</span> <span class="st0">" ------------------------"</span>
<span class="kw3">echo</span>
<span class="co0">#prompt for the session to join</span>
<span class="kw3">echo</span> <span class="re5">-n</span> <span class="st0">" Reattach to session, or ENTER for None: "</span>
<span class="kw2">read</span> session
<span class="kw1">if</span> <span class="re1">$session</span><span class="sy0">&</span>nbsp;<span class="sy0">!</span>= <span class="nu0">0</span>
<span class="kw1">then</span>
<span class="co0">#attach to specified session</span>
<span class="re2">linenum</span>=<span class="nu0">0</span>
<span class="re2">name</span>=<span class="sy0">`</span><span class="kw2">screen</span> <span class="re5">-ls</span> <span class="sy0">|</span> <span class="kw2">sed</span> <span class="re5">-ne</span> <span class="st_h">'s/:space://'</span> <span class="re5">-ne</span> <span class="st_h">'s/\((Attached)\|(Detached)\|(Multi, detached)\|(Multi, attached)\)// p'</span> <span class="sy0">|</span>
<span class="kw1">while</span> <span class="kw2">read</span> line
<span class="kw1">do</span>
<span class="kw3">let</span> <span class="st0">"linenum += 1"</span>
<span class="kw1">if</span> <span class="st0">"<span class="es2">$linenum</span>"</span> <span class="re5">-eq</span> <span class="st0">"<span class="es2">$session</span>"</span>
<span class="kw1">then</span>
<span class="kw3">echo</span> <span class="re1">$line</span>
<span class="kw3">break</span>
<span class="kw1">fi</span>
<span class="kw1">done</span><span class="sy0">`</span>
<span class="re2">shrtname</span>=<span class="sy0">`</span><span class="kw3">echo</span> <span class="re1">$name</span> <span class="sy0">|</span> <span class="kw2">sed</span> <span class="re5">-e</span> <span class="st_h">'s/:space://'</span> <span class="re5">-e</span> <span class="st_h">'s/(.*)//'</span> <span class="re5">-e</span> <span class="st_h">'s/\t//g'</span><span class="sy0">`</span>
<span class="kw3">echo</span> <span class="st0">"Reattaching to: "</span> <span class="re1">$shrtname</span>
<span class="kw1">if</span> <span class="st0">"<span class="es2">$name</span>"</span><span class="sy0">&</span>nbsp;<span class="sy0">!</span>= <span class="st0">""</span>; <span class="kw1">then</span>
<span class="kw1">if</span>
<span class="co0">#echo $shrtname</span>
<span class="kw2">screen</span> <span class="re5">-r</span> <span class="st0">"<span class="es2">$shrtname</span>"</span>
<span class="kw1">then</span>
<span class="kw3">echo</span> <span class="re5">-n</span> <span class="st0">" Bye bye screen.. Exit?[y] "</span>
<span class="kw2">read</span> nbye
<span class="kw1">if</span> <span class="st0">"<span class="es2">$nbye</span>"</span> == <span class="st0">""</span>
<span class="kw1">then</span>
<span class="kw3">exit</span>
<span class="kw1">else</span>
<span class="re2">goodbye</span>=<span class="re1">$nbye</span>
<span class="kw1">fi</span>
<span class="kw1">if</span> <span class="st0">"<span class="es2">$goodbye</span>"</span> == <span class="st0">"y"</span>
<span class="kw1">then</span>
<span class="kw3">exit</span>
<span class="kw1">else</span>
<span class="kw3">echo</span> <span class="st0">"What's next?"</span>
<span class="kw1">fi</span>
<span class="kw1">else</span>
<span class="kw1">if</span>
<span class="co0"># try multiuser connect</span>
<span class="kw2">screen</span> <span class="re5">-Ax</span> <span class="st0">"<span class="es2">$shrtname</span>"</span>
<span class="kw1">then</span>
<span class="kw3">echo</span> <span class="st0">"Hope that was fun!"</span>
<span class="kw1">else</span>
<span class="kw3">echo</span> <span class="st0">"Trying to Force connection"</span>
<span class="kw2">screen</span> <span class="re5">-D</span> <span class="re5">-r</span> <span class="st0">"<span class="es2">$shrtname</span>"</span>
<span class="kw1">fi</span>
<span class="kw1">fi</span>
<span class="kw1">else</span>
<span class="kw3">echo</span> <span class="st0">"Could not reattach to <span class="es2">$session</span>"</span>
<span class="kw1">fi</span>
<span class="kw1">else</span>
<span class="kw3">echo</span> <span class="st0">" Have fun..."</span>
<span class="kw1">fi</span>
<span class="kw1">else</span>
<span class="kw3">echo</span> <span class="st0">" No existing SCREEN session to reattach to..."</span>
<span class="kw1">fi</span>
Save this file in your $HOME/bin directory as 'screenlist' and make it executable: chmod +x screenlist. Then at the command prompt you can type:
[user@host ~]$screenlist
and you should get some output similar to:
CURRENT SESSIONS
------------------------
1 3841.coding (09/26/2011 11:35:01 PM)
2 3791.Ipython (09/26/2011 11:34:42 PM)
3 17716.test3 (09/26/2011 10:12:15 PM)
------------------------
Reattach to session, or ENTER for None:
