Difference between revisions of "Biotin2.hpc.uio.no"

From medicin.ncmm.IT
Jump to: navigation, search
Line 33: Line 33:
| [https://www.r-project.org/ R] with [https://www.bioconductor.org/ Bioconductor]    || R version 3.5.1 (2018-07-02) "Feather Spray" (3.5.1 is not in active maintenance )<br> R version 3.6.1 (2019-07-05) "Action of the Toes"
| [https://www.r-project.org/ R] with [https://www.bioconductor.org/ Bioconductor]    || R version 3.5.1 (2018-07-02) "Feather Spray" (3.5.1 is not in active maintenance )<br> R version 3.6.1 (2019-07-05) "Action of the Toes"
R version 4.0.2 (2020-04-02) "Taking Off Again"<br>see also <code>'''module list'''</code>                                                 
R version 4.0.2 (2020-04-02) "Taking Off Again"<br>see also <code>'''module list'''</code>                                                 
| Done  || - <code>'''module load R/3.6.1'''</code>- <code>'''module load R/4.0.2'''</code>   
| Done  || To access R v. 3.6.1, type:
- <code>'''module load R/3.6.1'''</code>
To access R v 4.0.2, type:
- <code>'''module load R/4.0.2'''</code>   

Revision as of 16:53, 2 November 2020

biotin2 is the departamental general usage server for the 'in silico' groups.

biotin3 is the Mathelier group server for their exclusive use.

Live computational resource usage information (cpu and memory) - Biotin2

Live computational resource usage information (cpu and memory) - Biotin3

Description of the biotin[2,3].hpc.uio.no servers and associated VMs with replicated applications

  • biotin2: 128 Intel Haswell cores, 1 Tb RAM, 162 Tb of local disk space allocated for various NCMM groups
  • biotin3: 192 Intel Broadwell cores, 1 Tb RAM, 5 Tb of local disk space allocated as scratch and local partitions for the Mathelier Group.
  • kuijjer2: 24 AMD EPYC virtual cores, 128 Gb RAM, 5 Tb of local disk space for the Kuijjer Group
  • waszak1: 48 AMD EPYC virtual cores, 128 Gb RAM, 4 Tb of local disk space for the Waszak Group

List of software installed on the biotin[2,3].hpc.uio.no servers and VMs

Application Version on biotin2.hpc.uio.no and biotin3.hpc.uio.no Status Comments
MATLAB R2019b ( 64-bit (glnxa64) Done University of Oslo Site License
R with Bioconductor R version 3.5.1 (2018-07-02) "Feather Spray" (3.5.1 is not in active maintenance )
R version 3.6.1 (2019-07-05) "Action of the Toes"

R version 4.0.2 (2020-04-02) "Taking Off Again"
see also module list

Done To access R v. 3.6.1, type:

- module load R/3.6.1

To access R v 4.0.2, type:

- module load R/4.0.2

Perl v5.16.3 Done Upgraded version should not cause any problems
Perl 6 6c Done Based on Rakudo Star release 17.4
Python Python 2.7.5, as well as 3,4,2, 3.5.1, 3.6.9 and 3.8.0 from RedHat Software Collections plus installed scientific computing modules. For a list of Python modules we maintain, please visit this page. Done  To access the default Python 2.7 environment, you need to type nothing.

Please note the End Of Life (EOL) of Python 2.7.x . Make an effort to use Python 3.x instead, as described below.

To enable the Python 3 version, please type on your shell

-For version 3.4.2:

scl enable rh-python34 bash

-For version 3.5.1:

scl enable rh-python35 bash

-For version 3.6.9:

scl enable rh-python36 bash

-For version 3.8.0:

scl enable rh-python38 bash

ataqv 1.0.0 Done ataqv was built against the HTSLIB in /lsc/htslib and with gcc 8.2.1 (devtoolset-8)
bamliquidator Latest commit c2c4865, Oct 3, 2017 Done Compiled against HDF5 lib version 1.8.22
BGSignature 0.2 Done Installed with Python 3.6, so to use it, do a:

scl enable rh-python36 bash

BiasAway norpy version Done -
Bismark v0.16.3_dev Done -
bowtie version 1.1.2 64-bit Done Old bowtie genomic indices were transferred from biotin in /lsc/bowtie1ind
bowtie2 version 64-bit Done Old bowtie2 genomic indices were transferred from biotin in /lsc/bowtie2ind
bedops version 2.4.14 Done -
bsExpress v0.5cegx Done The sofware was modified/fixed to run properly on our production environment. Patches are described in UiO RT case #2444978.
bsmap version 2.74 Done -
bwa Version: 0.7.12-r1039 Done -
bwa-meth Version 0.2.0 Done -
blast (NCBI)  2.2.30+ Done Blast DBs were transferred from biotin in /lsc/blastdbs
cta 0.1.2 Done Compiled with gcc 8.2.1 (devtoolset-8). It does not compile gcc 4.x .
chimera 1.15 Done - .
datamash 1.1.1 Done -
eman 2.31 Done source /lsc/eman/2.31/etc/profile.d/conda.sh WARNING: PYTHONPATH set, might interfere with eman2
DNAshapedTFBS Latest commit 418fff6 on 28 Sep 2016 Done -
samtools 1.3.1 Done Samtools is used by certain applications as a dependency. Some of these applications might require older versions of the samtools. These cases will be dealt on a "as needed" basis.
tophat and tophat2 v2.1.0 Done -
FastQ Screen v0.5.2 Done -
fastQC v0.11.4 Done -
gcc 6 compiler v6.3.1 Done Default compiler for RHEL 7 is gcc 4.8.x. We include as well gcc version 6. To use type at your shell prompt: scl enable devtoolset-6 bash
gcc 7 compiler v7.3.1 Done Default compiler for RHEL 7 is gcc 4.8.x. We include as well gcc version 7. To use type at your shell prompt: scl enable devtoolset-7 bash
gcc 8 compiler v8.2.1 Done To use type at your shell prompt: scl enable devtoolset-8 bash
Expression analysis ea-utils github repo 31/10/2017 Done -
edd v1.1.14 Done This is a new installation of the latest Enriched Domain Detector (EDD) software  based on the system wide installed Python 2.7.5 and the following dependency component versions Cython-0.23.4 numpy-1.10.1 pysam-0.8.3 scipy-0.16.1 pandas-0.17.0 patsy-0.4.0 and statsmodels-0.6.1. Biotin used to run this for specific users using manually compiled Python 2.7.9. Recommended to re-run some data to check if the same results are achieved prior using the software, as a precaution.
EMBOSS v6.6.0.0 Done -
BroadPeak - Done Depended on R and Perl versions. Working.
cufflinks v2.2.1 Done Installed against the RHEL7 dependency libraries boost version 1.53, eigen3-devel version 3.2.5, boost-bjam
Homer Tools v4.7.2 Done Configurable by the end users, installed genomes hg(17-19),mm(8-10),xenTro(2,3),ce(6,10),danRer7, galGal4,rn5
meme v.4.11.4 Done No sequence databases were downloaded on biotin, hence no databases on biotin2
ViennaRNA v2.1.9 Done -
F-Seq v1.8.4 Done In biotin2 we have JAVAOPTS="-Xmx16000M" as opposed to JAVAOPTS="-Xmx8000M"  of biotin (twice the heap size) configued in fseq
WEKA v3.7.13 Done Type launch_weka to start the GUI of the program. Check the functionality against Java 1.8 . Heap size is set to 4 Gig. (4 x biotin size)
IGV v2.3.88 Done Type igv.sh to start the browser. Expect long startup times and responsiveness issues, depending on the size of your genome files. This program is best executed on local workstations rather than a shared server.
iced v0.5.1 Done -
novoalign v3.03.01 Done -
cutadapt v1.12 Done One of the many dependencies for iMir
FASTA v36.3.8b Oct, 2015 Done -
HiC-Pro v2.11.0-beta Done Compiled for python 2.7.5 against bx-python 0.8.2, numpy 1.14.2, scipy 1.0.1
Hisat v2.1.0 Done Compiled locally with gcc version 7.3.1-5
iMir - Done Type launch_iMir to start the GUI
Intervene v0.5.6 git commit 22/02/2017 Done -
Lux latest commit 15/03/2016 Done Based on STAR version 2.11.0 which is also installed on the server.
macs14 v1.4.2 20120305 Done -
macs2 v2.1.0.20150731 Done -
MethPipe v3.4.2 Done -
Methy-Pipe v2.1 Done -
mixcr v2.1.5 Done -
ngs.plot v2.61 Done -
oPOSSUM3 Latest git commit 01/2016 Done RDBMS that holds the JASPER and other TFBS databases on hfaistos.uio.no. Biotin2.hpc only has the client programs. See UiO RT 235025.
GNU Octave v4.4.1 Done Dependencies on HDF5 and SuiteSparse. Built from source.
preseq v2.0.1 Done Tarball from source reports version 2.0.1, yet the tool reports version 2.0.0. Version error from the authors.
pyega3 pyEGA3 - EGA python client version 3.0.44 Done To use the pyega3, you need to load the Python 36 client by typing first:

scl enable rh-python36 bash

RSAT opting for the latest github version as dictated by the NCMM Computational Biology Group Done Command line version RSAT tools reference their own versions of NCBI BLAST and vmatch tools
StringTie v1.3.4d Done Compiled locally with gcc version 7.3.1-5
subread v1.6.3 Done Associated to this is the Rsubread Bioconductor package.
velvet v1.2.10 Done -
vmatch v2.3.0 Done Usually required for the RSAT tool, but offered as a standalone utility outside the RSAT program PATH.
ViewBS git repo date 2/11/2018 Done -
NCBI SRA Toolkit (sra-tools) v2.5.4.1 Done -
ChIPseeqer v2.0 Done Observed many compiler warnings during the build. Users should clarify whether version 2.1 should be installed. The software looks old and unmaintained.
XDS BUILT=20180319 Done KMP_STACKSIZE=8m  is exported by default on the shell. If you need to increase that, please raise an RT ticket
PHENIX v1.10.1-2155 Done -
CCP4 v7.1.003 Done Users need to manually do: source /lsc/ccp4/7.1.003/bin/ccp4.setup-sh prior using the package due to numpy conflicts with other applications.
bedtools v2.25.0 ; v2.29.2 Done to use, please run, e.g., module load bedtools/2.29.2
picard v1.140 Done Based on HTSJDK v1.140
circos v0.68 Done -
The Genome Analysis ToolKit (GATK) v3.4-46-gbc02625 Done -
HTSeq v0.6.1 Done -
Perl TFBS v0.7.0 Done The Perl TFBS module is installed but is not in active development. Instead, we have installed the TFBStools (R/Bioconductor).
preseq v2.0.1 Done -
pyDNase github repo clone 30/05/2017 Done -
RSEM v1.3.0 Done -
STAR v2.5.3a Done -
CEAS v1.0.2 Done -
Qualimap v2.2.1 Done linked to the Sun JAVA JRE (/lsc/sources/jre1.7.0_80/bin/java). All R dependencies installed.
Quest v.2.4 Done -
SICER v1.1 Done -
velocyto 0.17.17 Done Compiled against Python 3.6.9, so to initialize it properly do:

scl enable rh-python36 bash

To initialize the R module properly, do:

scl enable rh-python36 bash

module load R/3.6.1

XGBoost Latest github commit on 06/04/2017 Done Installed Python 2.7.x module, command line compiled utility and R module.
HMMRATAC v1.2.9 Done module load hmmratac/1.2.9; hmmratac (aliased to "/usr/bin/java -jar /$lsc/$package/$version/$package")
Genrich v0.6 Done module load genrich/0.6; genrich
DAStk Latest github commit on 2020--01-14 Done scl enable rh-python36
MultiQC 0.2.3 Done on biotin2: scl enable rh-python36; conda activate /lsc/multiqc/1.8 on biotin3: scl enable rh-python36; source activate /lsc/multiqc/1.8 (older version of conda)

Top Directory ( /lsc )

Top directory for all installed software is /lsc

How are applications are run on biotin2/3

R 3.6.1

R 3.6.1 is run as a module.

In order to use 3.6.1 type

   module load R/3.6.1

R 3.6.1, /usr/bin/screen and RedHat Software Collections (/usr/bin/scl)

Be careful when using the RedHat Software Collections: /usr/bin/scl resets your user environment, because it re-reads your .login/.bash_login/.profile/.bash_profile when it starts up (as it executes as a login shell)

/usr/bin/screen does the same thing: it re-reads your .login/.bash_login/.profile/.bash_profile and, therefore it resets your environment.

should you want to use modules, screen and the RedHat Software collection, the correct order is to:

  1. login in bioin2/3
  2. run /usr/bin/screen
  3. then scl rh-example8 enable bash
  4. finally, module load R/3.6.1