Available Software

All users have access to CCR's new environment of software and compute nodes. See here for more information

CCR maintains a suite of software programs, libraries, and toolchains commonly used in scientific computing. The HPC clusters can execute most software that runs under Linux. In many cases, the software you need will already be installed and available to you on the compute nodes. You access the software using what's called a "module". If the software you need is not available, you can ask our staff to install it for you or do it yourself.

Software Module Transition Required

Existing users can migrate to the new modules simply by running this command: touch ~/.ccr_new_modules. You will need to logout and log back in for the changes to take effect. As of August 2023, new users will automatically be setup in this environment and can skip this step. If you'd like to temporarily go back to using the old modules in /util, simply remove that file rm ~/.ccr_new_modules. However, the old modules will be decommissioned this fall. See here for more information on using the new software on CCR's new compute nodes, and what to consider if you're a user of the faculty cluster.

We highly recommend you start by watching this presentation on CCR's new software modules and how to utilize the new environment:

Using Modules

Modules are simply configuration files that dynamically change your environment allowing you to run a particular application or provide access to a particular library. The module system allows CCR to provide multiple versions of software concurrently and enables users to easily switch between different versions without conflicts. Users can also build and maintain their own modules for more fine grained control over your groups environment.

Modules are managed using a tool called Lmod developed by TACC. For a more comprehensive overview of Lmod you can read the user guide here.

To see what software modules are available, ssh into the login node and run:

$ module avail

This will return a list of modules available to load into your environment.

Note

If a module has dependencies you may not see the module listed until dependencies are loaded. For example, software compiled with the gcc toolchain will not be shown until you load gcc or foss modules. See the hierarchical modules section for more information.

If you cannot load a module because of dependencies, you can use the module spider command to find what dependencies you need to load the module:

module spider some_module

# For example, openmpi requires gcc:
$ module spider openmpi
    You will need to load all module(s) on any one of the lines below before 
    the "openmpi/4.1.1" module is available to load.

      gcc/11.2.0

To load your chosen modules into the environment run:

$ module load some_module_name

# For example:
$ module load python

You can specify the version of the software by appending a / with the version number:

module load some_module/version 

# For example:
$ module load python/3.9.5

Hierarchical Modules

CCR uses a hierarchical module naming convention to support programs built with specific compiler, library, and CPU architecture requirements. For example, when you run module avail on an Intel Skylake compute node (avx512), you will see three hierarchical levels of modules that look like this:

---------------------------- MPI-dependent avx512 modules -----------------------------

-------------------------- Compiler-dependent avx512 modules --------------------------

------------------------------------ Core modules -------------------------------------
Core modules
compiler and architecture independent
Compiler-dependent modules
depend on a specific CPU architecture and compiler toolchain
MPI-dependent modules
depend on a specific CPU architecture, compiler toolchain, and MPI library

Only core modules are shown by default

When you run module avail only the core modules are shown by default. To see modules for a specific compiler toolchain you will need to first load the module for the specific compiler toolchain of interest.

CCR supports the following compiler toolchains:

Toolchain Included compilers and libraries
intel Intel compilers, Intel MPI, and Intel MKL
foss GCC, OpenMPI, FlexiBLAS, OpenBLAS, LAPACK, ScaLAPACK, FFTW
GCC GCC compiler only

In addition to the compiler toolchains, the hierarchical module system is also "CPU architecture" aware. The module system will auto-detect what type of CPU you're running on and display modules built for that specific architecture.

CCR supports the following CPU architectures:

Architecture Supported CPUs
avx Intel Sandy Bridge, Ivy Bridge
avx2 Intel Haswell, Broadwell
avx512 Intel Skylake-SP, Skylake-X, Cascade Lake-SP

For specific compiler versions, see our releases page.

Loading Modules in a Job Script

Modules in a job script can be loaded after your #SBATCH directives and before your actual executable is called. A sample job script that loads Python into the environment is shown below:

#!bin/bash
#SBATCH --nodes=1
#SBATCH --time=00:01:00
#SBATCH --ntasks=1
#SBATCH --job-name=test-job

# Load your modules here
module load python

python3 test-program.py

For more information, see the Running Jobs section.

Python

Several python modules are available for use: python, anaconda3, and scipy-bundle. We encourage users to checkout these modules as they include lots of common scientific python packages.

module Included python packages
python Bare python3 interpreter only
scipy-bundle beniget, Bottleneck, deap, gast, mpi4py, mpmath, numexpr, numpy, pandas, ply, pythran, scipy
anaconda3 see here

We recommend using virtual environments to best manage your python projects. We provide additional recommendations for using Python and Anaconda environments in our How To section.

R

Two R modules are provided: r and r-bundle-bioconductor, both of which include many pre-built R libraries. To see a complete list of R libraries and packages included with each module run the spider command:

$ module spider r
$ module spider r-bundle-bioconductor

To request additional packages be installed in either of these modules, please submit a request as detailed here or follow these recommendations to install your own.

Perl

We provide a perl module which includes many pre-built CPAN modules. To see the complete list run:

$ module spider perl

To request additional packages be installed in this module, please submit a request as detailed here or follow these recommendations to install your own.