How to find and load software (aka modules)

On eRI we make software available using environment modules, sometime also referred to as lmod. This article describes how to use the module command to find, load and purge software.

Environment Modules

Modules are a convenient way to provide access to applications on the cluster. They prepare the environment you need to run an application.

For a full list of module commands run man module or visit the lmod documentation here.

`module avail`	Lists all available modules.
`module spider [module name]`	Searches available modules for [module name]
`module show [module name]`	Shows information about [module name]
`module load [module name]`	Loads [module name]
`module list [module name]`	Lists currently loaded modules.
`module purge`	Purge (remove) all modules from your environment

For example, if you want to use the R programming language you would:

$ module load R/4.3.2-foss-2023a

This will load all of the software and dependencies required to run Rversion 4.3.2. To see the list of software loaded when you run the above command you can run the module list command:

   ASReml/4.2.1.206                                  SQLite/3.42.0-GCCcore-12.3.0              (L)
   Apptainer/1.2.5                                   SWIG/3.0.8
   Arrow/14.0.1-GCC-12.3.0                  (L)      ScaLAPACK/2.2.0-gompi-2023a-fb            (L)
   BCL-Convert/4.2.4                                 Szip/2.1.1-GCCcore-12.3.0                 (L)
   BLASTDB/2024-04                                   Tcl/8.6.10-GCCcore-12.3.0
   BLIS/0.9.0-GCC-12.3.0                             TeXLive/2022
   BLIS/4.1-GCC-12.3.0-amd                  (D)      Tk/8.6.10-GCCcore-12.3.0
   Boost/1.83.0-GCC-12.3.0                           UCC/1.2.0-GCCcore-12.3.0                  (L)
   CMake/3.27.7                                      UCX/1.14.1-GCCcore-12.3.0                 (L)
   CUDA/11.8.0                                       UDUNITS/2.2.28-GCCcore-12.3.0             (L)
   Canu/2.2-GCC-12.3.0                               XZ/5.4.2-GCCcore-12.3.0                   (L)
   EasyBuild/4.8.1                                   ZeroMQ/4.3.5-GCCcore-12.3.0
   EasyBuild/4.9.2                          (D)      ant/1.10.9-Java-17
   FFTW.MPI/3.3.10-gompi-2023a              (L)      binutils/2.40-GCCcore-12.3.0              (L)
   FFTW/3.3.10-GCC-12.3.0                   (L)      binutils/2.40                             (D)
   FlexiBLAS/3.3.1-GCC-12.3.0               (L)      bzip2/1.0.8-GCCcore-12.3.0                (L)
   FriBidi/1.0.12-GCC-12.3.0                         cURL/8.3.0-GCCcore-12.3.0                 (L)
   GCC/12.3.0                               (L)      cuDNN/8.6.0.163-CUDA-11.8.0
   GCCcore/12.3.0                           (L)      expat/2.5.0-GCCcore-12.3.0                (L)
   GDAL/3.6.4-gompi-2023a                   (L)      fontconfig/2.14.2-GCCcore-12.3.0
   GEOS/3.11.3-GCC-12.3.0                   (L)      foss/2023a                                (L)
   GLib/2.78.1-GCC-12.3.0                            freetype/2.13.2-GCCcore-12.3.0
   GMP/6.2.1-GCCcore-12.3.0                 (L)      gettext/0.21-GCCcore-12.3.0
   Go/1.22.3                                         gompi/2023a                               (L)
   HDF5/1.14.3-gompi-2023a                  (L)      gperf/3.1
   HarfBuzz/4.4.1-GCC-12.3.0                         gquery-dev/0.7.0
   IGV/2.17.4                                        gquery-dev/0.8.1
   IRkernel/1.3.2-foss-2023a-R-4.3.2                 gquery-dev/0.9.0
   IRkernel/1.3.2-foss-2023a-R-4.3.3        (D)      gquery-dev/0.9.1
   Java/17                                           gquery-dev/0.9.2                          (D)
   Java/20.0.2                              (L,D)    gquery-test/0.8.1
   Julia/1.10.1-GCC-12.3.0-VTune                     gquery-test/0.9.0
   JupyterLab/2023.11.0-foss-2023a-3.6.3             gquery-test/0.9.1
   JupyterLab/2024.05.0-foss-2023a-4.2.0    (D)      gquery-test/0.9.2                         (D)
   KEALib/1.5.2-gompi-2023a                 (L)      gquery/0.8.1
   LLVM/14.0.6-GCC-12.3.0-static                     gquery/0.9.0
   LibTIFF/4.4.0-GCCcore-12.3.0                      gquery/0.9.1
   LibTIFF/4.5.1-GCCcore-12.3.0             (L,D)    gquery/0.9.2                              (D)
   Meson/0.62.1                                      graphite2/1.3.14
   Miniconda3/23.10.0-1                              json-c/0.17-GCC-12.3.0                    (L)
   NASM/2.15.05                                      libKML/1.3.0.2017-GCC-12.3.0              (L)
   NLopt/2.7.1-GCC-12.3.0                   (L)      libffi/3.4.6
   NanoPlot/1.43.0-foss-2023a-Python-3.11.6          libgd/2.3.3-GCCcore-12.3.0
   Nextflow/24.04.2                                  libgeotiff/1.7.1-GCC-12.3.0-PROJ-9.3.0    (L)
   Ninja/1.11.1                                      libgit2/1.6.4-GCC-12.3.0                  (L)
   OpenBLAS/0.3.23-GCC-12.3.0               (L)      libjpeg-turbo/2.1.5.1-GCCcore-12.3.0      (L)
   OpenJPEG/2.5.0-GCCcore-12.3.0            (L)      libpng/1.6.40-GCCcore-12.3.0              (L)
   OpenMPI/4.1.5-GCC-12.3.0                 (L)      libreadline/8.2-GCCcore-12.3.0            (L)
   OpenSSL/1.1                              (L)      libxml2/2.11.4-GCCcore-12.3.0             (L)
   OpenSSL/3.3.1-GCCcore-12.3.0             (D)      libxslt/1.1.38-GCCcore-12.3.0
   PCRE2/10.42-GCCcore-12.3.0               (L)      lz4/1.9.4-GCCcore-12.3.0                  (L)
   PROJ/9.3.0-GCC-12.3.0                    (L)      nanoget/1.19.3-foss-2023a-Python-3.11.6
   Perl/5.38.2-GCC-12.3.0                            ncurses/6.4-GCCcore-12.3.0                (L)
   PostgreSQL/14.12-GCC-12.3.0                       netCDF/4.9.2-gompi-2023a                  (L)
   Python/3.11.6-foss-2023a                          nodejs/18.18.2-GCCcore-12.3.0             (L)
   R-Geo/4.3.2-foss-2023a                            nullarbor/2.0.20191013
   R/4.3.2-foss-2023a                       (D)      numactl/2.0.16-GCCcore-12.3.0             (L)
   R/4.3.3-foss-2023a                                pixman/0.42.2-GCC-12.3.0
   R/4.4.0-foss-2023a                                snakemake/7.32.3-foss-2023a-Python-3.11.6
   R/4.4.1-foss-2023a                       (L)      snakemake/8.14.0-foss-2023a-Python-3.11.6 (D)
   RStudio-Server/2023.12.0-369                      snappy/1.1.10-GCCcore-12.3.0              (L)
   RStudio-Server/2024.04.2-764             (D)      yaml-cpp/0.8.0-GCC-12.3.0
   Rust/1.73.0-GCC-12.3.0                            zlib/1.2.13-GCCcore-12.3.0                (L)
   SAMtools/0.1.19-GCCcore-12.3.0                    zlib/1.2.13                               (D)
   SOCI/4.0.3-GCC-12.3.0                             zstd/1.5.5-GCCcore-12.3.0                 (L)
   SPAdes/4.0.0-foss-2023a-Python-3.11.6

Working with containers

Singularity

Available as Apptainer and aliasing enabled, both apptainer --version and singularity --version will work. There are NeSI/Mahuika based notes on building containers HERE

Example

# QIIME1
module purge
module load Apptainer/1.2.5

# recent Apptainer modules set APPTAINER_BIND, which typically breaks
# container builds, so unset it here
unset APPTAINER_BIND

# create a build and cache directory on scratch storage
export APPTAINER_CACHEDIR="/agr/scratch/projects/<YOURPROJECT>/$USER/apptainer_cache"
export APPTAINER_TMPDIR="/agr/scratch/projects/<YOURPROJECT>/$USER/apptainer_tmpdir"
mkdir -p $APPTAINER_CACHEDIR $APPTAINER_TMPDIR
setfacl -b $APPTAINER_TMPDIR

# pull the container
apptainer pull docker://mgrast/qiime:1.0

# execute the container
apptainer exec qiime_1.0.sif align_seqs.py -h