Available Software

Contents

1. The tap and module commands
   1. The tap command
   2. The module command
   3. Library packages
   4. Common errors
2. Software packages available
   1. Unclassified by field
   2. Classified by field, etc
      1. Software related to Biology
      2. Software related to Chemistry
      3. Software related to Meteorology
      4. Software related to Numerical Analysis
      5. Software related to Program Development
      6. GPU-enabled software packages
3. Packages for scripting languages
   1. Perl packages
   2. Python packages
   3. R modules

tap and module commands

A large number of software packages are available on each of the clusters, and for some of them, multiple versions of the same package. Not all of them like to co-exist with each other, so many of the packages are not available by default. You need to tell the system you want to use them with the tap and/or module.

Please note that the HPC compute nodes have abridged copies of the software directories available on other Glue/TerpConnect systems. The HPC login nodes have the full software directories. Please look at software lists below, and in particular click on the software package name for the full breakdown by version number to see which, if any, HPC clusters a specific version of the package is on.

The tap command is deprecated on Glue, and does not work on either of the Deepthought clusters. Use the module command instead.

For example, matlab is available on all Glued systems, but only after you module load it. E.g.

login-1: matlab
matlab: Command not found.
login-1: module load matlab
login-1: module help matlab

----------- Module Specific Help for 'matlab/2014b' ---------------


    This is Matlab Release 2014b

    Releases after 2009 require Red Hat 5 or later.
    Releases after 2013 require Red Hat 6 or later.
    Matlab 2009b is the last release supported under Solaris.

    Run command 'matlab' to start up the program,
    or 'matlab -h' to see various command-line options.

    ***  To see all releases, issue the command
	module avail matlab

login-1: matlab
                              < M A T L A B >
                  Copyright 1984-2014 The MathWorks, Inc.
                   R2014b (8.4.0.150421) 64-bit (glnxa64)
                             September 15, 2014


  To get started, type one of these: helpwin, helpdesk, or demo.
  For product information, visit www.mathworks.com.

>> 

You can use the module help and module whatis commands before (or without) loading the package.

The tap command

Historically, GLUE has used the tap command. It is fairly simple to use. Each package has a tag, like matlab in the example above.

The tap command is deprecated on Glue, and does not work on either of the Deepthought clusters. Use the module command instead.

This section is no longer applicable to the Deepthought clusters. It is being retained for historical reasons.

For example, matlab is available on all Glued systems, but

To enable a package, you simple issue the command

tap PACKAGE_TAG

This version of the command will set up your environment for using the software, and print a brief message about the package. Usually things like what version, very basic usage instructions, and maybe where to find help. You can add the -q flag to suppress this message. NOTE: If you put tap commands in your startup scripts, e.g ~/.cshrc.mine, be sure to include the -q flag. Otherwise, the extraneous output from the tap message when you log in might break file transfer protocols like scp or sftp.

We encourage you to read the message at least once when you start to use a new package. There might be something useful in it.

NOTE: If you are submitting a job using a bourne style shell (e.g. sh or bash), your dot files will not be read automatically if your default shell is something else (e.g. csh or tcsh). So you will need to include a . ~/.profile line near the top of your script, after the #SBATCH lines but before you try to use tap or module commands.

You can also give the commands

tap -h
tap -l

We are planning to phase out the tap command, and switch entirely to the module, as it is more flexible and has a broad user base outside of UMD. Some commands will be available via both mechanisms for a while, although in many cases not all options will be available via tap, and even when an application is available, the number of versions avaialable through tap will be restricted.

The tap command is NOT available on the two Deepthought HPC clusters. Please switch to the module command. If there is an application you need which is only available via tap, please open a help ticket.

The module command

The module is replacing the tap command across Glue, and has already replaced it on the Deepthought HPC clusters. The module command is community supported, and used at many other institutions, and has some features that are lacking in the tap command.

Just like the tap command, the module command associates a tag with the various packages. Unlike the tap command, the module tag has a hierarchical nature, so we have tags like matlab/2013b and matlab/2011a in addition to plain old matlab. The former specify a specific version of matlab, whereas the latter will enable the default version of matlab.

For many packages, the tag is basically just the base name of the package, followed by a slash (/) and a version number. E.g., the examples matlab/2013b and matlab/2011a refer to versions 2013b and 2011a, respectively, of matlab. You can also just use matlab as the tag when e.g. loading a module, in which case you will get one of the versions (usually the latest, sometimes we pin to an earlier version if the newest version is still being tested).

Some packages are fussy about the compiler and/or MPI libraries they are used with. This is particularly true of libraries which have fortran90 and/or C++ components; some examples are openmpi, hdf5, and netcdf. In these cases, you typically have at least two ways to refer to the same module/build of the software. The first starts like the simple case, with the base name of the package followed by the version of the package, but to this is appended the other dependencies (like compiler family and version, and perhaps MPI library family and version, etc.). For example, one tag for version 1.8.13 of hdf5, built with version 4.8.1 of the GNU compiler suite and version 1.8.1 of , with shared linkage would be:

hdf5/1.8.13/gnu/4.8.1/openmpi/1.8.1/shared  

There is also an alternative form giving the compiler, MPI library, and linkage portions of the tag immediately after the base package name, and ending with the version of the package, e.g.

hdf5/gnu/4.8.1/openmpi/1.8.1/shared/1.8.13

Both of these tags do exactly the same thing, and set up exactly the same build of hdf5, when loaded. You can use either.

The advantage of the module command is that you generally do NOT need to give these long tag names, but can leave off components on the right and it will try to do the right thing. In particular, if you try loading just the base package name, e.g. module load hdf5, the system will determine if you have previously loaded a compiler and/or MPI library. If so, then the system will see if there are any builds of any version of the package matching the compiler and MPI library versions you previously loaded, and if one is found load the build for the most recent version that matches. If no matching build is found, and error will be displayed and no module loaded.

The following except gives an example of this:

login-2:> module purge 
login-2:> module list 
No Modulefiles Currently Loaded.
login-2:> module load gcc/4.8.1
login-2:> module load openmpi/1.8.1
login-2:> module load hdf5
login-2:> module list
Currently Loaded Modulefiles:
  1) gcc/4.8.1(default)
  2) openmpi/1.8.1/gnu/4.8.1
  3) hdf5/gnu/4.8.1/openmpi/1.8.1/shared/1.8.13
login-2:> 
login-2:> module purge
login-2:> module list
No Modulefiles Currently Loaded.
login-2:> module load intel/2016
login-2:> module load openmpi/1.10.2
login-2:> module load hdf5
login-2:> module list
Currently Loaded Modulefiles:
  1) ofed/3.12(default)
  2) intel/2016.3.210(default)
  3) openmpi/1.10.2/intel/2016.3.210
  4) hdf5/intel/2016.3.210/openmpi/1.10.2/shared/1.10.0
login-2:> 
login-2:> module purge
login-2:> module list
No Modulefiles Currently Loaded.
login-2:> module load hdf5/1.8.15p1/sunstudio/12.4/openmpi/1.8.6/shared
login-2:> module list
Currently Loaded Modulefiles:
  1) sunstudio/12.4(default)
  2) openmpi/1.8.6/sunstudio/12.4
  3) hdf5/1.8.15p1/sunstudio/12.4/openmpi/1.8.6/shared

We start by issuing a module purge command to delete any previously loaded modules (in general, you should use this with caution. If you do use it, you should immediately load the modules hpcc/deepthought or hpcc/deepthought2, as appropriate for the cluster you are on, and slurm. These are the default modules on the Deepthought HPC clusters and things might not work properly without them). We then do a module list to confirm that there are no modules loaded. We then load a specific version of the gcc compiler (4.8.1) and the openmpi libraries (1.8.1) and then ask the system to load the best hdf5 module. We then list what was loaded, and note that the system loaded an openmpi build (version 1.8.1) for the compiler we loaded, and a hdf5 version (1.8.13) for the previously loaded compiler and MPI library.

In the second example above, we load the 2016 version of the intel compiler suite, and the 1.10.2 version of openmpi, and when we issue the simple module load hdf5, this time it loads hdf5 version 1.10.0 built for that compiler and openmpi library. (Note also that the openmpi library loaded was also built for the correct compiler.)

In the final example above, we do not explicitly load any compiler or MPI library, but instead give a full path the the hdf5 module, e.g. hdf5/1.8.15p1/sunstudio/12.4/openmpi/1.8.6/shared. In this case, we note that not only was the requested hdf5 module loaded, but we also loaded the corresponding compiler (sunstudio/12.4) and openmpi (openmpi/1.8.6) modules as well.

In summary, in almost all cases, you need only specify a portion of the tag, and the system will try to default it properly. Generally, you should load the compiler you wish to use first (or do not load anything if you want to use the default gcc 4.6.1 compiler) followed by the MPI library if needed. This will allow the system to pick the correct build of libraries which depend on such. Alternatively, if you specify the full path to a package which depends on a specific compiler and MPI library, these will be loaded for you if not already loaded. In general, if the package has dependencies, they will be loaded automatically for you if needed.

However, if you want reproducable results, it is generally recommended that you fully specify version numbers when loading modules in your job scripts. I.e., the latest version of the foo package might be 1.5 when you run your job today, but if you run it again a month from today, the latest version might by 1.7 at that time, and that could change the results unexpectedly if your script merely did a module load foo. If you wish to ensure the same version is always used, specifying the version number in the module load command is advised, e.g. module load foo/1.5; this way you will get the same version of the foo package both today and a month from now. But if you always prefer to use the latest package available, then in general you should leave the version numbers off.

NOTE: If you are submitting a job using a bourne style shell (e.g. sh or bash), your dot files will not be read automatically if your default shell is something else (e.g. csh or tcsh). So you will need to include a . ~/.profile line near the top of your script, after the #SBATCH lines but before you try to use tap or module commands.

The module command allows you to issue commands of the form module SUBCOMMAND OPTIONAL_ARGUMENT. Recognized subcommands include:

• list: This will list the packages you currently have loaded. Note that if a tap entry was migrated to modules, it will show up here if you loaded it using tap.
• avail OPTIONAL_TAG: this will list all the packages available. If a tag is given, it will list all packages which match the tag. Note the matching is purely lexical, so module avail netcdf will pick up all netcdf modules, as well as netcdf-fortran and netcdf-cxx modules. To restrict to just netcdf modules, use something like module avail netcdf/ (note the final /)
• load TAG: this is the most common usage, and it sets up the environment for the application associated with TAG. Again, if you do not specify the full tag, the system will attempt to default it, and usually it can do that with some success. Most of our application definitions will pull in any dependencies as well. Note that unlike tap, this is a silent operation when it succeeds.
• help TAG: This displays a brief intro message about the package. Generally the same message you see when you tap a package. It is advised that you read this once for each package you use.
• unload TAG: This unloads the package; i.e. all the environment that was set up for the package is returned to what it was before you loaded it. This is not 100% successful all the time, but for the most part works. You generally do not need this unless you which to change the version of an application you are using.
• purge: This unloads ALL currently loaded modules. This should be used sparingly. If you do use it, please follow it with the loading of either hpcc/deepthought or hpcc/deepthought2 (as appropriate for the cluster you are on) and slurm. These modules are loaded for you automatically when you log into the clusters, and if you remove them (e.g. with the purge command), things might not work as expected.

Please note that the HPC compute nodes have abridged copies of the software directories available on other Glue/TerpConnect systems. The HPC login nodes have the full software directories. Please look at software lists below, and in particular click on the software package name for the full breakdown by version number to see which, if any, HPC clusters a specific version of the package is on.

Library packages

Some packages are not so much applications as libraries; i.e. they provide libraries which your code can use, and might or might not have any binaries that you can run directly. To make use of these libraries in your code takes a little bit of work. Please see the section on compiling codes for more information on how to do this.

Common errors from module/tap commands

As stated early, not all packages co-exist nicely. This is particularly true of different versions of the same package, or related libraries (e.g. the various MPI libraries). The module command will generate an error if you try to load a package which does not play well with packages you have previously loaded; if this occurs you will need to unload the previous package and reload the new package. E.g.

login-1: module load matlab/2012b
login-1:~: module load matlab/2013b
matlab/2013b(8):ERROR:150: Module 'matlab/2013b' conflicts with the currently loaded module(s) 'matlab/2012b'
matlab/2013b(8):ERROR:102: Tcl command execution failed: source $moduledir/common
login-1: module unload matlab
login-1:~: module load matlab/2013b

Although not as user friendly as one would like, you can see that the first time I tried to load matlab/2013b it complained because matlab/2012b was already loaded. Also note that I did not need to specify the full tag to unload matlab/2012b, as there was only one matlab package loaded, so it could default it.

You can also run into similar incompatibillity problems if you mix packages (especially library packages) built for one compiler/MPI library/etc with a different compiler/MPI library/etc. For example

login-1: module avail fftw/3.3.3

------------------- /cell_root/system/common/modulefiles/sys -------------------
fftw/3.3.3/gnu/4.6.1/openmpi/1.6/shared
fftw/3.3.3/gnu/4.6.1/openmpi/1.6/static
fftw/3.3.3/gnu/4.6.1/openmpi/1.6.5/shared
fftw/3.3.3/gnu/4.6.1/openmpi/1.6.5/static
fftw/3.3.3/intel/2013.1.039/openmpi/1.6.5/shared
fftw/3.3.3/intel/2013.1.039/openmpi/1.6.5/static

login-1: module load intel/2013.1.039

login-1: module load fftw/3.3.3/gnu

***** Compiler mismatch

You have loaded compiler intel/2013.1.039
This package wants compiler gnu/4.6.1
login-1: 

In the above, we show that there are various builds of fftw/3.3.3 available. We then load the intel compiler module, and try to load a GNU compiler build of fftw/3.3.3, and get an error. That is because using a GNU build of fftw/3.3.3 with the intel compiler suite is likely not to work and cause you headaches. In general, you should not be specifying more than fftw/3.3.3 anyway, and the module command will figure out which build to load based on which compilers you already have loaded. Alternatively, you can give the full path (e.g. module load fftw/3.3.3/intel/2013.1.039/openmpi/1.6.5) and the module command will auto-load the appropriate intel compiler and openmpi module for if, if it can. If you have a conflicting module already loaded, you will still get the error.

For the most part, tap is just a wrapper around the module command for backwards compatibility, and so you will get the same errors with tap if you try to load conflicting modules. The tap command is limited, so you generally do NOT get a whole lot of choice as to which builds of packages get loaded, so it is less likely to be a problem, but it can still arise. Some packages have not been migrated from tap to module command, and these will NOT generate such errors when you tap incompatible modules. But things will probably NOT work as you expected; the incompatibility is REAL, not an artifact of the module command.

List of available software packages

We are currently in the process of integrating the software documentation for RHEL6 and RHEL8. This page only shows information for packages in the RHEL6 software library; information on the RHEL8 software library can be found at the new RHEL8 software documentation pages.

Please note that the HPC compute nodes have abridged copies of the software directories available on other Glue/TerpConnect systems. The HPC login nodes have the full software directories. Please look at software lists below, and in particular click on the software package name for the full breakdown by version number to see which, if any, HPC clusters a specific version of the package is on.

1. Unclassified by field
2. Classified by field, etc
   1. Software related to Biology
   2. Software related to Chemistry
   3. Software related to Meteorology
   4. Software related to Numerical Analysis
   5. Software related to Program Development
3. GPU-enabled software packages

All categories

The following shows all the software available on the HPC systems, unsorted by category.

Categorized by field, etc

The following tables basically contain the same information as above, but are filtered by field of research and other criteria to make them easier to browse.

1. Biology
2. Chemistry
3. Meteorology
4. Numerical Analysis
5. Program Development
6. GPU-enabled software packages

Biology

In addition to the packages listed below, Dr. Ian Misner maintains a large collection of bioinformatics packages, most of which are available for use (although with no promises of support) by anyone with access to the Deepthought2 HPC cluster. You will need to module load dept/bioinfo to get access to them, and then you can use the module avail bioinfo command to list what is available. Again, these were installed by Dr. Misner for his own research, and no claims are made that they are usable for anyone else's research, or that they will be maintained, but with these caveats you are free to use them. Also, access to some applications might be restricted due to licensing, etc.

Chemistry

Meteorology

Numerical Analysis

Program Development

GPU-enabled software packages

The following lists shows packages available on the HPC clusters for which at least one version of the package has been built to be able to make use of the GPUs available on some nodes. Typically, there will be special builds of the package (i.e. special module load commands needed) which can use the GPUs, and those versions will typically ONLY run on nodes with GPUs.

You should follow the link to the package specific page for more information on which versions of the package are built to support GPUs, and how to use them.

This list is quite small right now. If there is a package on one of the Deepthought HPCCs that you use that can support using GPUs but is not on the list, let us know and we will look into building a GPU aware version. NOTE: The Division of Information Technology does not have the expertise or resources to adapt codes to take advantage of GPUs; the above is for packages to which the software authors have already added GPU support, and only need a fresh build on the Deepthought HPCCs to take advantage of that support.

Packages installed for scripting languages

We are currently in the process of integrating the software documentation for RHEL6 and RHEL8. This page only shows information for packages in the RHEL6 software library; information on the RHEL8 software library can be found at the new RHEL8 software documentation pages.

The Division of Information Technology maintains several versions of several scripting languages on the clusters. These scripting languages can be enhanced by the addition of packages/modules/etc, and the Division of IT maintains a collection of packages for these. In cases where the package/module is large, tightly coupled to a specific version of an installed library, and/or is not applicable to many fields, we tend to install the packages/modules/etc as entities which need to be separately module loaded. These will be listed in the application tables above.

The tables below are primarily for packages stored in the standard location for the specific scripting language. We also do NOT list any packages/modules/etc. that come as part of the standard library for the particular scripting language. Note also that packages/modules/etc. are not necessarily available on all versions of the particular scripting language installed, and you might need to module load a specific version of the scripting language to get a specific package.

Perl packages

We have many perl modules installed; this is NOT a complete list.

You should also look at the Perl page for more information, including information re installing your own modules.

Python packages

This is NOT a complete list of python modules available, just a list of some major ones. Many more modules are actually installed.

NOTE: There have been some issues noted with theano on HPC nodes with Qlogic Infiniband cards. On such nodes (including DT1 login nodes), you will need to set the environmental variable OMPI_MCA_mtl to ^psm, e.g.

	setenv OMPI_MCA_mtl ^psm

	OMPI_MCA_mtl=^psm
	export OMPI_MCA_mtl

You should also look at the Python page for more information, including information re installing your own modules.

R packages

This is NOT a complete list of R modules available

You should also look at the R page for more information, including information re installing your own modules.