foss toolchain consists entirely of free and open source software
components. When building third-party software, or developing your own,
load the module for the toolchain:
$ module load foss/<version>
<version> should be replaced by the one to be used, e.g.,
2014a. See the documentation on the software module system for more details.
Three GCC compilers are available:
For example, to compile/link a Fortran program
fluid.f90 to an executable
fluid with architecture specific optimization for processors that support AVX instructions, use:
$ gfortran -O2 -march=corei7-avx -o fluid fluid.f90
Documentation on GCC compiler flags and options is available on the project's website. Do not forget to load the toolchain module first!
The compiler switch to use to compile/link OpenMP C/C++ or Fortran code is
-fopenmp. For example, to compile/link a OpenMP C program
scattter.c to an executable
scatter with optimization for processors that support the AVX instruction
$ gcc -fopenmp -O2 -march=corei7-avx -o scatter scatter.c
Remember to specify as many processes per node as the number of threads the
executable is supposed to run. This can be done using the
ppn resource, e.g.,
-l nodes=1:ppn=10 for an executable that should be run with 10 OpenMP threads.
The number of threads should not exceed the number of cores on a compute node.
Note that the OpenMP runtime library used by GCC is of inferior quality when compared
to Intel's, so developers are strongly encouraged to use the
intel toolchain when developing/building OpenMP software.
foss toolchain, Open MPI is used as the communications library.
To compile/link MPI programs, wrappers are supplied, so that the correct headers and
libraries are used automatically. These wrappers are:
The compiler wrappers take the same options as the corresponding compilers.
Using the MPI compilers from Open MPI
For example, to compile/link a C program
thermo.c to an executable
thermodynamics with architecture specific optimization for the AVX
instruction set, use:
$ mpicc -O2 -march=corei7-avx -o thermodynamics thermo.c
Extensive documentation is provided on the Open MPI project's website. Do not forget to load the toolchain module first.
Running an Open MPI program
Note that an MPI program must be run with the exact same version of the toolchain as
it was originally build with. The listing below shows a PBS job script
thermodynamics.pbs that runs the
#!/bin/bash -l module load intel/<version> cd $PBS_O_WORKDIR mpirun ./thermodynamics
The hosts and number of processes is retrieved from the queue system, that gets this information from the resource specification for that job.
The foss toolchain contains the basic HPC mathematical libraries, it offers:
- OpenBLAS (Basic Linear Algebra Subsystem)
- Lapack (Linear Algebra PACKage)
- ScaLAPACK (Scalable Linear Algebra PACKage)
- FFTW (Fastest Fourier Transform in the West)
- From the 2015b series on, binutils was added to the toolchain. The binutils package contains the assembler used by gcc, and the standard OS assembler doesn't always support the newer instructions that are used on newer cluster nodes.
Further information on FOSS components
- Overview of GCC manuals (all versions)
- OpenMPI documentation
- The OpenBLAS project page and documentation Wiki
- Generic BLAS/LAPACK/ScaLAPACK documentation
- FFTW documentation
- GNU binutils documentation