Compiling with MPI

Compiling MPI software

If you need to compile a software package written to use MPI, you must first load the appropriate version of MPI.

Examples

There are some simple example MPI programs in the /gpfs1/sw/examples/mpi directory that can be copied and used to practice building software with MPI. We will use those in the tutorial below.

Compiling with OpenMPI

To use this tutorial, please run these commands to set up the examples.

$ cd ~
$ mkdir mpi_example
$ cd mpi_example
$ cp /gpfs1/sw/examples/mpi/*.c .

One will most often be using OpenMPI on the VACC cluster to compile MPI enabled software. OpenMPI, which we will use for the examples, provides wrapper scripts that make including all the libraries and header files easier. There is a wrapper script for each type of compiler: C, C++, and Fortran: mpicc, mpic++1, and mpifort, respectively.

We now have two C programs that we can compile and run, so we use the mpicc wrapper. We will compile the cpi.c program, which calculates a numerical approximation of π. The -o option to mpicc is a standard compilation option and specifies the name of the executable program that will be produced.

$ mpicc -o cpi cpi.c

MPI refers to each independent computational process as a rank. To run, you should use the mpirun wrapper script, and with which you can specify how many ranks the program should start. For example, the command below will run it with four ranks using the -np 4 option; output is shown.

$ mpirun -np 4 cpi
Started with 4 ranks
> Enter the number of intervals to use in estimating: 
1000000000
Started with 1000000000 intervals
Pi is: 3.14159265358976824700

It is OK to run very small test runs on a login node, and in this case, the program prompts for input if no input is supplied on the command line.

Interactive programs and Slurm jobs

If your software will prompt for input, do NOT submit a job without making sure that all information is provided on the command line. If the software prompts, the job will run the full length of requested time waiting for you to type input and never do anything.

The cpi program will take the number of intervals as command line input, as in

$ mpirun -np 4 cpi 1000000000
Started with 4 ranks
Started with 1000000000 intervals
Pi is: 3.14159265358976824700

Once you have tested your MPI program with a small, short-running test, you would put the full command into your job script. Please see the Using MPI page for information about creating your job script.