This
page is the second of my MPI tips pages. The first was for compiling LAM-MPI,
this page is for compiling MPICH
with support for IBM
XLC and XL
Fortran compilers. Again, I had a lot of help from various colleagues. And
just to be sure, for everything to run correctly, you will have to install
MPICH and the IBM compilers on every compute node.
By
the way, if it looks similar to the LAM-MPI page, it should because I used the
other page as a template.
Document
Key:
Plain
black text are the descriptions of what we are doing
Green text are input to the terminal
To begin, make sure that passwordless ssh is enabled on ALL nodes in
which you will be installing MPICH. Because the configuration and building of
MPICH runs many tests during installation, even if are only installing MPICH on
a single machine, you will need to enable remote login capabilities and
passwordless ssh access on your local machine.
Download and untar
the latest MPICH distribution. As of the push of this page, the MPICH version
is 1.2.5.2 and we have IBM's XLC v6.0 and XLF v8.1 compilers. Go to the
top-level directory of the MPICH source code. Enter the following steps at the
command prompt in the terminal window to get MPICH correctly installed and
running on OSX (assuming bash shell, which is default on Panther):
To
begin, change directories to the MPICH directory. Next we will set the
environment or compiler flags for compiling MPICH with the XLC and XLF.
export RSHCOMMAND=ssh
export FC=/opt/ibmcmp/xlf/8.1/bin/xlf_r
export F90=/opt/ibmcmp/xlf/8.1/bin/xlf90_r
export CC=/opt/ibmcmp/vacpp/6.0/bin/xlc
If you do not have IBM XLC installed, you can use gcc instead
export CC=/usr/bin/gcc
We
will install MPICH in usr/local/mpich. In order to do this we will add the --prefix=/usr/local/mpich flag to the ./configure script. After setting the environment
variables, the sequence of steps should look like this:
./configure --with-device=ch_p4 -opt="-O"
--prefix=/usr/local/mpich
make
sudo make install
There
are two scripts created that are used to compile "c" and
"c++" mpi programs mpicc
and mpiCC. Because
the default OSX file system is not case sensitive one of these scripts will be
overwritten. To fix this we can enter the following commands:
sudo rm /usr/local/mpich/bin/mpiCC
sudo mv bin/mpicc /usr/local/mpich/bin/
sudo mv bin/mpicxx /usr/local/mpich/bin/
If
you need more information on configuring the "configure" script, you
can get help on the script by typing "./configure --help" at the
command prompt.
Compiling
the demo programs. Below is an example:
Compiling the Pi demo. Change directories to where your Pi
examples are stored, in this case, to the mpich directory.
cd /usr/local/mpich/examples
/usr/local/mpich/bin/mpicc -o cpi cpi.c -lm
/usr/local/mpich/bin/mpif77 -o pi3 pi3.f -lm
Runnig
the Pi demo.
/usr/local/mpich/bin/mpirun -np 1 cpi
/usr/local/mpich/bin/mpirun -np 1 fpi3
Note:
when running the MPICH demo, the -np option specifies the number of processes
to run on the machine, in this case, I was running the demo on my PowerBook
which has only a single processor.
If
you will be running MPICH across a cluster of machines, you will need to
install MPICH on all of the compute nodes as well as all of the compilers. IMPORTANT: every compute node will need to have
access to the application; this is usually done with a common shared directory
across all machines. Fortunately the MPICH demo examples are all located in the
same directory.
The
machinefile: For
MPICH to run jobs across the cluster, it will also need to have a list of
available machines. An example machine file containing four computers is shown
below:
# begin sample machinefile
node1.cluster.private
node2.cluster.private
node3.cluster.private
node4.cluster.private
# end sample machinefile
To
launch a job with these machines, we will also need to point to this machine
file with the -machinefile option. Running the Pi demo on the above machines
would look like this (assuming four dual processor G5 Xserves):
cd /usr/local/mpich/bin
./mpirun -np 8 -machinefile ~/myaccount/mpich_machines
/usr/local/mpich/examples/cpi