INSTALLING anuga_parallel

If you installed anuga and with the ANUGA_PARALLEL environment set via

export ANUGA_PARALLEL="mpich2"

or

export ANUGA_PARALLEL="openmpi"

then you should already have parallel support.

Setting up parallel support

Let's suppose that you initially only set up anuga to run in sequential mode. Then to setup parallel mode you will need to install an MPI environment (mpich2 or openmpi) and the python wrapper pypar.

We will assume you have install anuga from source and the source is in the directory anuga_core

Updating anuga_core

If you had already downloaded anuga_core then it is sensible to update to the most recent version of the code using the subversion update command. Run the following command from the anuga_core directory

svn update

and then

sudo python setup.py install
python runtests.py

This should update an old version to the most recent version.

Install anuga parallel

Now to get the parallel version of anuga to work, we need to install some other packages first, in particular MPI for the parallel message passing and pypar a simple python wrapper of MPI.

MPI

Now you need to install MPI on your system. OPENMPI and MPICH2 are supported by pypar (see below) so both should be ok. But I tend to use mpich2.

So install mpich2 on your system via apt-get

sudo apt-get install mpich2

Make sure mpi works. You should be able to run a program in parallel. Something as simple as

mpirun -np 4 pwd

should produce the output of pwd 4 times.

pypar

We use pypar as the interface between mpi and python. The most recent version of PYPAR is available from ​http://code.google.com/p/pypar/ Use svn to get the most recent version of the code. The tarred version is a little old.

(There is also an old version on sourceforge, do not use that)

From your home directory run the command

svn checkout http://pypar.googlecode.com/svn/ pypar

This produces a directory pypar

Change to that directory, and then run the command

sudo python setup.py install

This should install pypar.

Fire up python and see if you can import pypar

You should obtain

>>> import pypar
Pypar (version 2.1.4) initialised MPI OK with 1 processors

Also make sure that the pypar examples work.

By the way, I suggest firing up a new console to see if these installations work in a clean console.

Compile anuga parallel code

Actually the parallel code is already in the anuga_core directory. We just need to reinstall anuga.

From the anuga_core directory force a rebuild and reinstall of anuga via

sudo python setup.py build -f
sudo python setup.py install

Running anuga in parallel

You should now be ready to run some parallel anuga code.

python runtests.py

Hopefully that all works. If you are observant you should see that the number of unittests has increased by about 30, those are the parallel tests.

Example program

From the anuga_core/examples/parallel directory: Run run_parallel_sw_merimbula.py

First just run it as a sequential program, via

python run_parallel_sw_merimbula.py

Then try a parallel run using a command like

mpirun -np 4 python run_parallel_sw_merimbula.py

That should run on 4 processors

You should look at the code in run_parallel_sw_merimbula.py

Essentially this a fairly standard anuga script, with the extra command

domain = distribute(domain)

which sets up all the parallel stuff.

Also for efficiency reasons we only setup the original full sequential mesh on processor 0, hence the statement

if myid == 0:
     domain = create_domain_from_file(mesh_filename)
     domain.set_quantity('stage', Set_Stage(x0, x1, 2.0))
else:
     domain = None

The output will be an sww file associated to each processor.

sww_merge

There is a script anuga/utilities/sww_merge.py which provides a function to merge sww files into one sww file for viewing with the anuga viewer.

Suppose your parallel code produced 3 sww files, domain_P3_0.sww domain_P3_1.sww and domain_P3_2.sww

The base name would be "domain" and the number of processors would be 3. To stitch these 3 files together either run sww_merge.py as a script with the command

python /home/******/anuga_core/anuga/utilities/sww_merge.py -f domain -np 3

or you can add a command of the form

domain.sww_merge()

at the end of your simulation script, if you want to keep the individual parallel sww files or

domain.sww_merge(delete_old=True)

(check out the script run_parallel_sw_merimbula.py which demos this) if you are happy for the individual sww files to be deleted after the merge operation.