Version 21 (modified by steve, 12 years ago) (diff)


INSTALLING anuga_parallel

Install anuga

First you should install the most uptodate version of the code. Follow the instructions to install Anuga on Ubuntu. By following those instructions you should end up with a download of the anuga_core code (which contains the sequential code (in the source/anuga directory) and the anuga_parallel code (in source/anuga_parallel)).

You should end up with a directory


where username is of course your username on your machine.

Make sure you have setup your PYTHONPATH to point to the anuga source directory

For instance I have the following line in my .bashrc file

export PYTHONPATH=/home/username/anuga_core/source

At this stage you should have a working version of the sequential anuga program. I.e. you should be able to run command


from the anuga_core directory and have your installation pass all the unit tests (well nearly all, as this is the development version and there are sometimes a few minor unit tests that fail).

If you have already downloaded anuga_core then it is sensible to update to the most recent version of the code using the subversion update command. Run the following command from the anuga_core directory

svn update

and then


This should update an old version to the most recent version.


Now to get anuga_parallel to work, we need to install some other packages first, in particular MPI for the parallel message passing and pypar a simple python wrapper of MPI.


Now you need to install MPI on your system. OPENMPI and MPICH2 are supported by pypar (see below) so both should be ok. But I tend to use mpich2.

So install mpich2 on your system via apt-get

sudo apt-get install mpich2

Make sure mpi works. You should be able to run a program in parallel. Something as simple as

mpirun -np 4 pwd

should produce the output of pwd 4 times.


We use pypar as the interface between mpi and python. The most recent version of PYPAR is available from

(There is an old version on sourceforge, do not use that)

Install pypar following the instructions in the download. You should be able use the standard command

python install

or maybe

sudo python install

from the source directory in the pypar distribution.

Fire up python and see if you can import pypar

You should obtain

>>> import pypar
Pypar (version 2.1.4) initialised MPI OK with 1 processors

Also make sure the pypar examples work

Problem with pypar Installation

We have been seeing the following error when trying to import pypar.

>>> import pypar
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "", line 863, in <module>
    mpi = CDLL('', RTLD_GLOBAL)
  File "/usr/lib/python2.6/ctypes/", line 353, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: cannot open shared object file: No such file or directory

For us this was caused by there not being a file in the /usr/lib directory. This is really a pypar bug, but as a work around we created a link from the file /usr/lib/ to /usr/lib/ via the command

sudo ln /usr/lib/ /usr/lib/

Then import pypar should produce the following

>>> import pypar
Pypar (version 2.1.4) initialised MPI OK with 1 processors

By the way, it is sometimes useful to fire up a new console to see if these installations work in a clean console.


In the anuga_parallel directory there is a subdirectory pymetis.

Follow the instructions in README to install. Essentially just run make.

From the pymetis diectory run


From the pymetis directory, test using, i.e. run



You should now be ready to run some parallel anuga code. Go back to the anuga_parallel directory and run

Hopefully that all works.

Example program


First just run it as a sequential program, via


Then try a parallel run using a command like

mpirun -np 4 python

That should run on 4 processors

You should look at the code in

Essentially a fairly standard example, with the extra command

domain = distribute(domain)

which sets up all the parallel stuff.

Also for efficiency reasons we only setup the original full sequential mesh on processor 0, hence the statement

if myid == 0:
     domain = create_domain_from_file(mesh_filename)
     domain.set_quantity('stage', Set_Stage(x0, x1, 2.0))
     domain = None

The output will be an sww file associated to each processor.


There is a script anuga/utilities/ which provides a function to merge sww files into one sww file for viewing with the anuga viewer.

Suppose your parallel code produced 3 sww files, domain_P3_0.sww domain_P3_1.sww and domain_P3_2.sww

The base name would be "domain" and the number of processors would be 3. To stitch these 3 files together either run as a script with the command

python /home/username/anuga_core/source/anuga/utilities/ -f domain -np 3

or you can add a command of the form


at the end of your simulation script, if you want to keep the individual parallel sww files or


(check out the script which demos this). if you are happy for the individual sww files to be deleted after the merge operation.