
\chapter{Performance Analysis}


To evaluate the performance of the code on a parallel machine we ran some examples on a cluster of four nodes connected with PathScale InfiniPath HTX. 
Each node has two AMD Opteron 275 (Dual-core 2.2 GHz Processors) and 4 GB of main memory. The system achieves 60 Gigaflops with the Linpack benchmark, which is about 85% of peak performance.

For each test run we evaluated the parallel efficiency
\[
E_n = \frac{T_1}{nT_n} 100,
\]
where $T_n = \max_{0\le i < n}\{t_i\}$, $n$ is the total number of processors (subdomains) and $t_i$ is the time required to run the {\tt evolve} code on processor $i$.  Note that $t_i$ does not include the time required to build and subpartion the mesh etc., it only includes the time required to do the evolve calculations (eg. \code{domain.evolve(yieldstep = 0.1, finaltime = 3.0)}). 

\section{Advection, Rectangular Domain}

The first example looked at the rectangular domain example given in Section \ref{sec:codeRPA}, excecpt that we changed the finaltime time to 1.0 (\code{domain.evolve(yieldstep = 0.1, finaltime = 1.0)}).

For this particular example we can control the mesh size by changing the parameters \code{N} and \code{M} given in the following section of code taken from
 Section \ref{sec:codeRPA}.

\begin{verbatim}
#######################
# Partition the mesh
#######################
N = 20
M = 20

# Build a unit mesh, subdivide it over numproces processors with each
# submesh containing M*N nodes

points, vertices, boundary, full_send_dict, ghost_recv_dict =  \
    parallel_rectangle(N, M, len1_g=1.0)
\end{verbatim}

Tables \ref{tbl:rpa40}, \ref{tbl:rpa80} and \ref{tbl:rpa160} show the efficiency results for different values of \code{N} and \code{M}. The examples where $n \le 4$ were run on one node containing 4 processors, the $n = 8$ example was run on 2 nodes. The communication within a node is faster than the communication across nodes, so we would expect to see a decrease in efficiency when we jump from 4 to 8 nodes. Furthermore, as \code{N} and \code{M} are increased the ratio of ghost triangles to other triangles decreases. Or anothr way to think of it is that the size of the boundary layer decreses. Hence the amount of communication relative the amount of computation decreases and thus the efficiency should increase. 

The efficiency results shown here are competative.

\begin{table} 
\caption{Parallel Efficiency Results for the Advection Problem on a Rectangular Domain with {\tt N} = 40, {\tt M} = 40.\label{tbl:rpa40}}
\begin{center}
\begin{tabular}{|c|c c|}\hline
$n$ & $T_n$ (sec) & $E_n (\%)$ \\\hline
1 & 36.61 & \\
2 & 18.76 & 98 \\
4 & 10.16 & 90 \\
8 & 6.39 & 72 \\\hline
\end{tabular}
\end{center}
\end{table}

\begin{table} 
\caption{Parallel Efficiency Results for the Advection Problem on a Rectangular Domain with {\tt N} = 80, {\tt M} = 80.\label{tbl:rpa80}}
\begin{center}
\begin{tabular}{|c|c c|}\hline
$n$ & $T_n$ (sec) & $E_n (\%)$ \\\hline
1 & 282.18 & \\
2 & 143.14 & 99 \\
4 & 75.06 & 94 \\
8 & 41.67 & 85 \\\hline
\end{tabular}
\end{center}
\end{table}


\begin{table} 
\caption{Parallel Efficiency Results for the Advection Problem on a Rectangular Domain with {\tt N} = 160, {\tt M} = 160.\label{tbl:rpa160}}
\begin{center}
\begin{tabular}{|c|c c|}\hline
$n$ & $T_n$ (sec) & $E_n (\%)$ \\\hline
1 & 2200.35 & \\
2 & 1126.35 & 97 \\
4 & 569.49 & 97 \\
8 & 304.43 & 90 \\\hline
\end{tabular}
\end{center}
\end{table}


Another way of measuring the performance of the code on a parallel machine is
to increase the problem size as the number of processors are increased so that
the number of triangles per processor remains roughly the same.  We have node
carried out measurements of this kind as we usually have static grids and it
is not possible to increase the number of triangles.
  
\section{Advection, Merimbula Mesh}

We now look at another advection example, except this time the mesh comes from
the Merimbula test problem. Inother words, we ran the code given in Section
\ref{subsec:codeRPMM}, except the final time was reduced to 10000
(\code{\label{subsec:codeRPMM}). The results are given in Table \ref{tbl:rpm}.
These are good efficiency results, especially considering the structure of the
Merimbula mesh. Note that since we are solving an advection problem the amount
of calculation done on each triangle is relatively low, when we more to other
problems that involve more calculations we would expect the computation to
communication ratio to increase and thus get an increase in efficiency.

\begin{table} 
\caption{Parallel Efficiency Results for the Advection Problem on the
  Merimbula Mesh {\tt N} = 160, {\tt M} = 160.\label{tbl:rpm}}
\begin{center}
\begin{tabular}{|c|c c|}\hline
$n$ & $T_n$ (sec) & $E_n (\%)$ \\\hline
1 &145.17 & \\
2 &77.52 & 94 \\
4 & 41.24 & 88 \\
8 & 22.96 & 79 \\\hline
\end{tabular}
\end{center}
\end{table}
