Changeset 2786


Ignore:
Timestamp:
May 1, 2006, 10:36:09 AM (18 years ago)
Author:
linda
Message:

Continued working on parallel documentation. Finished (waiting on feedback) description of how to parallelise the code and example codes

Location:
inundation/parallel/documentation
Files:
6 edited

Legend:

Unmodified
Added
Removed

  • inundation/parallel/documentation/code.tex

    r2768 r2786  
    11
    2 \chapter{Code Listing}
    3 \section{run_parallel_advection.py}
    4 \verbatiminput{RunParallelAdvection.py}
     2\chapter{Code Listing}\label{chap:code}
     3\section{run_parallel_advection.py}\label{sec:codeRPA}
     4\verbatiminput{code/RunParallelAdvection.py}
    55\newpage
    6 \section{run_parallel_merimbula_metis.py}
    7 \verbatiminput{RunParallelMerimbulaMetis.py}
     6\section{run_parallel_merimbula_metis.py}\label{sec:codeRPMM}
     7\verbatiminput{code/RunParallelMerimbulaMetis.py}
    88\newpage
    9 \section{run_parallel_sw_merimbula_metis.py}
    10 \verbatiminput{RunParallelSwMerimbulaMetis.py}
     9\section{run_parallel_sw_merimbula_metis.py}\label{sec:codeRPSMM}
     10\verbatiminput{code/RunParallelSwMerimbulaMetis.py}

  • inundation/parallel/documentation/code/RunParallelAdvection.py

    r2785 r2786  
    8686T = Transmissive_boundary(domain)
    8787domain.default_order = 2
    88 domain.set_boundary( {'left': T, 'right': T, 'bottom': T, 'top': T, 'ghost': None} )
     88domain.set_boundary( {'left': T, 'right': T, 'bottom': T, 'top': T, \
     89                      'ghost': None} )
    8990
    9091# Ensure that the domain definitions make sense
    … …  
    112113if myid == 0:
    113114    print 'That took %.2f seconds' %(time.time()-t0)
    114     print 'Communication time %.2f seconds'%domain.communication_time
    115     print 'Reduction Communication time %.2f seconds'%domain.communication_reduce_time
     115    print 'Communication time %.2f seconds'\
     116          %domain.communication_time
     117    print 'Reduction Communication time %.2f seconds'\
     118          %domain.communication_reduce_time

  • inundation/parallel/documentation/code/RunParallelMerimbulaMetis.py

    r2785 r2786  
    9696    rect = array(domain_full.xy_extent, Float)
    9797
    98     # Subdivide the mes
     98    # Subdivide the mesh
    9999
    100100    nodes, triangles, boundary, triangles_per_proc, quantities  =\
    … …  
    154154T = Transmissive_boundary(domain)
    155155#R = Reflective_boundary(domain)
    156 domain.set_boundary( {'outflow': T, 'inflow': T, 'inner':T, 'exterior': T, 'open':T, 'ghost':None} )
     156domain.set_boundary( {'outflow': T, 'inflow': T, 'inner':T, \
     157                      'exterior': T, 'open':T, 'ghost':None} )
    157158
    158159# Set the initial quantities

  • inundation/parallel/documentation/code/RunParallelSwMerimbulaMetis.py

    r2785 r2786  
    189189    print 'That took %.2f seconds' %(time.time()-t0)
    190190    print 'Communication time %.2f seconds'%domain.communication_time
    191     print 'Reduction Communication time %.2f seconds'%domain.communication_reduce_time
    192     print 'Broadcast time %.2f seconds'%domain.communication_broadcast_time
     191    print 'Reduction Communication time %.2f seconds'\
     192          %domain.communication_reduce_time
     193    print 'Broadcast time %.2f seconds'\
     194          %domain.communication_broadcast_time

  • inundation/parallel/documentation/parallel.tex

    r2767 r2786  
    11\chapter{Running the Code in Parallel}
    22
    3 This chapter looks at how to run the code in parallel. The first section gives an overview of the main steps required to divide the mesh over the processors. The second sections gives some example code and the final section talks about how to run the code on a few specific architectures.
    4 
    5 
    6 \section {Partitioning the Mesh}
     3This chapter looks at how to run the code in parallel. The first section gives an overview of the main steps required to divide the mesh over the processors. The second sections describes some example code and the final section talks about how to run the code on a few specific architectures.
     4
     5
     6\section {Partitioning the Mesh}\label{sec:part}
    77There are four main steps required to run the code in parallel. They are;
    88\begin{enumerate}
    … …  
    1919
    2020\begin{figure}[hbtp]
    21   \centerline{ \includegraphics[scale = 0.75]{domain.eps}}
     21  \centerline{ \includegraphics[scale = 0.75]{figures/domain.eps}}
    2222  \caption{The main steps used to divide the mesh over the processors.}
    2323  \label{fig:subpart}
    2424\end{figure}
    2525
    26 \subsection {Subdividing the Global Mesh}
     26\subsection {Subdividing the Global Mesh}\label{sec:part1}
    2727
    2828The first step in parallelising the code is to subdivide the mesh
    … …  
    3939
    4040\begin{figure}[hbtp]
    41   \centerline{ \includegraphics[scale = 0.75]{mermesh.eps}}
     41  \centerline{ \includegraphics[scale = 0.75]{figures/mermesh.eps}}
    4242  \caption{The Merimbula grid.}
    4343 \label{fig:mergrid}
    … …  
    4747
    4848\begin{figure}[hbtp]
    49   \centerline{ \includegraphics[scale = 0.75]{mermesh4c.eps}
    50   \includegraphics[scale = 0.75]{mermesh4a.eps}}
    51 
    52 
    53   \centerline{ \includegraphics[scale = 0.75]{mermesh4d.eps}
    54   \includegraphics[scale = 0.75]{mermesh4b.eps}}
     49  \centerline{ \includegraphics[scale = 0.75]{figures/mermesh4c.eps}
     50  \includegraphics[scale = 0.75]{figures/mermesh4a.eps}}
     51
     52
     53  \centerline{ \includegraphics[scale = 0.75]{figures/mermesh4d.eps}
     54  \includegraphics[scale = 0.75]{figures/mermesh4b.eps}}
    5555  \caption{The Merimbula grid partitioned over 4 processors using Metis.}
    5656 \label{fig:mergrid4}
    … …  
    8383The number of submeshes found by Pymetis is equal to the number of processors; Submesh $p$ will be assigned to Processor $p$.
    8484
    85 \subsection {Building the Ghost Layer}
     85\subsection {Building the Ghost Layer}\label{sec:part2}
    8686The function {\tt build_submesh.py} is the work-horse and is responsible for
    8787setting up the communication pattern as well as assigning the local numbering scheme for the submeshes.
    … …  
    9191
    9292\begin{figure}[hbtp]
    93   \centerline{ \includegraphics[scale = 0.6]{subdomain.eps}}
     93  \centerline{ \includegraphics[scale = 0.6]{figures/subdomain.eps}}
    9494  \caption{An example subpartioning.}
    9595 \label{fig:subdomain}
    … …  
    9898
    9999\begin{figure}[hbtp]
    100   \centerline{ \includegraphics[scale = 0.6]{subdomainghost.eps}}
     100  \centerline{ \includegraphics[scale = 0.6]{figures/subdomainghost.eps}}
    101101  \caption{An example subpartioning with ghost triangles.}
    102102 \label{fig:subdomaing}
    … …  
    117117Finally, the ANUGA code assumes that the triangles (and nodes etc.) are numbered consecutively starting from 1. Consequently, if Submesh 1 in Figure \ref{fig:subdomaing} was passed into the \code{evolve} calculations it would crash. The \code{build_submesh} function determines a local numbering scheme for each submesh, but it does not actually update the numbering, that is left to \code{build_local}.
    118118
    119 \subsection {Sending the Submeshes}
     119\subsection {Sending the Submeshes}\label{sec:part3}
    120120
    121121All of functions described so far must be run in serial on Processor 0, the next step is to start the parallel computation by spreading the submeshes over the processors. The communication is carried out by
    … …  
    131131
    132132\begin{figure}[hbtp]
    133   \centerline{ \includegraphics[scale = 0.6]{subdomainfinal.eps}}
     133  \centerline{ \includegraphics[scale = 0.6]{figures/subdomainfinal.eps}}
    134134  \caption{An example subpartioning after the submeshes have been renumbered.}
    135135 \label{fig:subdomainf}
    … …  
    140140\section{Some Example Code}
    141141
    142 \begin{figure}
     142Chapter \ref{chap:code} gives full listings of some example codes.
     143
     144The first example in Section \ref{sec:codeRPA} solves the advection equation on a
     145rectangular mesh. A rectangular mesh is highly structured so a coordinate based decomposition can be use and the partitioning is simply done by calling the
     146routine \code{parallel_rectangle} as show below.
     147\begin{verbatim}
     148#######################
     149# Partition the domain
     150#######################
     151
     152# Build a unit mesh, subdivide it over numproces processors with each
     153# submesh containing M*N nodes
     154
     155points, vertices, boundary, full_send_dict, ghost_recv_dict =  \
     156    parallel_rectangle(N, M, len1_g=1.0)
     157\end{verbatim}
     158
     159This rectangular mesh is artificial, and the approach to subpartitioning the mesh is different to the one described above, however this example may be of interest to those who want to measure the parallel efficiency of the code on their machine. A rectangular mesh should give a good load balance and is therefore an important first test problem. 
     160
     161
     162A more \lq real life\rq\ mesh is the Merimbula mesh used in the code shown in Section \ref{sec:codeRPMM}. This example also solves the advection equation. In this case the techniques described in Section \ref{sec:part} must be used to partition the mesh. Figure \ref{fig:code} shows the part of the code that is responsible for spreading the domain over the processors. We now look at the code in detail.
     163
     164\begin{figure}[htbp]
    143165\begin{verbatim}
    144166if myid == 0:
    … …  
    146168    # Read in the test files
    147169
    148     filename = 'merimbula_10785_1.tsh'
    149 
    150     # Build the whole domain
    151    
    152     domain_full = pmesh_to_domain_instance(filename, Domain)
     170    filename = 'merimbula_10785.tsh'
     171
     172    domain_full = pmesh_to_domain_instance(filename, Advection_Domain)
     173    domain_full.set_quantity('stage', Set_Stage(756000.0,756500.0,4.0))
    153174
    154175    # Define the domain boundaries for visualisation
    155 
     176   
    156177    rect = array(domain_full.xy_extent, Float)
    157178
    158     # Initialise the wave
    159 
    160     domain_full.set_quantity('stage', Set_Stage(756000.0,756500.0,2.0))
    161 
    162179    # Subdivide the mesh
    163    
    164     nodes, triangles, boundary, triangles_per_proc, quantities = \
    165          pmesh_divide_metis(domain_full, numprocs)
     180
     181    nodes, triangles, boundary, triangles_per_proc, quantities  =\
     182            pmesh_divide_metis(domain_full, numprocs)
    166183
    167184    # Build the mesh that should be assigned to each processor,
    168185    # this includes ghost nodes and the communicaiton pattern
    169186   
    170     submesh = build_submesh(nodes, triangles, boundary,\
    171                             quantities, triangles_per_proc)
     187    submesh = build_submesh(nodes, triangles, boundary, quantities, \
     188                            triangles_per_proc)
    172189
    173190    # Send the mesh partition to the appropriate processor
    … …  
    183200
    184201else:
    185    
    186202    # Read in the mesh partition that belongs to this
    187203    # processor (note that the information is in the
    188     # correct form for the GA data structure)
    189 
    190     points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict \
    191             = rec_submesh(0)
    192 \end{verbatim}
    193 \end{figure}
     204    # correct form for the GA data structure
     205
     206    points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict = \
     207             rec_submesh(0)
     208\end{verbatim}
     209  \caption{A section of code taken from {\tt run_parallel_merimbula_metis.py} (Section \protect \ref{sec:codeRPMM}) showing how to subdivide the mesh.}
     210 \label{fig:code}
     211\end{figure}
     212\newpage
     213\begin{itemize}
     214\item
     215These first few lines of code read in and define the (global) mesh.
     216\begin{verbatim}
     217    filename = 'merimbula_10785.tsh'
     218    domain_full = pmesh_to_domain_instance(filename, Advection_Domain)
     219    domain_full.set_quantity('stage', Set_Stage(756000.0,756500.0,4.0))
     220\end{verbatim}
     221
     222\item
     223The \code{rect} array is used by the visualiser and records the domain size.
     224\item \code{pmesh_divide_metis} divides the mesh into a set of non-overlapping subdomains as described in Section \ref{sec:part1}.
     225\begin{verbatim}
     226    nodes, triangles, boundary, triangles_per_proc, quantities  =\
     227            pmesh_divide_metis(domain_full, numprocs)
     228\end{verbatim}
     229
     230\item The next step is to build a boundary layer of ghost triangles and define the communication pattern. This step is implemented by \code{build_submesh} as discussed in Section \ref{sec:part2}.
     231\begin{verbatim}       
     232    submesh = build_submesh(nodes, triangles, boundary, quantities, \
     233                            triangles_per_proc)
     234\end{verbatim}
     235
     236\item The actual parallel communication starts when the submesh partitions are sent to the processors by calling \code{send_submesh}.
     237\begin{verbatim}
     238    for p in range(1, numprocs):
     239      send_submesh(submesh, triangles_per_proc, p)
     240\end{verbatim}
     241
     242The processors receive a given subpartition by calling \code{rec_submesh}. The \code{rec_submesh} routine also calls \code{build_local_mesh}. The \code{build_local_mesh} routine described in Section \ref{sec:part4} ensures that the information is stored in a way that is compatible with the Domain datastructure. This means, for example, that the triangles and nodes must be numbered consecutively starting from 1.
     243\begin{verbatim}
     244    points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict = \
     245             rec_submesh(0)
     246\end{verbatim}
     247
     248Note that the submesh is not received by, or sent to, Processor 0. Rather     \code{hostmesh = extract_hostmesh(submesh)} extracts the appropriate information. This saves the cost of an unnecessary communication call. It is described further in Section \ref{sec:part3}.
     249\begin{verbatim}
     250    hostmesh = extract_hostmesh(submesh)
     251    points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict = \
     252             build_local_mesh(hostmesh, 0, triangles_per_proc[0], numprocs)
     253\end{verbatim}
     254
     255\end{itemize}
    194256
    195257\section{Running the Code}

  • inundation/parallel/documentation/visualisation.tex

    r2723 r2786  
    3030\end{itemize}
    3131Screenshot:\\
    32 \includegraphics{vis-screenshot.eps}\\
     32\includegraphics{figures/vis-screenshot.eps}\\
    3333
    3434Unlike the old VPython visualiser, the behaviour of the VTK
Note: See TracChangeset for help on using the changeset viewer.