Changeset 2786
- Timestamp:
- May 1, 2006, 10:36:09 AM (19 years ago)
- Location:
- inundation/parallel/documentation
- Files:
-
- 6 edited
Legend:
- Unmodified
- Added
- Removed
-
inundation/parallel/documentation/code.tex
r2768 r2786 1 1 2 \chapter{Code Listing} 3 \section{run_parallel_advection.py} 4 \verbatiminput{ RunParallelAdvection.py}2 \chapter{Code Listing}\label{chap:code} 3 \section{run_parallel_advection.py}\label{sec:codeRPA} 4 \verbatiminput{code/RunParallelAdvection.py} 5 5 \newpage 6 \section{run_parallel_merimbula_metis.py} 7 \verbatiminput{ RunParallelMerimbulaMetis.py}6 \section{run_parallel_merimbula_metis.py}\label{sec:codeRPMM} 7 \verbatiminput{code/RunParallelMerimbulaMetis.py} 8 8 \newpage 9 \section{run_parallel_sw_merimbula_metis.py} 10 \verbatiminput{ RunParallelSwMerimbulaMetis.py}9 \section{run_parallel_sw_merimbula_metis.py}\label{sec:codeRPSMM} 10 \verbatiminput{code/RunParallelSwMerimbulaMetis.py} -
inundation/parallel/documentation/code/RunParallelAdvection.py
r2785 r2786 86 86 T = Transmissive_boundary(domain) 87 87 domain.default_order = 2 88 domain.set_boundary( {'left': T, 'right': T, 'bottom': T, 'top': T, 'ghost': None} ) 88 domain.set_boundary( {'left': T, 'right': T, 'bottom': T, 'top': T, \ 89 'ghost': None} ) 89 90 90 91 # Ensure that the domain definitions make sense … … 112 113 if myid == 0: 113 114 print 'That took %.2f seconds' %(time.time()-t0) 114 print 'Communication time %.2f seconds'%domain.communication_time 115 print 'Reduction Communication time %.2f seconds'%domain.communication_reduce_time 115 print 'Communication time %.2f seconds'\ 116 %domain.communication_time 117 print 'Reduction Communication time %.2f seconds'\ 118 %domain.communication_reduce_time -
inundation/parallel/documentation/code/RunParallelMerimbulaMetis.py
r2785 r2786 96 96 rect = array(domain_full.xy_extent, Float) 97 97 98 # Subdivide the mes 98 # Subdivide the mesh 99 99 100 100 nodes, triangles, boundary, triangles_per_proc, quantities =\ … … 154 154 T = Transmissive_boundary(domain) 155 155 #R = Reflective_boundary(domain) 156 domain.set_boundary( {'outflow': T, 'inflow': T, 'inner':T, 'exterior': T, 'open':T, 'ghost':None} ) 156 domain.set_boundary( {'outflow': T, 'inflow': T, 'inner':T, \ 157 'exterior': T, 'open':T, 'ghost':None} ) 157 158 158 159 # Set the initial quantities -
inundation/parallel/documentation/code/RunParallelSwMerimbulaMetis.py
r2785 r2786 189 189 print 'That took %.2f seconds' %(time.time()-t0) 190 190 print 'Communication time %.2f seconds'%domain.communication_time 191 print 'Reduction Communication time %.2f seconds'%domain.communication_reduce_time 192 print 'Broadcast time %.2f seconds'%domain.communication_broadcast_time 191 print 'Reduction Communication time %.2f seconds'\ 192 %domain.communication_reduce_time 193 print 'Broadcast time %.2f seconds'\ 194 %domain.communication_broadcast_time -
inundation/parallel/documentation/parallel.tex
r2767 r2786 1 1 \chapter{Running the Code in Parallel} 2 2 3 This chapter looks at how to run the code in parallel. The first section gives an overview of the main steps required to divide the mesh over the processors. The second sections gives some example code and the final section talks about how to run the code on a few specific architectures.4 5 6 \section {Partitioning the Mesh} 3 This chapter looks at how to run the code in parallel. The first section gives an overview of the main steps required to divide the mesh over the processors. The second sections describes some example code and the final section talks about how to run the code on a few specific architectures. 4 5 6 \section {Partitioning the Mesh}\label{sec:part} 7 7 There are four main steps required to run the code in parallel. They are; 8 8 \begin{enumerate} … … 19 19 20 20 \begin{figure}[hbtp] 21 \centerline{ \includegraphics[scale = 0.75]{ domain.eps}}21 \centerline{ \includegraphics[scale = 0.75]{figures/domain.eps}} 22 22 \caption{The main steps used to divide the mesh over the processors.} 23 23 \label{fig:subpart} 24 24 \end{figure} 25 25 26 \subsection {Subdividing the Global Mesh} 26 \subsection {Subdividing the Global Mesh}\label{sec:part1} 27 27 28 28 The first step in parallelising the code is to subdivide the mesh … … 39 39 40 40 \begin{figure}[hbtp] 41 \centerline{ \includegraphics[scale = 0.75]{ mermesh.eps}}41 \centerline{ \includegraphics[scale = 0.75]{figures/mermesh.eps}} 42 42 \caption{The Merimbula grid.} 43 43 \label{fig:mergrid} … … 47 47 48 48 \begin{figure}[hbtp] 49 \centerline{ \includegraphics[scale = 0.75]{ mermesh4c.eps}50 \includegraphics[scale = 0.75]{ mermesh4a.eps}}51 52 53 \centerline{ \includegraphics[scale = 0.75]{ mermesh4d.eps}54 \includegraphics[scale = 0.75]{ mermesh4b.eps}}49 \centerline{ \includegraphics[scale = 0.75]{figures/mermesh4c.eps} 50 \includegraphics[scale = 0.75]{figures/mermesh4a.eps}} 51 52 53 \centerline{ \includegraphics[scale = 0.75]{figures/mermesh4d.eps} 54 \includegraphics[scale = 0.75]{figures/mermesh4b.eps}} 55 55 \caption{The Merimbula grid partitioned over 4 processors using Metis.} 56 56 \label{fig:mergrid4} … … 83 83 The number of submeshes found by Pymetis is equal to the number of processors; Submesh $p$ will be assigned to Processor $p$. 84 84 85 \subsection {Building the Ghost Layer} 85 \subsection {Building the Ghost Layer}\label{sec:part2} 86 86 The function {\tt build_submesh.py} is the work-horse and is responsible for 87 87 setting up the communication pattern as well as assigning the local numbering scheme for the submeshes. … … 91 91 92 92 \begin{figure}[hbtp] 93 \centerline{ \includegraphics[scale = 0.6]{ subdomain.eps}}93 \centerline{ \includegraphics[scale = 0.6]{figures/subdomain.eps}} 94 94 \caption{An example subpartioning.} 95 95 \label{fig:subdomain} … … 98 98 99 99 \begin{figure}[hbtp] 100 \centerline{ \includegraphics[scale = 0.6]{ subdomainghost.eps}}100 \centerline{ \includegraphics[scale = 0.6]{figures/subdomainghost.eps}} 101 101 \caption{An example subpartioning with ghost triangles.} 102 102 \label{fig:subdomaing} … … 117 117 Finally, the ANUGA code assumes that the triangles (and nodes etc.) are numbered consecutively starting from 1. Consequently, if Submesh 1 in Figure \ref{fig:subdomaing} was passed into the \code{evolve} calculations it would crash. The \code{build_submesh} function determines a local numbering scheme for each submesh, but it does not actually update the numbering, that is left to \code{build_local}. 118 118 119 \subsection {Sending the Submeshes} 119 \subsection {Sending the Submeshes}\label{sec:part3} 120 120 121 121 All of functions described so far must be run in serial on Processor 0, the next step is to start the parallel computation by spreading the submeshes over the processors. The communication is carried out by … … 131 131 132 132 \begin{figure}[hbtp] 133 \centerline{ \includegraphics[scale = 0.6]{ subdomainfinal.eps}}133 \centerline{ \includegraphics[scale = 0.6]{figures/subdomainfinal.eps}} 134 134 \caption{An example subpartioning after the submeshes have been renumbered.} 135 135 \label{fig:subdomainf} … … 140 140 \section{Some Example Code} 141 141 142 \begin{figure} 142 Chapter \ref{chap:code} gives full listings of some example codes. 143 144 The first example in Section \ref{sec:codeRPA} solves the advection equation on a 145 rectangular mesh. A rectangular mesh is highly structured so a coordinate based decomposition can be use and the partitioning is simply done by calling the 146 routine \code{parallel_rectangle} as show below. 147 \begin{verbatim} 148 ####################### 149 # Partition the domain 150 ####################### 151 152 # Build a unit mesh, subdivide it over numproces processors with each 153 # submesh containing M*N nodes 154 155 points, vertices, boundary, full_send_dict, ghost_recv_dict = \ 156 parallel_rectangle(N, M, len1_g=1.0) 157 \end{verbatim} 158 159 This rectangular mesh is artificial, and the approach to subpartitioning the mesh is different to the one described above, however this example may be of interest to those who want to measure the parallel efficiency of the code on their machine. A rectangular mesh should give a good load balance and is therefore an important first test problem. 160 161 162 A more \lq real life\rq\ mesh is the Merimbula mesh used in the code shown in Section \ref{sec:codeRPMM}. This example also solves the advection equation. In this case the techniques described in Section \ref{sec:part} must be used to partition the mesh. Figure \ref{fig:code} shows the part of the code that is responsible for spreading the domain over the processors. We now look at the code in detail. 163 164 \begin{figure}[htbp] 143 165 \begin{verbatim} 144 166 if myid == 0: … … 146 168 # Read in the test files 147 169 148 filename = 'merimbula_10785_1.tsh' 149 150 # Build the whole domain 151 152 domain_full = pmesh_to_domain_instance(filename, Domain) 170 filename = 'merimbula_10785.tsh' 171 172 domain_full = pmesh_to_domain_instance(filename, Advection_Domain) 173 domain_full.set_quantity('stage', Set_Stage(756000.0,756500.0,4.0)) 153 174 154 175 # Define the domain boundaries for visualisation 155 176 156 177 rect = array(domain_full.xy_extent, Float) 157 178 158 # Initialise the wave159 160 domain_full.set_quantity('stage', Set_Stage(756000.0,756500.0,2.0))161 162 179 # Subdivide the mesh 163 164 nodes, triangles, boundary, triangles_per_proc, quantities =\165 pmesh_divide_metis(domain_full, numprocs)180 181 nodes, triangles, boundary, triangles_per_proc, quantities =\ 182 pmesh_divide_metis(domain_full, numprocs) 166 183 167 184 # Build the mesh that should be assigned to each processor, 168 185 # this includes ghost nodes and the communicaiton pattern 169 186 170 submesh = build_submesh(nodes, triangles, boundary, \171 quantities,triangles_per_proc)187 submesh = build_submesh(nodes, triangles, boundary, quantities, \ 188 triangles_per_proc) 172 189 173 190 # Send the mesh partition to the appropriate processor … … 183 200 184 201 else: 185 186 202 # Read in the mesh partition that belongs to this 187 203 # processor (note that the information is in the 188 # correct form for the GA data structure) 189 190 points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict \ 191 = rec_submesh(0) 192 \end{verbatim} 193 \end{figure} 204 # correct form for the GA data structure 205 206 points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict = \ 207 rec_submesh(0) 208 \end{verbatim} 209 \caption{A section of code taken from {\tt run_parallel_merimbula_metis.py} (Section \protect \ref{sec:codeRPMM}) showing how to subdivide the mesh.} 210 \label{fig:code} 211 \end{figure} 212 \newpage 213 \begin{itemize} 214 \item 215 These first few lines of code read in and define the (global) mesh. 216 \begin{verbatim} 217 filename = 'merimbula_10785.tsh' 218 domain_full = pmesh_to_domain_instance(filename, Advection_Domain) 219 domain_full.set_quantity('stage', Set_Stage(756000.0,756500.0,4.0)) 220 \end{verbatim} 221 222 \item 223 The \code{rect} array is used by the visualiser and records the domain size. 224 \item \code{pmesh_divide_metis} divides the mesh into a set of non-overlapping subdomains as described in Section \ref{sec:part1}. 225 \begin{verbatim} 226 nodes, triangles, boundary, triangles_per_proc, quantities =\ 227 pmesh_divide_metis(domain_full, numprocs) 228 \end{verbatim} 229 230 \item The next step is to build a boundary layer of ghost triangles and define the communication pattern. This step is implemented by \code{build_submesh} as discussed in Section \ref{sec:part2}. 231 \begin{verbatim} 232 submesh = build_submesh(nodes, triangles, boundary, quantities, \ 233 triangles_per_proc) 234 \end{verbatim} 235 236 \item The actual parallel communication starts when the submesh partitions are sent to the processors by calling \code{send_submesh}. 237 \begin{verbatim} 238 for p in range(1, numprocs): 239 send_submesh(submesh, triangles_per_proc, p) 240 \end{verbatim} 241 242 The processors receive a given subpartition by calling \code{rec_submesh}. The \code{rec_submesh} routine also calls \code{build_local_mesh}. The \code{build_local_mesh} routine described in Section \ref{sec:part4} ensures that the information is stored in a way that is compatible with the Domain datastructure. This means, for example, that the triangles and nodes must be numbered consecutively starting from 1. 243 \begin{verbatim} 244 points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict = \ 245 rec_submesh(0) 246 \end{verbatim} 247 248 Note that the submesh is not received by, or sent to, Processor 0. Rather \code{hostmesh = extract_hostmesh(submesh)} extracts the appropriate information. This saves the cost of an unnecessary communication call. It is described further in Section \ref{sec:part3}. 249 \begin{verbatim} 250 hostmesh = extract_hostmesh(submesh) 251 points, vertices, boundary, quantities, ghost_recv_dict, full_send_dict = \ 252 build_local_mesh(hostmesh, 0, triangles_per_proc[0], numprocs) 253 \end{verbatim} 254 255 \end{itemize} 194 256 195 257 \section{Running the Code} -
inundation/parallel/documentation/visualisation.tex
r2723 r2786 30 30 \end{itemize} 31 31 Screenshot:\\ 32 \includegraphics{ vis-screenshot.eps}\\32 \includegraphics{figures/vis-screenshot.eps}\\ 33 33 34 34 Unlike the old VPython visualiser, the behaviour of the VTK
Note: See TracChangeset
for help on using the changeset viewer.