Changeset 123 for pypar/DOC

Timestamp:

Jul 11, 2005, 3:20:03 PM (20 years ago)

Author:

ole

Message:

Files for 64 bit machine + latest Cvs version

File:

• pypar/DOC (modified) (4 diffs)

pypar/DOC

@@ -96 +96 @@
 
 PROGRAMMING FOR EFFICIENCY    
-  For really fast communication one must stick to Numeric arrays and use 
-  'raw' versions of send and receive, e.g.:
+  Really low latency communication can be achieved by sticking 
+  to Numeric arrays and specifying receive buffers whenever possible.
+
   To send a Numeric array A to processor p, write
-    pypar.raw_send(A, p)
+    pypar.raw_send(A, p, use_buffer=True)
   and to receive the array from processor q, write
-    X = pypar.raw_receive(X, q)    
+    X = pypar.receive(q, buffer=X)    
   Note that X acts as a buffer and must be pre-allocated prior to the 
   receive statement as in Fortran and C programs using MPI.
   
+  These forms have superseded the raw forms present in pypar
+  prior to version 1.9. The raw forms have been recast in terms of the 
+  above and have been retained for backwars compatibility.
   See the script pytiming for an example of communication of Numeric arrays.
       
@@ -123 +127 @@
   (See section on Data types for explanation of 'vanilla').  
   
+  Identification:
+  ---------------
+    
   size() -- Number of processors
   rank() -- Id of current processor
-  Get_processor_name() -- Return host name of current node  
-  
-  send(x, destination, tag=0, vanilla=0) -- Blocking send (all types)
-    Sends data in x to destination with given tag. 
-    
-  y=receive(source, tag=0) -- Blocking receive (all types)
-    receives data (y) from source (possible with specified tag).
-  
-  y, status=receive(source, tag=0, return_status=True) -- Blocking receive (all types)
-    receives data (y) and status object from source (possible with specified tag).    
-    
-  raw_send(x, destination, tag=0, vanilla=0): -- Blocking send (Fastest)
-    Sends data in x to destination with given tag.     
-    It differs from send in that the receiver MUST provide a buffer
-    to store the received data.
-    Although it will accept all types raw_send is thought to be used 
-    mainly for Numeric arrays.
-  
-  raw_receive(x, source, tag=0, vanilla=0):  -- Raw blocking receive (Fastest)
-    receives data from source (possible with specified tag) and puts
+  get_processor_name() -- Return host name of current node  
+  
+  Basic send forms:
+  --------------------
+  send(x, destination)
+    Sends data x of any type to destination with default tag. 
+        
+  send(x, destination, tag=t)
+    Sends data x of any type to destination with tag t. 
+    
+  send(x, destination, use_buffer=True)
+    Sends data x of any type to destination 
+    assuming that recipient will specify a suitable buffer.
+    
+  send(x, destination, bypass=True)
+    Send Numeric array of any type to recipient assuming 
+    that a suitable buffer has been specified and that 
+    recipient also specifies bypass=True
+        
+        
+  Basic receive forms:  
+  --------------------
+  y=receive(source)
+    receives data y of any type from source with default tag.
+      
+  y=receive(source, tag=t)
+    receives data y of any type from source with tag t.
+  
+  y,status=receive(source, return_status=True)
+    receives data y and status object from source 
+    
+  y=receive(source, buffer=x)    
+    receives data y from source and puts
     it in x (which must be of compatible size and type).
     It also returns a reference to x.
-    Although it will accept all types raw_send is thought to be used 
-    mainly for Numeric arrays.    
-
-  x, status = raw_receive(x, source, tag=0, vanilla=0, return_status=True):  -- Raw blocking receive (Fastest)
-    receives data and status object from source (possible with specified tag) and puts
-    it in x (which must be of compatible size and type).
-    
-
-  bcast(X, rootid) -- Broadcasts X from rootid to all other processors.
-                      All processors must issue the same bcast.
-
-
-  raw_scatter(x, nums, buffer, source, vanilla=0):  
-     Scatter the first nums elements in x to buffer
-     (of size given by nums) from source.
-
-    
-  scatter(x, source, vanilla=0):
-     Scatter all elements in x to a buffer 
-     created by this function and returned.
-
-
-  raw_gather(x, buffer, source, vanilla=0):
-     Gather all elements in x to buffer
+    (Although it will accept all types this form is thought to be used 
+    mainly for Numeric arrays).    
+
+  Collective Communication:
+  -------------------------  
+    
+  broadcast(x, root):
+    Broadcasts x from root to all other processors.
+    All processors must issue the same bcast.
+
+  gather(x, root):
+     Gather all elements in x to buffer of 
+     size len(x) * numprocs 
+     created by this function.
+     If x is multidimensional buffer will have
+     the size of zero'th dimension multiplied by numprocs.
+     A reference to the created buffer is returned.
+
+  gather(x, root, buffer=y):
+     Gather all elements in x to specified buffer y
      from source.
      Buffer must have size len(x) * numprocs and 
-     shape[0] == x.shape[0]*numprocs
-
-  gather(x, source, vanilla=0):
-     Gather all elements in x to buffer of 
-     size len(x) * numprocs
-     created by this function and returned.     
-     If x is multidimensional buffer will have
-     the size of zero'th dimension multiplied by numprocs
-
-
-  raw_reduce(x, buffer, op, source, vanilla=0):
-     Reduce all elements in x to buffer (of the same size as x)
+     shape[0] == x.shape[0]*numprocs.
+     A reference to the buffer y is returned.     
+
+  scatter(x, root):
+     Scatter all elements in x from root to all other processors
+     in a buffer created by this function.
+     A reference to the created buffer is returned.
+
+  scatter(x, root, buffer=y):
+     Scatter all elements in x from root to all other processors
+     using specified buffer y.
+     A reference to the buffer y is returned.     
+
+  reduce(x, op, root):
+     Reduce all elements in x at root
+     applying operation op elementwise and return result in 
+     buffer created by this function.  
+     A reference to the created buffer is returned.          
+
+  reduce(x, op, root, buffer=y):
+     Reduce all elements in x to specified buffer y
+     (of the same size as x)
      at source applying operation op elementwise.
+     A reference to the buffer y is returned.               
 
      
-  reduce(x, op, source, vanilla=0):
-     Reduce all elements in x at source
-     applying operation op elementwise and return result in new buffer.  
-     Buffer is created and returned.
-
+  Other functions:
+  ----------------     
                                          
-  Wtime() -- MPI wall time
-  Barrier() -- Synchronisation point. Makes processors wait until all 
+  time() -- MPI wall time
+  barrier() -- Synchronisation point. Makes processors wait until all 
                processors have reached this point.
-  Abort() -- Terminate all processes. 
-  Finalize() -- Cleanup MPI. No parallelism can take place after this point. 
-
+  abort() -- Terminate all processes. 
+  finalize() -- Cleanup MPI. No parallelism can take place after this point. 
+  initialized() -- True if MPI has been initialised
+
+  
   See pypar.py for doc strings on individual functions.   
 
-  
+
+    
 DATA TYPES
   Pypar automatically handles different data types differently
   There are three protocols:
-    'array': Numeric arrays of type 'i', 'l', 'f', or 'd' can be communicated
+    'array': Numeric arrays of type Int ('i', 'l'), Float ('f', 'd'),
+             or Complex ('F', 'D') can be communicated
              with the underlying mpiext.send_array and mpiext.receive_array. 
              This is the fastest mode.
+             Note that even though the underlying C implementation does not
+             support Complex as a native datatype, pypar handles them 
+             efficiently and seemlessly by transmitting them as arrays of 
+             floats of twice the size.  
     'string': Text strings can be communicated with mpiext.send_string and
               mpiext.receive_string.
@@ -215 +247 @@
                can be serialised using
                pickle (or cPickle). The latter mode is less efficient than the
-               first two but it can handle complex structures.
+               first two but it can handle general structures.
 
      Rules:
@@ -252 +284 @@
 PERFORMANCE
   If you are passing simple Numeric arrays around you can reduce 
-  the communication time by using the '_raw' versions of send and 
-  receive (see REFERENCE above). These version are closer to the underlying MPI 
+  the communication time by using the 'buffer' keyword arguments 
+  (see REFERENCE above). These version are closer to the underlying MPI 
   implementation in that one must provide receive buffers of the right size.
-  However, you will find that this can be somewhat faster as they bypass 
+  However, you will find that these version have lower latency and 
+  can be somewhat faster as they bypass 
   pypar's mechanism for automatically transferring the needed buffer size.
-  Also, using simple numeric arrays will bypass pypar's pickling of complex 
+  Also, using simple numeric arrays will bypass pypar's pickling of general 
   structures.