Helper fuctions for applying

Functions

void RTOp_MPI_type_signature (const int num_values, const int num_indexes, const int num_chars, int *num_entries, int block_lengths[], MPI_Aint displacements[], MPI_Datatype datatypes[])
int RTOp_extract_reduct_obj_ext_state (const struct RTOp_RTOp *op, RTOp_ReductTarget reduct_obj, int num_values, int num_indexes, int num_chars, void *reduct_obj_ext)
int RTOp_load_reduct_obj_ext_state (const struct RTOp_RTOp *op, const void *reduct_obj_ext, RTOp_ReductTarget reduct_obj)
int RTOp_MPI_apply_op (MPI_Comm comm, const struct RTOp_RTOp *op, int root_rank, const int num_cols, const int num_vecs, const struct RTOp_SubVector sub_vecs[], const int num_targ_vecs, const struct RTOp_MutableSubVector sub_targ_vecs[], RTOp_ReductTarget reduct_objs[])

Detailed Description

reduction/transformation operations in C with MPI.


Function Documentation

void RTOp_MPI_type_signature ( const int  num_values,
const int  num_indexes,
const int  num_chars,
int *  num_entries,
int  block_lengths[],
MPI_Aint  displacements[],
MPI_Datatype  datatypes[] 
)

Initialize MPI compatible type signature arrays for reduction/transformation operator object instance data and reduction target object data.

Parameters:
num_values [in] Number of value members
num_indexes [in] Number of index members
num_chars [in] Number of character members
num_entries [out] Number of entries in output arrays set
block_lengths [out] array (length >= RTOp_NUM_DATA_TYPES)
displacements [out] array (length >= RTOp_NUM_DATA_TYPES)
datatypes [out] array (length >= RTOp_NUM_DATA_TYPES)

See the MPI function MPI_Type_struct(...) for a discription of these arrays.

Definition at line 40 of file RTOpToMPI.c.

int RTOp_extract_reduct_obj_ext_state ( const struct RTOp_RTOp op,
RTOp_ReductTarget  reduct_obj,
int  num_values,
int  num_indexes,
int  num_chars,
void *  reduct_obj_ext 
)

Fill a compacted representation for a reduction object.

Parameters:
op [in] RTOp operator object
reduct_obj [in] Reduction object
num_values [in] Number of value members
num_indexes [in] Number of index members
num_chars [in] Number of character members
reduct_obj_ext [out] (size = sizeof(RTOp_value_type)*(3 + num_values) + sizeof(RTOp_index_type)*num_indexes + sizeof(RTOp_char_type)*num_chars). On output, this memory will be set according to as described in RTOp_reduct_op_func_ptr_t.
Returns:
Returns 0 if successful otherwise returns an error code.

Definition at line 75 of file RTOpToMPI.c.

int RTOp_load_reduct_obj_ext_state ( const struct RTOp_RTOp op,
const void *  reduct_obj_ext,
RTOp_ReductTarget  reduct_obj 
)

Copy from a compacted representation for a reduction object.

Parameters:
op [in] RTOp operator object
reduct_obj_ext [in] (size = sizeof(RTOp_value_type)*(3 + num_values) + sizeof(RTOp_index_type)*num_indexes + sizeof(RTOp_char_type)*num_chars) On input, this memory should be set according to as described in RTOp_reduct_op_func_ptr_t.
reduct_obj [in/out] Reduction object set to the values in reduct_obj_ext (must have been previously constructed)
Returns:
Returns 0 if successful otherwise returns an error code.

Definition at line 104 of file RTOpToMPI.c.

int RTOp_MPI_apply_op ( MPI_Comm  comm,
const struct RTOp_RTOp op,
int  root_rank,
const int  num_cols,
const int  num_vecs,
const struct RTOp_SubVector  sub_vecs[],
const int  num_targ_vecs,
const struct RTOp_MutableSubVector  sub_targ_vecs[],
RTOp_ReductTarget  reduct_objs[] 
)

Apply a reduction operation over a set of local sub-vectors using MPI.

Parameters:
comm [in] MPI communicator
op [in] reduction/transformation operator object
root_rank [in] See below
num_cols [in] The number of columns for each vector
num_vecs [in] See RTOpPack::RTOp::apply_op()
sub_vecs [in] Array (size num_targ_vecs * num_cols) of nonmutable subvectors. The vectors for each column kc begin at sub_vecs+kc*num_vecs where kc=0...num_cols-1 Can be NULL if there are no local vector elements.
num_targ_vecs [in] See RTOpPackRTOp::apply_op()
sub_targ_vecs [in] Array (size num_targ_vecs * num_cols) of mutable subvectors. The vectors for each column kc begin at sub_vecs+kc*num_targ_vecs where kc=0...num_cols-1 Can be NULL if there are no local vector elements.
reduct_objs [in/out] Array (size num_cols) See below. If reduct_objs != NULL then on output, reduct_objs[i] will contain the reduction target over all the processes along with the reduction on input if reduct_objs[i] has already been through one or more reductions already and not reinitialized.
Returns:
Returns 0 if successful otherwise returns an error code. This function encapsulates a lot of the details of using MPI to perform reduction operations. For this function to work properly, each processor must contain a single sub-vector for each vector argument and every process (in the communicator) must call this function. Each process's sub-vector for each vector argument may be a different size and even dense on some processes and sparse in others. It is also allowed for a process to have empty sub-vectors, but still needs to participate in the collective operation and may want the value of the reduction object returned. The main responsibly of the client is to setup the sub_vecs[] and sub_targ_vecs[] arguments and to deside what type of reduction operation to perform (MPI_Allreduce() or MPI_Reduce()).

Let rank be the value returned from MPI_Comm_rank(comm,&rank) in this process. The expected arguments to this function depend on the argument root_rank and rank as follows:

  • root_rank < 0 : In this case we are performing a MPI_Allreduce(...) reduction operation over all of the processes with the results collected in all of the processes. In this case reduct_obj!=RTOp_REDUCT_OBJ_NULL must be true in all of the processes.
    • root_rank < 0 in all processes
    • reduct_obj != RTOp_REDUCT_OBJ_NULL in all processes if a reduction is being performed.
  • root_rank >= 0 : In this case we are performing a MPI_Reduce(...) reduction operation over all of the processes with the result only collected in the process with rank root_rank. Here all of the processes must pass the same value for root_rank. The reduction target object is only passed in for the processes with rank == root_rank.
    • root_rank same in all of the processes
    • If rank == root_rank then reduct_obj != RTOp_REDUCT_OBJ_NULL
    • If rank != root_rank then reduct_obj == RTOp_REDUCT_OBJ_NULL

Note that if a reduction operation is not performed then no synchronization of the the processes are performed. This is the fastest behavior and should be fine for a single thread but may cause problems with multiple threads.

Definition at line 131 of file RTOpToMPI.c.

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Friends
Generated on Wed Apr 13 10:09:10 2011 for RTOpPack: Extra C/C++ Code for Vector Reduction/Transformation Operators by  doxygen 1.6.3