Isorropia: Partitioning, Coloring, and Ordering

Version 3.0

Introduction

Isorropia is a package for combinatorial scientific computing, with focus on partitioning (load balancing), but also supports coloring and ordering. Its main purpose is to assist with redistributing objects such as matrices and matrix-graphs in a parallel execution setting, to allow for more efficient computations. Isorropia produces good maps for Epetra matrices/graphs. Isorropia should be called after the matrix/graph is filled, so the sparsity pattern is known.

Isorropia is primarily an interface to the Zoltan toolkit, which is now required. Zoltan became a Trilinos package in 9.0 and is automatically enabled by Isorropia. To enable Isorropia in your Trilinos build, provide the argument '--enable-isorropia' to configure. If you wish to use third-party libraries (such as ParMetis or PT-Scotch) with Isorropia via Zoltan, see the configure options for Zoltan.

• Overview of Isorropia
• Partitioning
• Example Programs
• Coloring
• Ordering

Overview of Isorropia.

Isorropia currently contains algorithms for three separate problems, all of which can be viewed as a combinatorial problem based on graphs:

• Partitioning (load-balancing) and data redistribution.
• Graph coloring.
• Fill-reducing ordering (for sparse direct solvers).

Isorropia contains separate classes to solve each of these problems, all of which derive from Isorropia::Operator and Isorropia::EpetraOperator. Note that an operator in Isorropia is different from an operator in Epetra!

Partitioning and Load Balancing

We focus on the partitioning (load balancing), which is the most heavily used feature. Isorropia's load balancing user interface consists primarily of the Isorropia::Partitioner, Isorropia::Redistributor and Isorropia::CostDescriber classes, as well as the Isorropia::Epetra::create_balanced_copy functions.

The Isorropia::create_balanced_copy functions are free-standing functions (not class members) which simply take an input object (Epetra_CrsGraph, Epetra_CrsMatrix, etc.) and return a copy which is balanced across processes (parts). The reference-counter pointer class Teuchos::RCP is used.

In many use cases, such as performing one partitioning and then using it to redistribute multiple objects, it is more efficient to use the Isorropia::Partitioner and Isorropia::Redistributor classes. The general usage model is to:

1. Create a Isorropia::Partitioner instance (providing as input the matrix-graph or row-matrix object that is to be partitioned, and optionally a Isorropia::CostDescriber instance with weights/costs to influence the partitioning).
2. Create a Isorropia::Redistributor object (providing as input a Isorropia::Partitioner instance)
3. Use the Isorropia::Redistributor to redistribute one or more objects to the new partitioning.

Isorropia was designed such that these steps can usually be done in three lines of code. For example, the Partitioner will by default compute the partitioning at construction time.

Isorropia currently supports partitioning/redistributing of several Epetra objects, including Epetra_CrsGraph, Epetra_RowMatrix (which includes Epetra_CrsMatrix and Epetra_VbrMatrix) and Epetra_MultiVector.

Several Isorropia classes and methods accept Teuchos::ParameterList objects, allowing the user to control certain behavior. Isorropia parameters can be either lower or upper case.

• Choice of partitioner
  By default Zoltan is used as the partitioner (with hypergraph model of sparse matrices). In rare cases one may wish to use the simple built-in linear partitioner instead. Set the following parameter:

    Teuchos::ParameterList params;
    params.set("PARTITIONING_METHOD", "SIMPLE_LINEAR");
  

• Passing parameters through to Zoltan
  Isorropia can relay Zoltan parameters directly to Zoltan. Before passing a Teuchos::ParameterList to Isorropia, create a sublist in it named "Zoltan". Then, any parameters that should be passed to Zoltan itself, can be stored in the sublist exactly as they appear in the Zoltan users guide. Example:

    Teuchos::ParameterList params;
    Teuchos::ParameterList& sublist = params.sublist("Zoltan");
    sublist.set("LB_METHOD", "GRAPH"); // Only works for symmetric matrices!
  

  Refer to the Zoltan User's guide (available at the Zoltan web site) for available Zoltan parameters. In many cases, no parameters are necessary. Note that Isorropia will override a few default Zoltan settings: The default method (LB_METHOD) is HYPERGRAPH. The default approach (LB_APPROACH) is PARTITION. Isorropia will balance the number of nonzeros in a matrix, not the rows.

Example Programs

A number of example programs which demonstrate using Isorropia are provided in the 'packages/isorropia/example' subdirectory.

matrix_1.cpp demonstrates creating a balanced copy of Epetra_CrsGraph and Epetra_CrsMatrix objects using Isorropia::Epetra::create_balanced_copy functions.

part_redist.cpp demonstrates repartitioning and redistributing the contents of an Epetra_LinearProblem object, using the Isorropia::Partitioner and Isorropia::Redistributor classes. This program does not use user-specified weights/costs.

Other programs in the example subdirectory demonstrate the use of weights/costs data to influence the partitioner. See vert_weights.cpp, graphedge_weights.cpp and hgedge_weights.cpp.

Coloring

Isorropia supports vertex coloring of a graph via Zoltan. Both distance-1 and distance-2 coloring is supported. See the Zoltan User's guide (available at the Zoltan web site) for details. Currently, coloring is limited to structurally symmetric matrices but we expect to support nonsymmetric matrices (e.g. Jacobians) in the future.

Ordering

Isorropia currently supports a subset of the graph (sparse matrix) ordering features in Zoltan. The intended main use is to produce a permutation that can be used as fill-reducing ordering for direct solvers. Currently Zoltan relies on third-party libraries (ParMetis and PT-Scotch) for this.