Amesos: An Interface to Serial and Parallel Direct Sparse Solver Packages.

TrilinosRelease5.0Branch

Table of Contents

Amesos provides an object-oriented interface to several direct sparse solvers, both sequential and parallel. Amesos is developed by M. Sala, K. Stanley, R. Hoekstra, T. Davis, and M. Heroux.

The main page of the Doxygen documentation is organized as follows:

• Introduction
• Copyright and licensing of the third party codes
• Quick comparison of the Amesos classes
• Supported Matrix Formats.
• Installing Amesos
• Example of Use of Amesos Classes
• Known bugs and future developments
• Final Remarks

Introduction

Amesos supports the following classes:

• Amesos_Lapack - Interface to LAPACK's DGETRF. Amesos_Lapack is the only dense solver supported by Amesos.
• Amesos_Klu - Interface to Tim Davis serial solver KLU (distributed within Amesos).
• Amesos_Umfpack - Interface to Tim Davis's UMFPACK (version 4.3 or later)
• Amesos_Superlu - Interface to Xiaoye Li's SuperLU serial memory code with serial input interface.
• Amesos_Superludist - Interface to Xiaoye Li's SuperLU Distributed memory code with serial input interface (version 2.0 or later).
• Amesos_Dscpack - Interface to Padma Raghavan's DSCPACK
• Amesos_Mumps - Interface to CERFACS' MUMPS (version 4.3.1 or later)

We refer to the Sandia report SAND-2004-2188 for a detailed overview of Amesos. A PDF version of this report can be found the Trilinos/packages/amesos/doc/AmesosReferenceGuide/AmesosReferenceGuide.pdf.

Copyright and licensing of the third party codes

Most of the Amesos classes are based on a third party code (that is, not distributed within Trilinos). Each third party code comes with its own copyright and/or licensing requirements. It is responsibility of the user to fulfill the requirements of each supported package' copyright.

Most of these third party codes are intended to be made available at no cost to users. Much of the copyright and licensing restrictions concern rights to modify, redistribute the code and generally include a request that credit be given in any papers which make use of their code. Please refer to the web page for the package that you are interested in for details.

Quick comparison of the Amesos classes

The following table reports an overview of the Amesos classes.

Features	Amesos_Lapack	Amesos_Klu	Amesos_Umfpack	Amesos_Superlu	Amesos_Superludist	Amesos_Mumps	Amesos_Dscpack
Default behavior	Enabled	Enabled	Disabled	Disabled	Disabled	Disabled	Disabled
Package is written in	FORTRAN77	C	C	C	C	FORTRAN90	C
Communicator	none	none	none	none	MPI	FORTRAN MPI	MPI
Processes used for Factorization/Solve	Process 0	Process 0	Process 0	Process 0	Any	Any (*)	Any (**)
Distributed input matrix	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Unsymmetric matrices	Yes	Yes	Yes	Yes	Yes	Yes	No
Requires BLAS	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Requires LAPACK	Yes	No	No	No	No	No	No
Requires BLACS	No	No	No	No	No	Yes	No
Requires ScaLAPACK	No	No	No	No	No	Yes	No

(*) MUMPS required FORTRAN communicators. Some architectures (e.g., SGI) does not allow portable conversions from C/C++ communicator and FORTRAN. On these architectures, therefore, MPI_COMM_WORLD will be used by MUMPS. Instead, if C/C++ communicators can be converted to FORTRAN ones, then any number of prosesses can be used by Amesos_Mumps.

(**) DSCPACK requires a number of processes that is a power of 2.

Supported Matrix Formats.

The following table details the supported matrix formats for each class.

	Amesos_Lapack	Amesos_Klu	Amesos_Umfpack	Amesos_Superlu	Amesos_Superludist	Amesos_Mumps	Amesos_Dscpack
Epetra_RowMatrix	yes	yes	yes	yes	yes	yes	no
Epetra_CrsMatrix	yes	yes	yes	yes	yes	yes	yes
Epetra_VbrMatrix	yes	yes	yes	yes	yes	yes	no

Installing Amesos

Each of the Amesos classes provides an interface to a third party direct sparse solver code. In order to install a particular class, you must first install the underlying direct sparse solver code.

Amesos is distributed through the Trilinos project, and can be downloaded from the web site http://software.sandia.gov/trilinos/packages/amesos.

As all other Trilinos packages, Amesos is configured and built using the GNU autoconf and automake tools. To configure Amesos from the Trilinos top directory, a possible procedure is as follows. Let TRILINOS_HOME be a shell variable representing the location of the Trilinos source directory, and the shell prompt sign. Let us suppose that we want to configure Amesos on a LINUX machine with MPI, with support for KLU and UMFPACK. Header files for UMFPACK are located in directory /usr/local/umfpack/include, while the library, called libumfpack.a is located in /usr/local/umfpack/lib. The configure like will look like:

% cd $TRILINOS_HOME
% mkdir LINUX_MPI
% cd LINUX_MPI
% ../configure \
  --with-mpi-compilers \
  --prefix=$TRILINOS_HOME/LINUX_MPI \
  --enable-amesos \
  --enable-amesos-klu \
  --enable-amesos-umfpack \
  --with-incdirs="-I/usr/local/umfpack/include" \
  --with-ldflags="-L/usr/local/umfpack/lib" \
  --with-libs="-lumfpack"
% make
% make install

Other flags may be required depending on the location of MPI, BLAS and LAPACK. The table below reports the architectures and compilers tested with Amesos.

architecture	Amesos_Lapack	Amesos_Klu	Amesos_Umfpack	Amesos_Superlu	Amesos_Superludist	Amesos_Mumps	Amesos_Dscpack
LINUX (GNU), SERIAL	yes	yes	yes	yes	yes	no	yes
LINUX (Intel), MPI	yes	yes	yes	no	no	yes	no
LINUX (GNU), MPI	yes	yes	yes	yes	yes	no	yes
SGI 64, MPI	yes	yes	yes	no	no	yes	no
DEC/Alpha MPI	yes	yes	yes	no	no	no	no
MAC OSX, SERIAL	no	yes	yes	no	no	no	no
Sandia Cplant, MPI	no	yes	yes	yes	yes	no	no
Sandia ASCI Red, MPI	no	yes	yes	yes	yes	no	no

Once Amesos has been compiled, it can be tested using the test harness. A quick test (using MPI) can be as follows:

% cd amesos/test/Test_Basic
% make
% ./TestBasic.csh LAPACK
% ./TestBasic.csh KLU
% ./TestBasic.csh <other-solvers>

where <other-solvers> can be: SUPERLU, SUPERLUDIST, MUMPS, DSCPACK. The tests exercise some of the Amesos features. Other tests are available for more intensive analysis.

Example of Use of Amesos Classes

A simple fragment of code using Amesos can read as follows. First, we need to include the header files for Amesos:

#include "Amesos.h"
#include "Amesos_BaseSolver.h"

Note that these header files will not include the header files for the supported libraries (which are of course needed to compile the Amesos library itself). Now, let define the linear system matrix, the vector that will contain the solution, and the right-hand side as:

Epetra_LinearProblem Problem;
Epetra_RowMatrix* A;            // linear system matrix
Epetra_MultiVector* LHS;        // vector that will contain the solution
Epetra_MultiVector* RHS;        // right-hand side

All Amesos object (derived from pure virtual class Amesos_BaseSolver) can be created using the function class Amesos, as follows:

Amesos_BaseSolver * Solver;
Amesos Factory;
char* SolverType = "Amesos_Klu";
Solver = Factory.Create(SolverType, *Problem);

Note:

It is important to note that all available solvers can be selected simply by changing an input parameter. Some solvers are serial, other parallel; generally, each solver has its own matrix format. However, the user will still have the same interface to all of them.

The Factory object will create an Amesos_Klu object (if Amesos has been configure to support this solver). Factory.Create() returns 0 if the requested solver is not available. Parameter names are case-sensitive; misspelled parameters will not be recognized. Method Factory.Query() can be used to query the factory about the availability of a given solver:

char* SolverType = "Amesos_Klu";
bool IsAvailable = Factory.Query(SolverType);

Here, we simply recall that the parameters list can be created as

Teuchos::ParameterList List;

and parameters can be set as

List.set(ParameterName,ParameterValue);

Here, ParameterName is a string containing the parameter name, and ParameterValue is any valid C++ object that specifies the parameter value (for instance, an integer, a pointer to an array or to an object). Please consult the Amesos Reference Guide for more details. The Doxygen documentation of each class can also be of help.

Note that Problem is still empty. After setting the pointer to the linear system matrix, we can perform the symbolic factorization of the linear system matrix:

AMESOS_CHK_ERR(Solver->SymbolicFactorization());

(AMESOS_CHK_ERR is a macro, that checks the return error and, if not null, prints out a message and returns.) This phase does not require the numerical values of A, that can therefore be changed after the call to SymbolicFactorization(). However, the nonzero pattern of A cannot be changed.

The numeric factorization is performed by

AMESOS_CHK_ERR(Solver->NumericFactorization());

The values of RHS must be set before solving the linear system, which simply reads

Problem.SetLHS(LHS);
Problem.SetRHS(RHS);
AMESOS_CHK_ERR(Solver->Solve());

Should the user need to re-factorize the matrix, he or she must call NumericFactorization(). If the structure of the matrix is changed, he or she must call SymbolicFactorization(). However, it is supposed that the linear system matrix and the solution and right-hand side vectors are still defined with the same Epetra_Map.

Please consult the examples reported in the amesos/example subdirectory:

• am_factory shows the use of the Amesos factory class;
• am_compare reports how to compare the time required by different solvers;
• am_hb details how to read a Harwell/Boeing matrix

Known bugs and future developments

Depending on the version of SuperLU and SuperLU_DIST, you may or may not configure Amesos with support for both. This is due to conflicts in the header files.

You may encounter problems while trying to use the SuperLU/SuperLU_DIST classes with KLU enables. This is due to a conflict in the colamd algorithm. The version of colamd which is distributed within Amesos is not compatible with the version distributed with SuperLU. This cause the code to crash. You might need to disable KLU to use SuperLU/SuperLU_DIST.

DSCPACK does not support Epetra_RowMatrix's and Epetra_VbrMatrix. DSCPACK has memory leaks. A fix is under development for both of these issues.

Amesos has an incomplete interface to ScaLAPACK under development.

Final Remarks

This page is maintained by Marzio Sala, SNL 9214.

This page last updated 04-Nov-2004.

---