Trilinos/Tpetra: Templated Linear Algebra Services Package

Version 2.0

Introduction to Tpetra

Tpetra is an attempt at applying generic algorithm techniques to the serial and parallel linear algebra libraries of Epetra. It is hoped that Tpetra will complement Epetra by providing an alternative foundation for future Trilinos solvers. The lessons learned and discoveries made while creating Tpetra are also being applied to improving their Epetra cousins.

Tpetra provides the same functionality as Epetra: the fundamental construction routines and services functions that are required for serial and parallel linear algebra libraries. Unlike Epetra, Tpetra makes extensive use of templates and the Standard Template Library (STL). This provides numerous benefits, including the ability to template the ordinal and scalar fields on any well-defined types, and the use of automatic resource allocation.

Tpetra captures a significant subset of the functionality present in the Epetra package. Functionality not included is either a consequence of different design motivations or of Tpetra's status as a work in progress.

Overview of Tpetra

Templated Types in Tpetra

All of all classes in Tpetra utilize templates, which allows the user to specify any type they want. In some cases, the choice of data type allows increased functionality. For example, 64-bit ordinals allow for problem sizes to break the 2 billion element barrier present in Epetra, whereas complex scalar types allow the native description and solution of complex-valued problems.

Most of the classes in Tpetra are templated according to the data types which constitute the class. These are the following:

• Scalar: A Scalar is the data structure used for storing values. This is the type most likely to be changed by many users. The most common use cases are float, double, complex<float> and complex<double>. However, any data type can be used so long as it implements Teuchos::ScalarTraits and supports the necessary arithmetic operations, such as addition, subtraction, division and multiplication.
• LocalOrdinal: A LocalOrdinal is used to store indices representing local IDs. The standard use case, as well as the default for most classes, is int. Any type can be used that implements Teuchos::OrdinalTraits and supports the necessary arithmetic. The purpose of template local and global ordinals as two (possibly) different types is for efficiency purposes. For example, the indices of a completed CrsGraph contain objects of type LocalOrdinal. If the specific application allows it, using smaller local ordinals requires less storage and potentially higher rates of computation.
• GlobalOrdinal: A GlobalOrdinal is used to store indices representing global IDs and to describe global properties of a distributed object (e.g., global number of non-zeros in a sparse matrix, global number of rows in a vector.) The GlobalOrdinal therefore dictates the maximum size of a distributed object.

The Tpetra::Distributor class is unique in that it is not templated instantiated on any templated types. However, the class includes some templated member functions. The Tpetra::Distributor::createFromRecvs() method is templated on the ordinal type used to encode IDs, while Tpetra::Distributor::doPosts() and the other post methods are templated on the Packet, the data type being communicated by a particular invocation of the Tpetra::Distributor.

Tpetra Classes

Tpetra contains a number of classes. The primary parallel classes, employed by most users, are:

• Communication classes: Tpetra::Platform and Teuchos::Comm
  
  • Comm provides collective operations among nodes. The abstraction of the communication allows Tpetra objects to be coded to a generic communicator, which may be specified by the user as one of a number of available communicators (e.g., Teuchos::MpiComm, Teuchos::SerialComm). Teuchos::Comm is analogous to Epetra_Comm.
  • Tpetra::Platform contains specific information about the parallel machine we are using. Currently, Tpetra::Platform exists to provide a communicator object. It is anticipated that future versions of Tpetra will utilize Tpetra::Platform to collect other information about the node on which the computations are being executed. Current implementations are Tpetra::SerialPlatform and Tpetra::MpiPlatform. Future work will provide platforms specific to multi-core nodes, such as homogenous multicore CPUs and GPUs.

• Map class: Tpetra::Map
  Contain information used to distribute vectors, matrices and other objects. This class is analogous to the class Epetra_Map.

• Distributed vector class: Tpetra::MultiVector, Tpetra::Vector
  Provides vector services such as scaling, norms, and dot products.

• Distributed sparse matrix classes: Tpetra::RowMatrix, Tpetra::CrsMatrix
  Tpetra::RowMatrix is a abstract interface for row-distributed sparse matrices. Tpetra::CrsMatrix is a specific implementation of Tpetra::RowMatrix, utilizing compressed row storage format. Both of these classes derive from Tpetra::Operator, the base class for linear operators.

• Import/Export classes: Tpetra::Import and Tpetra::Export
  Allows efficient transfer of objects built using one mapping to a new object with a new mapping. Supports local and global permutations, overlapping Schwarz operations and many other data movement operations.

Trilinos and Tpetra

Tpetra can be used mostly as a stand-alone package, with an explicit dependence only on Teuchos. There are adapters allowing the use of Tpetra operators and multivectors in both the Belos linear solver package and the Anasazi eigensolver package.

---