ConstrainedOptPack: C++ Tools for Constrained (and Unconstrained) Optimization Version of the Day
Classes | Public Member Functions
ConstrainedOptPack::MatrixSymPosDefLBFGS Class Reference

Implementation of limited Memory BFGS matrix for arbitrary vector spaces. More...

#include <ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp>

Inheritance diagram for ConstrainedOptPack::MatrixSymPosDefLBFGS:
Inheritance graph
[legend]

List of all members.

Classes

class  PostMod
 PostMod class to use with MemMngPack::AbstractFactorStd. More...

Public Member Functions

 MatrixSymPosDefLBFGS (size_type max_size=0, size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
 Calls initial_setup(,,,)
 STANDARD_MEMBER_COMPOSITION_MEMBERS (bool, auto_rescaling)
 Set whether automatic rescaling is used or not.
void initial_setup (size_type max_size=0, size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
 Initial setup for the matrix.
size_type m () const
 
size_type m_bar () const
 
size_type k_bar () const
 
value_type gamma_k () const
 
const DMatrixSlice S () const
 
const DMatrixSlice Y () const
 
bool maintain_original () const
 
bool maintain_inverse () const
 
size_type num_secant_updates () const
 Returns the total number of successful secant updates performed.
size_type rows () const
 

Public types

typedef Teuchos::RCP< const
MultiVector
multi_vec_ptr_t
 

Constructors and initializers

 MatrixSymPosDefLBFGS (size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
 Calls this->initial_setup()
 STANDARD_MEMBER_COMPOSITION_MEMBERS (bool, auto_rescaling)
 Set whether automatic rescaling is used or not.
void initial_setup (size_type m=10, bool maintain_original=true, bool maintain_inverse=true, bool auto_rescaling=false)
 Initial setup for the matrix.
size_type m () const
 
size_type m_bar () const
 
value_type gamma_k () const
 
const multi_vec_ptr_t S () const
 
const multi_vec_ptr_t Y () const
 
bool maintain_original () const
 
bool maintain_inverse () const
 
size_type num_secant_updates () const
 Returns the total number of successful secant updates performed since this->init_identity() was called.

Overridden from MatrixOp

const VectorSpace & space_cols () const
 
std::ostream & output (std::ostream &out) const
 
MatrixOp & operator= (const MatrixOp &mwo)
 
void Vp_StMtV (VectorMutable *v_lhs, value_type alpha, BLAS_Cpp::Transp trans_rhs1, const Vector &v_rhs2, value_type beta) const
 
std::ostream & output (std::ostream &out) const
 
MatrixOp & operator= (const MatrixOp &m)
 
void Vp_StMtV (DVectorSlice *vs_lhs, value_type alpha, BLAS_Cpp::Transp trans_rhs1, const DVectorSlice &vs_rhs2, value_type beta) const
 

Overridden from MatrixOpNonsing

void V_InvMtV (VectorMutable *v_lhs, BLAS_Cpp::Transp trans_rhs1, const Vector &v_rhs2) const
 

Overridden from MatrixSymSecant

void init_identity (const VectorSpace &space_diag, value_type alpha)
 
void init_diagonal (const Vector &diag)
 Actually this calls init_identity( diag.space(), diag.norm_inf() ).
void secant_update (VectorMutable *s, VectorMutable *y, VectorMutable *Bs)
 
void init_identity (size_type n, value_type alpha)
 
void init_diagonal (const DVectorSlice &diag)
 Actually this calls init_identity( (&diag)->size(), norm_inf(diag) ).
void secant_update (DVectorSlice *s, DVectorSlice *y, DVectorSlice *Bs)
 

Overridden from MatrixWithOpFactorized

void V_InvMtV (DVectorSlice *v_lhs, BLAS_Cpp::Transp trans_rhs1, const DVectorSlice &vs_rhs2) const
 

Overridden from MatrixSymAddDelUpdateble

void initialize (value_type alpha, size_type max_size)
 This is fine as long as alpha > 0.0.
void initialize (const DMatrixSliceSym &A, size_type max_size, bool force_factorization, Inertia inertia, PivotTolerances pivot_tols)
 Sorry, this will throw an exception!
size_type max_size () const
 
Inertia inertia () const
 Returns (0,0,rows())
void set_uninitialized ()
 Will set rows() == 0.
void augment_update (const DVectorSlice *t, value_type alpha, bool force_refactorization, EEigenValType add_eigen_val, PivotTolerances pivot_tols)
 Augment the matrix to add a row and column.
void delete_update (size_type jd, bool force_refactorization, EEigenValType drop_eigen_val, PivotTolerances pivot_tols)
 Should always succeed unless user gives a wrong value for drop_eigen_val.

Detailed Description

Implementation of limited Memory BFGS matrix for arbitrary vector spaces.

Implementation of limited Memory BFGS matrix.

The function set_num_updates_stored() must be called first to set the maximum number of the most recent updates that can be stored. The storage requirements for this class are O( n*m + m*m ) which is O(n*m) when n >> m which is expected (where n is the dimension of the vector space and m is the maximum number of updates stored).

This implementation is based on:

Byrd, Nocedal, and Schnabel, "Representations of quasi-Newton matrices and their use in limited memory methods", Mathematical Programming, 63 (1994)

Consider BFGS updates of the form:


 ( B^{k-1}, s^{k-1}, y^{k-1} ) -> B^{k}
 
 where:
 
 B^{k} = B^{k-1} - ( (B*s)*(B*s)' / (s'*B*s) )^{k-1} + ( (y*y') / (s'*y) )^{k-1}
 
 B <: R^(n x n)
 s <: R^(n)
 y <: R^(n)
 

Now let us consider limited memory updating. For this implementation we set:


 Bo = ( 1 / gamma_k ) * I
 
 where:
               / (s^{k-1}'*y^{k-1})/(y^{k-1}'*y^{k-1})           :  if auto_rescaling() == true
     gamma_k = |
             \  1/alpha from last call to init_identity(n,alpha) :  otherwise
 
 

Now let us define the matrices S and Y that store the update vectors s^{i} and y^{i} for i = 1 ... m_bar:


 S = [ s^{1}, s^{2},...,s^{m_bar} ] <: R^(n x m)
 Y = [ y^{1}, y^{2},...,y^{m_bar} ] <: R^(n x m)
  

Here we are only storing the m_bar <= m most recent update vectors and their ordering is significant. The columns S(:,m_bar) and Y(:,m_bar) contain the most recent update vectors. This is all client needs to know in order to reconstruct the updates themselves.

This class allows matrix-vector products x = B*y and the inverse matrix-vector products x = inv(B)*y to be performed at a cost of about O(n*m_bar^2).

The function set_num_updates_stored(l) must be called first to set the maximum number of the most recent updates that can be stored. The storage requirements for this class are O( n_max*l + l*l ) which is O(n_max*l) when n_max >> l which is expected.

This implementation is based on:

Byrd, Nocedal, and Schnabel, "Representations of quasi-Newton matrices and their use in limited memory methods", Mathematical Programming, 63 (1994)

Consider BFGS updates of the form: {verbatim}

( B^{k-1}, s^{k-1}, y^{k-1} ) -> B^{k}

where:

B^{k} = B^{k-1} - ( (B*s)*(B*s)' / (s'*B*s) )^{k-1} + ( (y*y') / (s'*y) )^{k-1}

B <: R^(n x n) s <: R^(n) y <: R^(n)

{verbatim} Given that we start from the same initial matrix #Bo#, the updated matrix #B^{k}# will be the same independent of the order the #(s^{i},y^{i})# updates are added.

Now let us consider limited memory updating. For this implementation we set: {verbatim}

Bo = ( 1 / gamma_k ) * I

where: / (s^{k-1}'*y^{k-1})/(y^{k-1}'*y^{k-1}) : if auto_rescaling() == true gamma_k = | \ alpha from last call to init_identity(n,alpha) : otherwise

{verbatim} Now let us define the matrices S# and Y# that store the update vectors #s^{i}# and #y^{i}# for #i = 1 ... m_bar#: {verbatim}

S = [ s^{1}, s^{2},...,s^{m_bar} ] <: R^(n x m) Y = [ y^{1}, y^{2},...,y^{m_bar} ] <: R^(n x m)

{verbatim} Here we are only storing the m_bar <= m# most recent update vectors and their ordering is arbitrary. The columns #S(:,k_bar)# and #Y(:,k_bar)# contain the most recent update vectors. The next most recent vectors are to the left (i.e. #p = k_bar-1#) and so forth until #p = 1#. Then the next most recent update vectors start at m_bar# and move to the left until you reach the oldest update vector stored at column k_bar+1#. This is all client need to know in order to reconstruct the updates themselves.

This class allows matrix vector products x = B*y and the inverse matrix vector products x = inv(B)*y to be performed at a cost of about O(n*m^2).

In addition, the class supports the MatixSymAddDelUpdateable interface with a few major restrictions. This allows the client to add and remove rows and columns from the matrix.

Definition at line 110 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.


Member Typedef Documentation

Definition at line 120 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.


Constructor & Destructor Documentation

ConstrainedOptPack::MatrixSymPosDefLBFGS::MatrixSymPosDefLBFGS ( size_type  m = 10,
bool  maintain_original = true,
bool  maintain_inverse = true,
bool  auto_rescaling = false 
)

Calls this->initial_setup()

Definition at line 161 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

ConstrainedOptPack::MatrixSymPosDefLBFGS::MatrixSymPosDefLBFGS ( size_type  max_size = 0,
size_type  m = 10,
bool  maintain_original = true,
bool  maintain_inverse = true,
bool  auto_rescaling = false 
)

Calls initial_setup(,,,)


Member Function Documentation

ConstrainedOptPack::MatrixSymPosDefLBFGS::STANDARD_MEMBER_COMPOSITION_MEMBERS ( bool  ,
auto_rescaling   
)

Set whether automatic rescaling is used or not.

This function must be called before a BFGS update is performed in order for it to take effect for that update.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initial_setup ( size_type  m = 10,
bool  maintain_original = true,
bool  maintain_inverse = true,
bool  auto_rescaling = false 
)

Initial setup for the matrix.

This function must be called before init_identity(n) is called in order for these setting to have affect. When this function is called all current updates are lost and the matrix becomes uninitialized.

Parameters:
m[in] Max number of recent update vectors s and y stored.
maintain_original[in] If true then quantities needed to compute x = Bk*y will be maintained, otherwise they will not be unless needed. This is to save computational costs in case matrix-vector products will never be needed. However, if a matrix vector product is needed then these quantities will be computed on the fly in order to satisfy the request.
maintain_inverse[in] If true then quantities needed to compute x = inv(Bk)*y = x = Hk*y will be maintained, otherwise they will not be unless needed. This is to save computational costs in case inverse matrix-vector products will never be needed. However, if the inverse product is ever needed then the needed quantities will be computed on the fly in order to satisfiy the request. Because it takes so little extra work to maintain the quantities needed for Hk it is recommended to always set this to true.
auto_rescaling[in] See intro.

Definition at line 171 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m ( ) const [inline]

Definition at line 360 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m_bar ( ) const [inline]

Definition at line 366 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

value_type ConstrainedOptPack::MatrixSymPosDefLBFGS::gamma_k ( ) const [inline]

Definition at line 372 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::S ( ) const [inline]

Definition at line 379 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::Y ( ) const [inline]

Definition at line 386 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_original ( ) const [inline]

Definition at line 392 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_inverse ( ) const [inline]

Definition at line 398 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::num_secant_updates ( ) const [inline]

Returns the total number of successful secant updates performed since this->init_identity() was called.

Definition at line 404 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.hpp.

const VectorSpace & ConstrainedOptPack::MatrixSymPosDefLBFGS::space_cols ( ) const [virtual]
std::ostream & ConstrainedOptPack::MatrixSymPosDefLBFGS::output ( std::ostream &  out) const [virtual]

Reimplemented from AbstractLinAlgPack::MatrixOp.

Definition at line 203 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

MatrixOp & ConstrainedOptPack::MatrixSymPosDefLBFGS::operator= ( const MatrixOp &  mwo)

Definition at line 227 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::Vp_StMtV ( VectorMutable v_lhs,
value_type  alpha,
BLAS_Cpp::Transp  trans_rhs1,
const Vector v_rhs2,
value_type  beta 
) const [virtual]
void ConstrainedOptPack::MatrixSymPosDefLBFGS::V_InvMtV ( VectorMutable v_lhs,
BLAS_Cpp::Transp  trans_rhs1,
const Vector v_rhs2 
) const [virtual]
void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_identity ( const VectorSpace &  space_diag,
value_type  alpha 
)

Definition at line 477 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_diagonal ( const Vector diag)

Actually this calls init_identity( diag.space(), diag.norm_inf() ).

This initialization is not convienent for this implementation. Besides, when we are using automatric rescaling (auto_rescaling == true) then this will really not matter much anyway.

Definition at line 509 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::secant_update ( VectorMutable s,
VectorMutable y,
VectorMutable Bs 
)

Definition at line 514 of file ConstrainedOptPack_MatrixSymPosDefLBFGS.cpp.

ConstrainedOptPack::MatrixSymPosDefLBFGS::STANDARD_MEMBER_COMPOSITION_MEMBERS ( bool  ,
auto_rescaling   
)

Set whether automatic rescaling is used or not.

This function must be called before a BFGS update is performed in order for it to take effect for that update.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initial_setup ( size_type  max_size = 0,
size_type  m = 10,
bool  maintain_original = true,
bool  maintain_inverse = true,
bool  auto_rescaling = false 
)

Initial setup for the matrix.

This function must be called before init_identity(n) is called. When this function is called all current updates are lost and the matrix becomes uninitialized.

Parameters:
max_size[in] If max_size > 0 then this is the max size the matrix is allowed to become. If max_size == 0 then this size will be determined by one of the initialization methods.
m[in] Max number of recent update vectors stored
maintain_original[in] If true then quantities needed to compute x = Bk*y will be maintained, otherwise they will not be unless needed. This is to save computational costs in case matrix vector products will never be needed. However, if a matrix vector product is needed then these quantities will be computed on the fly in order to satisfy the request.
maintain_inverse[in] If true then quantities needed to compute x = inv(Bk)*y = x = Hk*y will be maintained , otherwise they will not be unless needed. This is to save computational costs in case inverse matrix vector products will never be needed. However, if the inverse product is ever needed then the needed quantities will be computed on the fly in order to satisfiy the request. Because it takes so little extra work to maintain the quantities needed for Hk it is recommended to always set this to true.
auto_rescaling[in] See intro.
size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m ( ) const

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::m_bar ( ) const

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::k_bar ( ) const [inline]
value_type ConstrainedOptPack::MatrixSymPosDefLBFGS::gamma_k ( ) const

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::S ( ) const

const DMatrixSlice ConstrainedOptPack::MatrixSymPosDefLBFGS::Y ( ) const

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_original ( ) const

bool ConstrainedOptPack::MatrixSymPosDefLBFGS::maintain_inverse ( ) const

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::num_secant_updates ( ) const

Returns the total number of successful secant updates performed.

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::rows ( ) const [virtual]

Reimplemented from AbstractLinAlgPack::MatrixBase.

std::ostream& ConstrainedOptPack::MatrixSymPosDefLBFGS::output ( std::ostream &  out) const [virtual]

Reimplemented from AbstractLinAlgPack::MatrixOp.

MatrixOp& ConstrainedOptPack::MatrixSymPosDefLBFGS::operator= ( const MatrixOp &  m)

void ConstrainedOptPack::MatrixSymPosDefLBFGS::Vp_StMtV ( DVectorSlice *  vs_lhs,
value_type  alpha,
BLAS_Cpp::Transp  trans_rhs1,
const DVectorSlice &  vs_rhs2,
value_type  beta 
) const

void ConstrainedOptPack::MatrixSymPosDefLBFGS::V_InvMtV ( DVectorSlice *  v_lhs,
BLAS_Cpp::Transp  trans_rhs1,
const DVectorSlice &  vs_rhs2 
) const

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_identity ( size_type  n,
value_type  alpha 
)

void ConstrainedOptPack::MatrixSymPosDefLBFGS::init_diagonal ( const DVectorSlice &  diag)

Actually this calls init_identity( (&diag)->size(), norm_inf(diag) ).

This initialization is not convienent for this implementation. Besides, when we are using automatric rescaling (auto_rescaling == true) then this will really not matter much anyway.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::secant_update ( DVectorSlice *  s,
DVectorSlice *  y,
DVectorSlice *  Bs 
)

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initialize ( value_type  alpha,
size_type  max_size 
)

This is fine as long as alpha > 0.0.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::initialize ( const DMatrixSliceSym A,
size_type  max_size,
bool  force_factorization,
Inertia  inertia,
PivotTolerances  pivot_tols 
)

Sorry, this will throw an exception!

size_type ConstrainedOptPack::MatrixSymPosDefLBFGS::max_size ( ) const

Inertia ConstrainedOptPack::MatrixSymPosDefLBFGS::inertia ( ) const

Returns (0,0,rows())

void ConstrainedOptPack::MatrixSymPosDefLBFGS::set_uninitialized ( )

Will set rows() == 0.

void ConstrainedOptPack::MatrixSymPosDefLBFGS::augment_update ( const DVectorSlice *  t,
value_type  alpha,
bool  force_refactorization,
EEigenValType  add_eigen_val,
PivotTolerances  pivot_tols 
)

Augment the matrix to add a row and column.

This function is very limited in what it will do. It will throw exceptions if alpha <= 0.0 or t != NULL or add_eigen_val == EIGEN_VAL_NEG or this->rows() == this->max_size(). The obvious postconditions for this function will only technically be satisfied if alpha == this->gamma_k().

void ConstrainedOptPack::MatrixSymPosDefLBFGS::delete_update ( size_type  jd,
bool  force_refactorization,
EEigenValType  drop_eigen_val,
PivotTolerances  pivot_tols 
)

Should always succeed unless user gives a wrong value for drop_eigen_val.


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends