Efficient Second Order Minimization (ESM)

This module implements the ESM template tracking algorithm for homography-based image transformations as described by Benhimane & Mails, "Real-time image-based tracking of planes using efficient second-order minimization", 2004. More...

Collaboration diagram for Efficient Second Order Minimization (ESM):

Classes
struct	CVD::ESMResult
	Result class storing some information about the optimization. More...
class	CVD::Homography< PARAMS >
	a generic implementation for 2D homography-based transformations parameterized by a 3x3 matrix with determinant == 1. More...
class	CVD::HomographyPrefix< PARAMS >
	This class provides a generic 2D homography transformation, but also applies a fixed transformation on the input pixel locations. More...
class	CVD::CameraRotation
	a special implementation for 2D homography-based transformations described as a camera rotating around its centre. More...
class	CVD::StaticAppearance
	Basic appearance model implementing no change in the image. More...
class	CVD::OffsetAppearance
	Simple appearance model that assumes a constant offset in the intensities of the image vs. More...
class	CVD::BlurAppearance
	Blur appearance model that assumes that the input image was subject to blur. More...
class	CVD::ESMEstimator< TRANSFORM, APPEARANCE, IMAGE, GRADIENT >
	The main class for the ESM module. More...
class	CVD::RotationEstimator< APPEARANCE, IMAGE, GRADIENT >
	a specialization of ESMEstimator for pure camera rotation only. More...

Functions
template<typename TRANSFORM , typename APPEARANCE , typename IMAGE , typename GRADIENT >
ESMResult	CVD::Internal::esm_opt (TRANSFORM &T, APPEARANCE &A, const BasicImage< IMAGE > &templateImage, const BasicImage< GRADIENT > &templateGradient, const BasicImage< IMAGE > &target, const int max_iterations=40, const double min_delta=1e-8, const double max_RMSE=1.0)
	a full ESM optimization function. More...

Detailed Description

This module implements the ESM template tracking algorithm for homography-based image transformations as described by Benhimane & Mails, "Real-time image-based tracking of planes using efficient second-order minimization", 2004.

Within this module the geometric transformation of the template and the radiometric (appearance) transformation are separated into two different concepts. Concrete type instances are used to parameterize the generic ESMEstimator class. The following are the two important concepts.

The transform object T has to implement the following concept:

class Transform {
    static const int dimensions = X;
    Matrix<3> get_matrix();
    Vector<dimensions> get_jacobian( const Vector<2> & point, const Vector<2> & gradient );
    void update( const Vector<dimensions> & delta );
};

The implementations are Homography, HomographyPrefix and CameraRotation.

A second object deals with the appearance model used and has to implement the following concept:

class Appearance {
    static const int dimensions = X;
    template <typename PIXEL> double difference( const PIXEL & warped, const PIXEL & templatePixel );
    template <typename PIXEL> tuple<double, Vector<dimensions> > difference_jacobian( const PIXEL & warped, const PIXEL & templatePixel );
    Vector<2> image_jacobian( const Vector<2> & gradWarped, const Vector<2> & gradTemplate );
    void update( const Vector<dimensions> & delta );
};

The implementations are StaticAppearance, OffsetAppearance and BlurAppearance.

Function Documentation

esm_opt()

template<typename TRANSFORM , typename APPEARANCE , typename IMAGE , typename GRADIENT >

ESMResult CVD::Internal::esm_opt ( TRANSFORM &  T, APPEARANCE &  A, const BasicImage< IMAGE > &  templateImage, const BasicImage< GRADIENT > &  templateGradient, const BasicImage< IMAGE > &  target, const int  max_iterations = 40, const double  min_delta = 1e-8, const double  max_RMSE = 1.0  )

inline

a full ESM optimization function.

It takes a template image and its gradient, a target image and a general transform object, plus some parameters and iterates until convergence.

Parameters

T	the geometric transformation, used as output as well
A	the appearance transformation, used as output as well
templateImage	the reference template image
templateGradient	and its gradient of the same size
target	the target image, can be any size
max_iterations	maximal number of iterations to perform
min_delta	minimal change vector, if the vector is smaller than this, assume convergence and stop

Returns

struct carrying the convergence data of the optimization