mlpack::ann::GELUFunction Class Reference

The GELU function, defined by. More...

#include <gelu_function.hpp>

Static Public Member Functions
static double	Fn (const double x)
	Computes the GELU function. More...
template<typename InputVecType , typename OutputVecType >
static void	Fn (const InputVecType &x, OutputVecType &y)
	Computes the GELU function. More...
static double	Deriv (const double y)
	Computes the first derivative of the GELU function. More...
template<typename InputVecType , typename OutputVecType >
static void	Deriv (const InputVecType &y, OutputVecType &x)
	Computes the first derivatives of the GELU function. More...

Detailed Description

The GELU function, defined by.

\begin{eqnarray*} f(x) = 0.5 * x * {1 + tanh[(2/pi)^(1/2) * (x + 0.044715 * x^3)]} \\ f'(x) = 0.5 * tanh(0.0356774 * x^3) + 0.797885 * x) + (0.0535161x^3 + 0.398942 * x) * sech^2(0.0356774 * x^3+0.797885 * x) + 0.5\\ \end{eqnarray*}

Member Function Documentation

Deriv() [1/2]

static double mlpack::ann::GELUFunction::Deriv ( const double  y )

inlinestatic

Computes the first derivative of the GELU function.

Parameters

y	Input data.

Returns

f'(x)

Deriv() [2/2]

template<typename InputVecType , typename OutputVecType >

static void mlpack::ann::GELUFunction::Deriv ( const InputVecType &  y, OutputVecType &  x  )

inlinestatic

Computes the first derivatives of the GELU function.

Parameters

y	Input data.
x	The resulting derivatives.

Fn() [1/2]

static double mlpack::ann::GELUFunction::Fn ( const double  x )

inlinestatic

Computes the GELU function.

Parameters

x	Input data.

Returns

f(x).

Fn() [2/2]

template<typename InputVecType , typename OutputVecType >

static void mlpack::ann::GELUFunction::Fn ( const InputVecType &  x, OutputVecType &  y  )

inlinestatic

Computes the GELU function.

Parameters

x	Input data.
y	The resulting output activation.

---

The documentation for this class was generated from the following file:

• src/mlpack/methods/ann/activation_functions/gelu_function.hpp