#include <mlpack/core.hpp>#include <mlpack/methods/ann/layer/layer.hpp>#include <mlpack/methods/ann/layer/layer_types.hpp>#include <mlpack/methods/ann/init_rules/random_init.hpp>#include <mlpack/methods/ann/init_rules/glorot_init.hpp>#include <mlpack/methods/ann/init_rules/const_init.hpp>#include <mlpack/methods/ann/init_rules/nguyen_widrow_init.hpp>#include <mlpack/methods/ann/loss_functions/mean_squared_error.hpp>#include <mlpack/methods/ann/loss_functions/binary_cross_entropy_loss.hpp>#include <mlpack/methods/ann/ffn.hpp>#include <mlpack/methods/ann/rnn.hpp>#include "test_catch_tools.hpp"#include "catch.hpp"#include "ann_test_tools.hpp"#include "serialization.hpp"
Include dependency graph for ann_layer_test.cpp:
Functions | |
| template<typename MatType = arma::cube, typename ModelType > | |
| void | CheckRNNCopyFunction (ModelType *network1, MatType &trainData, MatType &trainLabels, const size_t maxEpochs) |
| template<typename MatType = arma::cube, typename ModelType > | |
| void | CheckRNNMoveFunction (ModelType *network1, MatType &trainData, MatType &trainLabels, const size_t maxEpochs) |
| TEST_CASE ("SimpleAddLayerTest", "[ANNLayerTest]") | |
| Simple add module test. | |
| TEST_CASE ("JacobianAddLayerTest", "[ANNLayerTest]") | |
| Jacobian add module test. | |
| TEST_CASE ("GradientAddLayerTest", "[ANNLayerTest]") | |
| Add layer numerical gradient test. | |
| TEST_CASE ("AddLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the outSize parameter of the Add layer works. | |
| TEST_CASE ("SimpleConstantLayerTest", "[ANNLayerTest]") | |
| Simple constant module test. | |
| TEST_CASE ("JacobianConstantLayerTest", "[ANNLayerTest]") | |
| Jacobian constant module test. | |
| TEST_CASE ("ConstantLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the outSize parameter of the Constant layer works. | |
| TEST_CASE ("SimpleDropoutLayerTest", "[ANNLayerTest]") | |
| Simple dropout module test. | |
| TEST_CASE ("DropoutProbabilityTest", "[ANNLayerTest]") | |
| Perform dropout x times using ones as input, sum the number of ones and validate that the layer is producing approximately the correct number of ones. | |
| TEST_CASE ("NoDropoutTest", "[ANNLayerTest]") | |
| TEST_CASE ("SimpleAlphaDropoutLayerTest", "[ANNLayerTest]") | |
| TEST_CASE ("AlphaDropoutProbabilityTest", "[ANNLayerTest]") | |
| Perform AlphaDropout x times using ones as input, sum the number of ones and validate that the layer is producing approximately the correct number of ones. | |
| TEST_CASE ("NoAlphaDropoutTest", "[ANNLayerTest]") | |
| Perform AlphaDropout with probability 1 - p where p = 0, means no AlphaDropout. | |
| TEST_CASE ("SimpleLinearLayerTest", "[ANNLayerTest]") | |
| Simple linear module test. | |
| TEST_CASE ("JacobianLinearLayerTest", "[ANNLayerTest]") | |
| Jacobian linear module test. | |
| TEST_CASE ("GradientLinearLayerTest", "[ANNLayerTest]") | |
| Linear layer numerical gradient test. | |
| TEST_CASE ("SimpleLinear3DLayerTest", "[ANNLayerTest]") | |
| Simple Linear3D layer test. | |
| TEST_CASE ("JacobianLinear3DLayerTest", "[ANNLayerTest]") | |
| Jacobian Linear3D module test. | |
| TEST_CASE ("GradientLinear3DLayerTest", "[ANNLayerTest]") | |
| Simple Gradient test for Linear3D layer. | |
| TEST_CASE ("SimpleNoisyLinearLayerTest", "[ANNLayerTest]") | |
| Simple noisy linear module test. | |
| TEST_CASE ("JacobianNoisyLinearLayerTest", "[ANNLayerTest]") | |
| Jacobian noisy linear module test. | |
| TEST_CASE ("GradientNoisyLinearLayerTest", "[ANNLayerTest]") | |
| Noisy Linear layer numerical gradient test. | |
| TEST_CASE ("SimpleLinearNoBiasLayerTest", "[ANNLayerTest]") | |
| Simple linear no bias module test. | |
| TEST_CASE ("SimplePaddingLayerTest", "[ANNLayerTest]") | |
| Simple padding layer test. | |
| TEST_CASE ("JacobianLinearNoBiasLayerTest", "[ANNLayerTest]") | |
| Jacobian linear no bias module test. | |
| TEST_CASE ("GradientLinearNoBiasLayerTest", "[ANNLayerTest]") | |
| LinearNoBias layer numerical gradient test. | |
| TEST_CASE ("JacobianNegativeLogLikelihoodLayerTest", "[ANNLayerTest]") | |
| Jacobian negative log likelihood module test. | |
| TEST_CASE ("JacobianLeakyReLULayerTest", "[ANNLayerTest]") | |
| Jacobian LeakyReLU module test. | |
| TEST_CASE ("JacobianFlexibleReLULayerTest", "[ANNLayerTest]") | |
| Jacobian FlexibleReLU module test. | |
| TEST_CASE ("GradientFlexibleReLULayerTest", "[ANNLayerTest]") | |
| Flexible ReLU layer numerical gradient test. | |
| TEST_CASE ("JacobianMultiplyConstantLayerTest", "[ANNLayerTest]") | |
| Jacobian MultiplyConstant module test. | |
| TEST_CASE ("CheckCopyMoveMultiplyConstantTest", "[ANNLayerTest]") | |
| Check whether copying and moving network with MultiplyConstant is working or not. | |
| TEST_CASE ("JacobianHardTanHLayerTest", "[ANNLayerTest]") | |
| Jacobian HardTanH module test. | |
| TEST_CASE ("SimpleSelectLayerTest", "[ANNLayerTest]") | |
| Simple select module test. | |
| TEST_CASE ("SelectLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can access the parameters of the Select layer work. | |
| TEST_CASE ("SimpleJoinLayerTest", "[ANNLayerTest]") | |
| Simple join module test. | |
| TEST_CASE ("SimpleAddMergeLayerTest", "[ANNLayerTest]") | |
| Simple add merge module test. | |
| TEST_CASE ("LSTMRrhoTest", "[ANNLayerTest]") | |
| Test the LSTM layer with a user defined rho parameter and without. | |
| TEST_CASE ("GradientLSTMLayerTest", "[ANNLayerTest]") | |
| LSTM layer numerical gradient test. | |
| TEST_CASE ("LSTMLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the LSTM layer work. | |
| TEST_CASE ("FastLSTMRrhoTest", "[ANNLayerTest]") | |
| Test the FastLSTM layer with a user defined rho parameter and without. | |
| TEST_CASE ("GradientFastLSTMLayerTest", "[ANNLayerTest]") | |
| FastLSTM layer numerical gradient test. | |
| TEST_CASE ("FastLSTMLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the Fast LSTM layer work. | |
| TEST_CASE ("CheckCopyMoveFastLSTMTest", "[ANNLayerTest]") | |
| Check whether copying and moving network with FastLSTM is working or not. | |
| TEST_CASE ("CheckCopyMoveLSTMTest", "[ANNLayerTest]") | |
| Check whether copying and moving network with LSTM is working or not. | |
| TEST_CASE ("ReadCellStateParamLSTMLayerTest", "[ANNLayerTest]") | |
| Testing the overloaded Forward() of the LSTM layer, for retrieving the cell state. More... | |
| TEST_CASE ("WriteCellStateParamLSTMLayerTest", "[ANNLayerTest]") | |
| Testing the overloaded Forward() of the LSTM layer, for retrieving the cell state. More... | |
| TEST_CASE ("GRULayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the GRU layer work. | |
| TEST_CASE ("GradientGRULayerTest", "[ANNLayerTest]") | |
| Check if the gradients computed by GRU cell are close enough to the approximation of the gradients. | |
| TEST_CASE ("ForwardGRULayerTest", "[ANNLayerTest]") | |
| GRU layer manual forward test. | |
| TEST_CASE ("SimpleConcatLayerTest", "[ANNLayerTest]") | |
| Simple concat module test. | |
| TEST_CASE ("ConcatAlongAxisTest", "[ANNLayerTest]") | |
| Test to check Concat layer along different axes. | |
| TEST_CASE ("ConcatLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the axis parameter of the Concat layer works. | |
| TEST_CASE ("GradientConcatLayerTest", "[ANNLayerTest]") | |
| Concat layer numerical gradient test. | |
| TEST_CASE ("SimpleConcatenateLayerTest", "[ANNLayerTest]") | |
| Simple concatenate module test. | |
| TEST_CASE ("GradientConcatenateLayerTest", "[ANNLayerTest]") | |
| Concatenate layer numerical gradient test. | |
| TEST_CASE ("SimpleLookupLayerTest", "[ANNLayerTest]") | |
| Simple lookup module test. | |
| TEST_CASE ("GradientLookupLayerTest", "[ANNLayerTest]") | |
| Lookup layer numerical gradient test. | |
| TEST_CASE ("LookupLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can access the parameters of the Lookup layer work. | |
| TEST_CASE ("SimpleLogSoftmaxLayerTest", "[ANNLayerTest]") | |
| Simple LogSoftMax module test. | |
| TEST_CASE ("SimpleSoftmaxLayerTest", "[ANNLayerTest]") | |
| Simple Softmax module test. | |
| TEST_CASE ("GradientSoftmaxTest", "[ANNLayerTest]") | |
| Softmax layer numerical gradient test. | |
| TEST_CASE ("SimpleNearestInterpolationLayerTest", "[ANNLayerTest]") | |
| TEST_CASE ("SimpleBilinearInterpolationLayerTest", "[ANNLayerTest]") | |
| TEST_CASE ("BilinearInterpolationLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the Bilinear Interpolation layer work. | |
| TEST_CASE ("BatchNormTest", "[ANNLayerTest]") | |
| Tests the BatchNorm Layer, compares the layers parameters with the values from another implementation. More... | |
| TEST_CASE ("GradientBatchNormTest", "[ANNLayerTest]") | |
| BatchNorm layer numerical gradient test. | |
| TEST_CASE ("BatchNormLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can access the parameters of the Batch Norm layer work. | |
| TEST_CASE ("GradientVirtualBatchNormTest", "[ANNLayerTest]") | |
| VirtualBatchNorm layer numerical gradient test. | |
| TEST_CASE ("VirtualBatchNormLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the Virtual Batch Norm layer work. | |
| TEST_CASE ("MiniBatchDiscriminationTest", "[ANNLayerTest]") | |
| MiniBatchDiscrimination layer numerical gradient test. | |
| TEST_CASE ("SimpleTransposedConvolutionLayerTest", "[ANNLayerTest]") | |
| Simple Transposed Convolution layer test. | |
| TEST_CASE ("GradientTransposedConvolutionLayerTest", "[ANNLayerTest]") | |
| Transposed Convolution layer numerical gradient test. | |
| TEST_CASE ("SimpleMultiplyMergeLayerTest", "[ANNLayerTest]") | |
| Simple MultiplyMerge module test. | |
| TEST_CASE ("CheckCopyMoveMultiplyMergeTest", "[ANNLayerTest]") | |
| Check whether copying and moving network with MultiplyMerge is working or not. | |
| TEST_CASE ("SimpleAtrousConvolutionLayerTest", "[ANNLayerTest]") | |
| Simple Atrous Convolution layer test. | |
| TEST_CASE ("GradientAtrousConvolutionLayerTest", "[ANNLayerTest]") | |
| Atrous Convolution layer numerical gradient test. | |
| TEST_CASE ("AtrousConvolutionLayerParametersTest", "[ANNLayerTest]") | |
| Test the functions to access and modify the parameters of the AtrousConvolution layer. | |
| TEST_CASE ("AtrousConvolutionLayerPaddingTest", "[ANNLayerTest]") | |
| Test that the padding options are working correctly in Atrous Convolution layer. | |
| TEST_CASE ("LayerNormTest", "[ANNLayerTest]") | |
| Tests the LayerNorm layer. | |
| TEST_CASE ("GradientLayerNormTest", "[ANNLayerTest]") | |
| LayerNorm layer numerical gradient test. | |
| TEST_CASE ("LayerNormLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can access the parameters of the Layer Norm layer work. | |
| TEST_CASE ("AddMergeRunTest", "[ANNLayerTest]") | |
| Test if the AddMerge layer is able to forward the Forward/Backward/Gradient calls. | |
| TEST_CASE ("MultiplyMergeRunTest", "[ANNLayerTest]") | |
| Test if the MultiplyMerge layer is able to forward the Forward/Backward/Gradient calls. | |
| TEST_CASE ("SimpleSubviewLayerTest", "[ANNLayerTest]") | |
| Simple subview module test. | |
| TEST_CASE ("SubviewIndexTest", "[ANNLayerTest]") | |
| Subview index test. | |
| TEST_CASE ("SubviewBatchTest", "[ANNLayerTest]") | |
| Subview batch test. | |
| TEST_CASE ("SubviewLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the Subview layer work. | |
| TEST_CASE ("SimpleReparametrizationLayerTest", "[ANNLayerTest]") | |
| TEST_CASE ("ReparametrizationLayerStochasticTest", "[ANNLayerTest]") | |
| Reparametrization module stochastic boolean test. | |
| TEST_CASE ("ReparametrizationLayerIncludeKlTest", "[ANNLayerTest]") | |
| Reparametrization module includeKl boolean test. | |
| TEST_CASE ("JacobianReparametrizationLayerTest", "[ANNLayerTest]") | |
| Jacobian Reparametrization module test. | |
| TEST_CASE ("GradientReparametrizationLayerTest", "[ANNLayerTest]") | |
| Reparametrization layer numerical gradient test. | |
| TEST_CASE ("GradientReparametrizationLayerBetaTest", "[ANNLayerTest]") | |
| Reparametrization layer beta numerical gradient test. | |
| TEST_CASE ("ReparametrizationLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can access the parameters of the Reparametrization layer work. | |
| TEST_CASE ("SimpleResidualLayerTest", "[ANNLayerTest]") | |
| Simple residual module test. | |
| TEST_CASE ("SimpleHighwayLayerTest", "[ANNLayerTest]") | |
| Simple Highway module test. | |
| TEST_CASE ("HighwayLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the inSize parameter of the Highway layer works. | |
| TEST_CASE ("GradientHighwayLayerTest", "[ANNLayerTest]") | |
| Sequential layer numerical gradient test. | |
| TEST_CASE ("GradientSequentialLayerTest", "[ANNLayerTest]") | |
| Sequential layer numerical gradient test. | |
| TEST_CASE ("GradientWeightNormLayerTest", "[ANNLayerTest]") | |
| WeightNorm layer numerical gradient test. | |
| TEST_CASE ("WeightNormRunTest", "[ANNLayerTest]") | |
| Test if the WeightNorm layer is able to forward the Forward/Backward/Gradient calls. | |
| template<typename LayerType > | |
| void | ANNLayerSerializationTest (LayerType &layer) |
| TEST_CASE ("BatchNormSerializationTest", "[ANNLayerTest]") | |
| Simple serialization test for batch normalization layer. | |
| TEST_CASE ("LayerNormSerializationTest", "[ANNLayerTest]") | |
| Simple serialization test for layer normalization layer. | |
| TEST_CASE ("ConvolutionLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the Convolution layer work. | |
| TEST_CASE ("ConvolutionLayerPaddingTest", "[ANNLayerTest]") | |
| Test that the padding options are working correctly in Convolution layer. | |
| TEST_CASE ("TransposedConvolutionLayerPaddingTest", "[ANNLayerTest]") | |
| Test that the padding options in Transposed Convolution layer. | |
| TEST_CASE ("LpMaxPoolingTestCase", "[ANNLayerTest]") | |
| Simple test for Lp Pooling layer. | |
| TEST_CASE ("MeanPoolingTestCase", "[ANNLayerTest]") | |
| Simple test for Mean Pooling layer. | |
| TEST_CASE ("MaxPoolingTestCase", "[ANNLayerTest]") | |
| Simple test for Max Pooling layer. | |
| TEST_CASE ("GlimpseLayerParametersTest", "[ANNLayerTest]") | |
| Test that the functions that can modify and access the parameters of the Glimpse layer work. | |
| TEST_CASE ("ReinforceNormalLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the stdev parameter of the Reinforce Normal layer works. | |
| TEST_CASE ("VRClassRewardLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the parameters of the VR Class Reward layer works. | |
| TEST_CASE ("AdaptiveMaxPoolingTestCase", "[ANNLayerTest]") | |
| Simple test for Adaptive pooling for Max Pooling layer. | |
| TEST_CASE ("AdaptiveMeanPoolingTestCase", "[ANNLayerTest]") | |
| Simple test for Adaptive pooling for Mean Pooling layer. | |
| TEST_CASE ("TransposedConvolutionalLayerOptionalParameterTest", "[ANNLayerTest]") | |
| TEST_CASE ("BatchNormWithMinBatchesTest", "[ANNLayerTest]") | |
| TEST_CASE ("GradientBatchNormWithMiniBatchesTest", "[ANNLayerTest]") | |
| Batch Normalization layer numerical gradient test. | |
| TEST_CASE ("ConvolutionLayerTestCase", "[ANNLayerTest]") | |
| TEST_CASE ("BatchNormDeterministicTest", "[ANNLayerTest]") | |
| TEST_CASE ("LinearLayerWeightInitializationTest", "[ANNLayerTest]") | |
| Linear module weight initialization test. | |
| TEST_CASE ("AtrousConvolutionLayerWeightInitializationTest", "[ANNLayerTest]") | |
| Atrous Convolution module weight initialization test. | |
| TEST_CASE ("ConvolutionLayerWeightInitializationTest", "[ANNLayerTest]") | |
| Convolution module weight initialization test. | |
| TEST_CASE ("TransposedConvolutionWeightInitializationTest", "[ANNLayerTest]") | |
| Transposed Convolution module weight initialization test. | |
| TEST_CASE ("ChannelShuffleLayerTest", "[ANNLayerTest]") | |
| Simple Test for ChannelShuffle layer. | |
| TEST_CASE ("PixelShuffleLayerTest", "[ANNLayerTest]") | |
| Simple Test for PixelShuffle layer. | |
| TEST_CASE ("PixelShuffleLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the parameters of the PixelShuffle layer works. | |
| TEST_CASE ("SpatialDropoutLayerTest", "[ANNLayerTest]") | |
| TEST_CASE ("SpatialDropoutLayerParametersTest", "[ANNLayerTest]") | |
| Test that the function that can access the parameters of the SpatialDropout layer works. | |
| TEST_CASE ("SimplePositionalEncodingTest", "[ANNLayerTest]") | |
| Simple Positional Encoding layer test. | |
| TEST_CASE ("JacobianPositionalEncodingTest", "[ANNLayerTest]") | |
| Jacobian test for Positional Encoding layer. | |
| TEST_CASE ("SimpleMultiheadAttentionTest", "[ANNLayerTest]") | |
| Simple Multihead Attention test. | |
| TEST_CASE ("JacobianMultiheadAttentionTest", "[ANNLayerTest]") | |
| Jacobian MultiheadAttention module test. | |
| TEST_CASE ("GradientMultiheadAttentionTest", "[ANNLayerTest]") | |
| Numerical gradient test for MultiheadAttention layer. | |
Detailed Description
Tests the ann layer modules.
mlpack is free software; you may redistribute it and/or modify it under the terms of the 3-clause BSD license. You should have received a copy of the 3-clause BSD license along with mlpack. If not, see http://www.opensource.org/licenses/BSD-3-Clause for more information.
Function Documentation
◆ TEST_CASE() [1/3]
| TEST_CASE | ( | "ReadCellStateParamLSTMLayerTest" | , |
| "" | [ANNLayerTest] | ||
| ) |
Testing the overloaded Forward() of the LSTM layer, for retrieving the cell state.
Besides output, the overloaded function provides read access to cell state of the LSTM layer.
◆ TEST_CASE() [2/3]
| TEST_CASE | ( | "WriteCellStateParamLSTMLayerTest" | , |
| "" | [ANNLayerTest] | ||
| ) |
Testing the overloaded Forward() of the LSTM layer, for retrieving the cell state.
Besides output, the overloaded function provides write access to cell state of the LSTM layer.
◆ TEST_CASE() [3/3]
| TEST_CASE | ( | "BatchNormTest" | , |
| "" | [ANNLayerTest] | ||
| ) |
Tests the BatchNorm Layer, compares the layers parameters with the values from another implementation.
Link to the implementation - http://cthorey.github.io./backpropagation/
