Kohonen Neural Network Library Tutorial


In the all tutorial the authors will put just examples. The library construction is as flexible as possible, so if anyone would like to specialize, or overload templates, classes or methods we hope it is possible. If there is any problem, we are open for discussions.

Building the Kohonen neuron

In the first step define the type of the single data.
typedef std::vector < double > V_d;
The next step is defining the type of the container of the data.
typedef std::vector < V_d > V_v_d;
The Kohonen neuron is built from distance functional, activation function, and weights - what we know from theory. Then we can construct the activation function. Let's assume that we prefer to use Cauchy hat function, which formula is similar to the Cauchy distribution function, but it is more general e.g. there is no assumption that integral over the basis is equal one and so on. We choose Cauchy_function template class. Of course, we have to specialize it very carefully. The first parameter of the template class is type of single value, in more general aspect it is a result from activation function. The second is a type of the scaling factor. The scaling factor is very important, because it changes the shape of the curve of the Cauchy hat function - expand or collapse it. The third type is a type of an exponent used in the function. Setting exponent type as a one of the integral type can increase precision and speed of the calculations, because of proper specialization of the power function for the integral exponents. For the details of the Cauchy_function class look in the documentation.
typedef neural_net::Cauchy_function < V_d::value_type, V_d::value_type, int > C_a_f;
Now we can construct an object of the class Cauchy_function. The first parameter of the constructor is scaling factor, the second one is exponent.
C_a_f c_a_f ( 2.0, 1 );
Independently we can define distance function. For example we use Euclidean distance function (see distances description). We have to put proper parameter for the template class Euclidean_distance_function. The parameter of this template is a type of the data. In our case it is V_d.
typedef distance::Euclidean_distance_function < V_d > E_d_t;
As before we need construct object that will represent distance class.
E_d_t e_d;
We have both: the activation function and the distance function. Then we can combine all to define Kohonen neuron type. The first parameter of the template is a type of the activation function. The second is a type of the distance function. The last one is a type of data.
typedef neural_net::Basic_neuron < C_a_f, E_d_t > Kohonen_neuron;
Of course, we know that one neuron is little useless, so we will configure a network. A parameter of the template class is a type of the elements of the network container, so in our case it is Kohonen_neuron.

Building the network

typedef neural_net::Rectangular_container < Kohonen_neuron > Kohonen_network;
Kohonen_network kohonen_network;
We have to choose one of the randomization policies either
neural_net::Internal_randomise IR;
neural_net::External_randomize ER;
Next we have to generate network, based on designed neuron. The template function is as below. Of course here we explicit set all template parameters, but it is not reasonable in target application. The R is a number of rows in Kohonen matrix, C is number of columns, c_a_f is previously constructed Cauchy hat function, e_d is a distance function, data is a data container, kohonen_network is network, and at the end ER is an external randomize policy - the policy is just for proper using or not std::srand() function.
> ( R, C, c_a_f, e_d, data, kohonen_network, ER );
Or in the short form
neural_net::generate_kohonen_network ( R, C, c_a_f, e_d, data, kohonen_network, ER );
The network is constructed and weights of the neurons are set.

Training the network - WTA

We have network, thus we can continue to prepare training algorithm. At the beginning we have to declare what kind of training function we will use. We take the simplest training algorithm. The first parameter of template is a type of the data, second one is a type of parameters, and the last one is type of internal iteration value. The constructor has two parameters but the most important is the first one, which should be between 0 and 1, and is a weight of influence of the given data on training process bigger value means higher influence.
typedef neural_net::Wta_proportional_training_functional < V_d, double, int > Wta_train_func;
Wta_train_func wta_train_func ( 0.2, 0 );
We construct WTA training algorithm, so the first parameter of the template is the network type, the second one is data type, the third one is iterator through the container of the data, the fourth one is type of training functional, and the fifth one is taken as default at the beginning not important. Parameters of the constructor are: training functional and numeric iterator. The numeric iterator can be very useful for scientific work with training algorithms, because its wide functionality can be more less the same as window function for Fast Fourier Transform, but at this time we will relay on the defaults.
typedef neural_net::Wta_training_algorithm
  V_d, V_v_d::iterator,
Learning_algorithm training_alg ( wta_train_func );
And to finish all construction we can start training process. We assume that one epoch is one pass through all training data.
for ( int i = 0; i < no_epochs; ++i )
  training_alg ( data.begin(), data.end(), &kohonen_network );
  std::random_shuffle ( data.begin(), data.end() );

Building WTM algorithm.

To construct WTM algorithm we have to prepare more part than in WTA algorithm. At the beginning we have to decide what topology will be used in the network. Let assume that it will be hexagonal topology. For some details look in topology description.
typedef neural_net::Hexagonal_topology < size_t > Hex_top;
Hex_top hex_top ( kohonen_network.get_no_rows() );
After that we have to define proper parts of training functional part for weighting network and space. For example using Gaussian hat function:
typedef neural_net::Gauss_function < V_d::value_type, V_d::value_type, int > G_f_space;
G_f_space g_f_space ( 100, 1 );

typedef neural_net::Gauss_function < size_t, V_d::value_type, int > G_f_net;
G_f_net g_f_net ( 10, 1 );
There are also Cauchy hat functions available in library and constant one, too. In the next step we have to define generalized weight.
typedef neural_net::Classic_training_weight
> Classic_weight;

Classic_weight classic_weight ( g_f_net, g_f_space, hex_top, e_d );
Construction of the training functional
typedef neural_net::Wtm_classical_training_functional
> Wtm_c_l_f;

Wtm_c_l_f wtm_c_l_f ( classic_weight, 0.3 );
Construction of algorithm and training
typedef neural_net::Wtm_training_algorithm
> Wtm_c_training_alg;

Wtm_c_training_alg wtm_c_train_alg ( wtm_c_l_f );

for ( int i = 0; i < no_epochs; ++i )
  wtm_c_train_alg ( data.begin(), data.end(), &kohonen_network );

  // decrease sigma parameter in network
  // will make training process more sharpen with each epoch,
  // but it have to be done slowly :-)
      *= 2.0/3.0;

  // shuffle data
  std::random_shuffle ( data.begin(), data.end() );

neural_net::print_network_weights ( std::cout, kohonen_network_2 );

Usage of the network.

The network after training can be used in the same way as other neural networks. The input value can simply put outputs can be read. Function below will return output value of the (i,j)-th neuron.
network ( i, j ) ( input_value );
Or directly:
network.objects[i][j] ( input_value );
There are other useful functions for printing values from network. The first one (from the print_network.hpp file) is:
Other is for printing weights:
All details are available in reference manual.
Valid HTML 4.01 Transitional Valid CSS!