Standard linear error function.

Tanh error function, usually better but can require a lower learning rate.

Periodical cosinus activation function.

Fast (sigmoid like) activation function defined by David Elliott

Fast (symmetric sigmoid like) activation function defined by David Elliott

Gaussian activation function.

Symmetric gaussian activation function.

Linear activation function.

Bounded linear activation function.

Bounded Linear activation function.

Sigmoid activation function.

Stepwise linear approximation to sigmoid.

Symmetric sigmoid activation function, aka.

Periodical sinus activation function.

Threshold activation function.

Threshold activation function.

THE SARPROP ALGORITHM: A SIMULATED ANNEALING ENHANCEMENT TO RESILIENT BACK PROPAGATION http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.47.8197&rep=rep1&type=pdf

Each layer only has connections to the next layer

Each layer has connections to all following layers

Stop criteria is number of bits that fail.

Stop criteria is Mean Square Error (MSE) value.

Standard backpropagation algorithm, where the weights are updated after calculating the mean square error for the whole training set.

Standard backpropagation algorithm, where the weights are updated after each training pattern.

A more advanced batch training algorithm which achieves good results for many problems.

A more advanced batch training algorithm which achieves good results for many problems.