Loss function

A loss function is a mathematical function commonly used in statistics. Loss functions are frequently used to create machine learning algorithms.

The loss function computes the error for a single training example in contrast to a Cost function, which is the average of the loss functions from each example within the training data set.

Loss functions are used in machine learning to measure the mathematical distance between predicted values and actual real values.

For example in image classification if an image of the color red is classified as an image of the color red then the loss function is 0; however, if it is classified as an image of blue then the loss function is greater than zero. Any correctly chosen loss function for this example will give a greater value for blue than orange or pink.

The optimization of backpropagation in artificial neural nets is dependent upon loss functions.

There are many types of loss functions including mean absolute loss, mean squared error and mean bias error.