Prediction Error in Neural Network

1. What is Error or Prediction Error in a Neural Network?

In neural networks, prediction error (often simply called error) refers to the difference between the predicted output and the actual/true output (also called ground truth or label).

Simple Explanation:

Suppose:

  • Our neural network predicts: ŷ = 7.5
  • Actual value (true label): y = 10

Then:

Error = y – ŷ = 10 – 7.5 = 2.5

This error tells us how wrong the prediction is. The training process involves reducing this error using optimization (e.g., gradient descent).

What is Backpropagation?

Backpropagation is the learning algorithm used in neural networks to:

  1. Calculate the error at the output,
  2. Propagate it backward through the network,
  3. Update weights and biases to reduce that error over time.

This is done using gradient descent, which adjusts the weights in the direction that minimally reduces error.

Prediction Error in Neural Network – Prediction Error example with Simple Python