Cost function

A cost function \(L\) is used to measure the error of the prediction with the actual value in machine learning. Minimizing the cost function gives the best fit.

A common loss function is the residual sum of squares:

$$ L = \Sigma^{n}_{i = 1} (y_i - \hat y_i)^2 $$