ML and AI - Regularization Python

What is Regularization in Machine Learning?

Regularization - Regularization in machine learning is a process of introducing additional information in order to prevent overfitting.

• The green and blue functions both incur zero loss on the given data points.
• Regularization will induce a model to prefer the green function, which may generalize better to unseen data.

         

Use of  regularization in classification:

One particular use of regularization is in the field of classification. Empirical learning of classifiers(learning from a finite data set) is always an undermined problem.

because in general, we are trying to infer a function of any X given only some example

x1,x2,x3,x4,x5,x6............xn.

A regularization term(or regularization) R(f) is added to the loss function:

 
where V is an underlying loss function that describes the cost of predicting f(x) when the label is y, such as the square loss or hinge loss and lamda is a parameter that controls the importance of the regularization term.R(f) is typically chosen to impose a penalty.