Data Science, Machine Learning and Artificial Intelligence using Python part-3

Overfitting/underfitting

We can see that linear function is not sufficient to fit the training samples. This is called underfitting.

A polynomial of degree 2 approximates the true function almost perfectly.
However, for higher degrees of polynomial, the model will overfit the training data.
We evaluate quantitively overfitting/underfitting by using cross-validation. we calculate the error on the test set and compare it with error on training set to determine overfitting or underfitting.

High Bias versus High Variance

High Bias - Both training and test errors are high and both errors are more or less the same.
High Variance - Training error is low but testing is very high compared to training error.

Data Science, Machine Learning and Artificial Intelligence using Python part-2

The bias-variance tradeoff

In machine learning,bias-variance tradeoff is the problem of simultaneously minimizing two sources of error that prevent supervised learning algorithms from generalizing beyond their training data.

High bias(underfitting) can cause an algorithm to miss the relevant relations between features and target outputs.

High variance(overfitting) can cause an algorithm to model the random noise in the training data, rather than the intended outputs.

Variance

Variance refers to the amount by which "f" would change if we estimated it using different training data sets.

Since the training data are used to fit the statistical learning method, different training data sets will result in a different "f" . But ideally, the estimate for "f" should not vary too much between training sets.

However, if a method has high variance then small changes in the training data can result in large changes in " f " .In general, more flexible statistical methods have higher variance.

Bias 

Bias refers to the error that is introduced by approximating a real-life problem, which may be extremely complicated, by a much simpler model.

For example, linear regression assumes that there is a linear relationship between Y and X1, X2, X3.....XP.It is unlikely that any real-life problem truly has such a simple linear relationship, and So, performing linear regression will undoubtedly result in some bias in the estimate of " f ".

Bias Variance trade-off 

As a general rule, as we use more flexible methods, the variance will increase and the bias will decrease.

The relative rate of change of these two quantities determines whether the test ERROR increases or decreases.

As we increase the flexibility of a class of methods, the bias tends to initially decrease faster than the variance increases. Consequently, the expected test ERROR declines.

However, at some point, increasing flexibility has little impact on the bais but starts to significantly increase the variance. When this happens the test ERROR increases.

Data Science, Machine Learning and Artificial Intelligence using Python part-1

What is Machine Learning?

Ans: Machine Learning is the science of programming computers so they can learn from data.
Machine Learning is a field of computer science that uses statistical techniques to give computer systems the ability to learn(ie. progressively improve performance on a specific task) with data, without being explicitly programmed.

Example:
An E-mail spam filter is an ML program that can learn to flag spam given examples of spam emails and examples of non-spam/ham emails.

"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks T, as measured by P, improves with experience E."

Types of Machine Learning

• Supervised learning
• Unsupervised Learning
• Semi-supervised Learning
• Reinforcement Learning

Supervised Learning: In this learning, the training data you feed to the algorithm includes the labels/outcome(dependent variable to predict). you show the model during the training phase what are the right examples.

The Dependent variable may be :

1)Categorical(Classification problem)

2)Continuous(Regression Problem)

Unsupervised learning: In this learning, the sample data is unlabelled ie. no outcome to predict. An example of this learning is - clustering, dimensionality reduction, etc. here, there is no right example against which the accuracy of the model can be judged. 

Training and Test sample

The total sample is randomly split into training and test samples.

The training sample provides the data from which the predictive model is generated.

The test sample provides us a generalization assessment of the performance of our predictive models to unseen data.

This is just beginning the topic will be continued.......