What is Linear Regression

Basic concepts

Regression is one of the most popular supervised learning tasks.
Its goal is to predict a numerical value (for example, the price of a house) called the target, given a set of parameters (size, age, location, etc.), which are called features.
To train the model, you must provide many examples of such houses, both features and a target. The set of examples you train the model on is called the training set.

The simplest model capable of performing regression tasks is a Linear Regression.
Let's look at the example of a Simple Linear Regression first.
Consider this scatterplot displaying a person's height and his father's height.

What Simple Linear Regression does is just fitting the straight line to the data so that the line is as close to the data points as possible.

Making the predictions

Now we can use this line to predict the target for a new point.
For example, let's say you want to predict the person's height if his father is 63.5 inches tall. Just pick a point from the line that corresponds to X=63.5, and its y value is our prediction, easy peasy.
The model predicts the person to be 64.3 inches tall.

Simple Linear Regression Equation

As you may remember from school, the function of a line is y=ax+b (=b+ax), so during the training, simple linear regression just learns what values should a and b have to form a desired line.
The values that the model learns are called parameters, and further in a course, we will denote parameters using 𝛽 instead of a, b.
So our simple linear regression equation is: