Linear Regression with Python

Regression is a fundamental statistical method in machine learning used to model the relationship between a dependent variable (also called the target) and one or more independent variables (also called features or predictors) by fitting an equation to the observed data.

The goal of regression is to find the best-fitting line (linear/ polynomial/ logarithmic equation) that minimizes the difference between the predicted values and the actual values of the dependent variable.

This line is used to make predictions or understand the relationship between the variables.

Why do we need regression?

Predictive Modeling: Regression is fundamental for building predictive models. Data scientists use regression to predict future outcomes based on historical data. For example, predicting sales figures based on advertising spending or predicting housing prices based on various features;
Understanding Relationships: Regression helps in understanding relationships between variables. Data scientists can analyze how changes in one variable influence changes in another, allowing them to uncover meaningful insights and correlations;
Feature Importance: In machine learning and predictive modeling, regression can help identify which features (independent variables) are most important in explaining the variability in the target variable. This assists in feature selection and engineering.

Example

Let's consider a different real-life regression task: predicting a person's annual salary based on their years of experience.


              1234567891011121314151617181920212223242526272829303132
            
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

# Generate synthetic data
np.random.seed(0)
years_experience = np.random.randint(0, 30, 50)  # Random years of experience between 0 and 30
salary = 5000 * years_experience + 30000 + np.random.normal(0, 10000, 50)  # Linear relationship with noise

# Reshape data
years_experience = years_experience.reshape(-1, 1)
salary = salary.reshape(-1, 1)

# Create a linear regression model
model = LinearRegression()

# Train the model on the data
model.fit(years_experience, salary)

# Predict salaries
predicted_salary = model.predict(years_experience)

# Visualize the data and regression line
plt.figure(figsize=(8, 6))
plt.scatter(years_experience, salary, label='Actual Data')
plt.plot(years_experience, predicted_salary, color='red', label='Regression Line')
plt.xlabel('Years of Experience')
plt.ylabel('Annual Salary')
plt.title('Predicting Salary based on Years of Experience')
plt.legend()
plt.savefig('salary_regression_visualization.png', transparent=True)
plt.show()

We can see that red line well determines the wage growth trend.

As a result, we now understand the dependencies between quantities and can make predictions for the future.

Everything was clear?

Thanks for your feedback!

Section 1. Chapter 3