Зміст курсу
Preprocessing Data
Preprocessing Data
Data Normalization
Data Normalization & Standardization provides rescaling numerical data into the appropriate interval. For example, ML models usually process the values in the interval [0; 1]. It's much more convenient to process the finite data, and also the data that all scaled to the same interval. There are two approaches:
- to normalize data: move it to the interval [0; 1]
- to standartizate data.
Data normalization
After normalization, each value will be represented as the value from the interval [0;1]. This allows us to easily understand how close the value is to the left or right bound. There is a quick demo.
For example, you got the 4th value from column fare:
How can you know, how many it is? How close is it to the mean value? Is it many or not?
You have to know the bounds for the fare price. If you normalized the data, you would get the value:
which is more informative: the value is small enough, only 10% out of the max price.
Normalization formula is:
Let's implement it manually:
x_min, x_max = data['Fare'].min(), data['Fare'].max() normalized_fare = (data['Fare'] - x_min) / (x_max-x_min) print(normalized_fare[3]) # output: 0.10364429745562033
Great! Now we see that 4th value is 0.10364, so this price is only 10% out of the maximum one.
Of course, there is a built-in way that does all the work: let's use the MinMaxScaler() from sklearn.
import pandas as pd from sklearn.preprocessing import MinMaxScaler # reading data # creating a scaler scaler = MinMaxScaler() data_scaled = scaler.fit_transform(data) # now the data is normalized
Завдання
Use the fit_transform() method with Fare data as an argument. Compare if the normalized fare value of 4th record is equal to the received manually one.
Note that your data should be a 2D-array.
Перейдіть на комп'ютер для реальної практикиПродовжуйте з того місця, де ви зупинились, використовуючи один з наведених нижче варіантівДякуємо за ваш відгук!
Data Normalization
Data Normalization & Standardization provides rescaling numerical data into the appropriate interval. For example, ML models usually process the values in the interval [0; 1]. It's much more convenient to process the finite data, and also the data that all scaled to the same interval. There are two approaches:
- to normalize data: move it to the interval [0; 1]
- to standartizate data.
Data normalization
After normalization, each value will be represented as the value from the interval [0;1]. This allows us to easily understand how close the value is to the left or right bound. There is a quick demo.
For example, you got the 4th value from column fare:
How can you know, how many it is? How close is it to the mean value? Is it many or not?
You have to know the bounds for the fare price. If you normalized the data, you would get the value:
which is more informative: the value is small enough, only 10% out of the max price.
Normalization formula is:
Let's implement it manually:
x_min, x_max = data['Fare'].min(), data['Fare'].max() normalized_fare = (data['Fare'] - x_min) / (x_max-x_min) print(normalized_fare[3]) # output: 0.10364429745562033
Great! Now we see that 4th value is 0.10364, so this price is only 10% out of the maximum one.
Of course, there is a built-in way that does all the work: let's use the MinMaxScaler() from sklearn.
import pandas as pd from sklearn.preprocessing import MinMaxScaler # reading data # creating a scaler scaler = MinMaxScaler() data_scaled = scaler.fit_transform(data) # now the data is normalized
Завдання
Use the fit_transform() method with Fare data as an argument. Compare if the normalized fare value of 4th record is equal to the received manually one.
Note that your data should be a 2D-array.
Перейдіть на комп'ютер для реальної практикиПродовжуйте з того місця, де ви зупинились, використовуючи один з наведених нижче варіантівДякуємо за ваш відгук!
Data Normalization
Data Normalization & Standardization provides rescaling numerical data into the appropriate interval. For example, ML models usually process the values in the interval [0; 1]. It's much more convenient to process the finite data, and also the data that all scaled to the same interval. There are two approaches:
- to normalize data: move it to the interval [0; 1]
- to standartizate data.
Data normalization
After normalization, each value will be represented as the value from the interval [0;1]. This allows us to easily understand how close the value is to the left or right bound. There is a quick demo.
For example, you got the 4th value from column fare:
How can you know, how many it is? How close is it to the mean value? Is it many or not?
You have to know the bounds for the fare price. If you normalized the data, you would get the value:
which is more informative: the value is small enough, only 10% out of the max price.
Normalization formula is:
Let's implement it manually:
x_min, x_max = data['Fare'].min(), data['Fare'].max() normalized_fare = (data['Fare'] - x_min) / (x_max-x_min) print(normalized_fare[3]) # output: 0.10364429745562033
Great! Now we see that 4th value is 0.10364, so this price is only 10% out of the maximum one.
Of course, there is a built-in way that does all the work: let's use the MinMaxScaler() from sklearn.
import pandas as pd from sklearn.preprocessing import MinMaxScaler # reading data # creating a scaler scaler = MinMaxScaler() data_scaled = scaler.fit_transform(data) # now the data is normalized
Завдання
Use the fit_transform() method with Fare data as an argument. Compare if the normalized fare value of 4th record is equal to the received manually one.
Note that your data should be a 2D-array.
Перейдіть на комп'ютер для реальної практикиПродовжуйте з того місця, де ви зупинились, використовуючи один з наведених нижче варіантівДякуємо за ваш відгук!
Data Normalization & Standardization provides rescaling numerical data into the appropriate interval. For example, ML models usually process the values in the interval [0; 1]. It's much more convenient to process the finite data, and also the data that all scaled to the same interval. There are two approaches:
- to normalize data: move it to the interval [0; 1]
- to standartizate data.
Data normalization
After normalization, each value will be represented as the value from the interval [0;1]. This allows us to easily understand how close the value is to the left or right bound. There is a quick demo.
For example, you got the 4th value from column fare:
How can you know, how many it is? How close is it to the mean value? Is it many or not?
You have to know the bounds for the fare price. If you normalized the data, you would get the value:
which is more informative: the value is small enough, only 10% out of the max price.
Normalization formula is:
Let's implement it manually:
x_min, x_max = data['Fare'].min(), data['Fare'].max() normalized_fare = (data['Fare'] - x_min) / (x_max-x_min) print(normalized_fare[3]) # output: 0.10364429745562033
Great! Now we see that 4th value is 0.10364, so this price is only 10% out of the maximum one.
Of course, there is a built-in way that does all the work: let's use the MinMaxScaler() from sklearn.
import pandas as pd from sklearn.preprocessing import MinMaxScaler # reading data # creating a scaler scaler = MinMaxScaler() data_scaled = scaler.fit_transform(data) # now the data is normalized
Завдання
Use the fit_transform() method with Fare data as an argument. Compare if the normalized fare value of 4th record is equal to the received manually one.
Note that your data should be a 2D-array.
Перейдіть на комп'ютер для реальної практикиПродовжуйте з того місця, де ви зупинились, використовуючи один з наведених нижче варіантівПерейдіть на комп'ютер для реальної практикиПродовжуйте з того місця, де ви зупинились, використовуючи один з наведених нижче варіантів