Зміст курсу

Advanced Techniques in pandas

1. Getting Familiar With Indexing and Selecting Data

Outputting Columns by Title Outputting Rows by Index Selecting Specific Rows and Columns Learning More About Indexation Getting Familiar With lambda Functions Expanding Functionality of the .iloc[] Attribute

2. Dealing With Conditions

Setting Condition Selecting Data Based on Condition Dealing With Several Conditions Making Your Code Beautiful Conditions Quiz

3. Extracting Data

Is Data in ...?Combining Your Knowledge Between Method Extracting Specific Data Finding the Smallest Values of a Column Finding the Largest Values of a Column Finding the Correlation

4. Aggregating Data

Getting Familiar With the .groupby() Method Grouping by Several Columns Complicated Grouping Advanced Grouping Dealing With Pivot Tables Creating a Pivot Table

5. Preprocessing Data

Checking for Missing Values Calculating the Number of Missing Values What Will We Do With the NaN Values?How to Delete Only NaN Values?Filling In the Missing Values Managing Categorical Variables Checking the Column Type Managing an Incorrect Column Renaming the Column

Managing Categorical Variables

Now, you will work with the data set that doesn't contain missing values. The NaN values from the column 'Age' were replaced with the mean of the column, and the NaN value from the 'Fare' column was deleted. So, now it's time to learn how to manage categorical variables. Categorical means that they have some categories. For instance, in the column 'Sex', there is 'male' and 'female'; or in the column 'Embarked', there is 'Q', 'S', and 'C'.

What should we do to calculate the number of values in each category or to find out information on them?

You already know .loc[], .isin(), .between() and a lot of functions, but in pandas, there is a more beautiful and convenient way to do this. Use the function .get_dummies(). As an example, we will apply it to the column 'Embarked'. Look at the implementation and the result (we will output 5 random passengers' names and new columns that we created).


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/titanic3.csv', index_col = 0)
data = pd.get_dummies(data, columns = ['Embarked'])
print(data[['Name', 'Embarked_C', 'Embarked_Q', 'Embarked_S']].sample(5))

Let's examine one of the possible outputs, specifically one of the possible combinations of five randomly selected rows. You can scroll horizontally through the table to view all the columns:

Explanation:

As a result, our function split the column 'Embarked' into three columns: 'Embarked_C', and 'Embarked_Q', 'Embarked_S'. In total, we have three categories. Each passenger has their category in the 'Embarked' column. Thus, our function creates three columns corresponding to each category, and in line with each passenger, it fills the row of the column with 1 if the person was initially related to the geography; otherwise, it says 0. Thus, we get 1 in just one column.

pd.get_dummies() - this function converts categorical variables into dummy ones (1 or 0);
data - the data frame that you want to use;
columns = ['Embarked'] - columns have categorical variables that you want to transform into dummy ones. Pay attention; it is obligatory to put column names into the list.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Перейдіть на комп'ютер для реальної практикиПродовжуйте з того місця, де ви зупинились, використовуючи один з наведених нижче варіантів

Все було зрозуміло?

Секція 5. Розділ 6

Managing Categorical Variables

What should we do to calculate the number of values in each category or to find out information on them?


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/titanic3.csv', index_col = 0)
data = pd.get_dummies(data, columns = ['Embarked'])
print(data[['Name', 'Embarked_C', 'Embarked_Q', 'Embarked_S']].sample(5))

Let's examine one of the possible outputs, specifically one of the possible combinations of five randomly selected rows. You can scroll horizontally through the table to view all the columns:

Explanation:

pd.get_dummies() - this function converts categorical variables into dummy ones (1 or 0);
data - the data frame that you want to use;
columns = ['Embarked'] - columns have categorical variables that you want to transform into dummy ones. Pay attention; it is obligatory to put column names into the list.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Все було зрозуміло?

Секція 5. Розділ 6

Managing Categorical Variables

What should we do to calculate the number of values in each category or to find out information on them?


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/titanic3.csv', index_col = 0)
data = pd.get_dummies(data, columns = ['Embarked'])
print(data[['Name', 'Embarked_C', 'Embarked_Q', 'Embarked_S']].sample(5))

Let's examine one of the possible outputs, specifically one of the possible combinations of five randomly selected rows. You can scroll horizontally through the table to view all the columns:

Explanation:

pd.get_dummies() - this function converts categorical variables into dummy ones (1 or 0);
data - the data frame that you want to use;
columns = ['Embarked'] - columns have categorical variables that you want to transform into dummy ones. Pay attention; it is obligatory to put column names into the list.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Все було зрозуміло?

What should we do to calculate the number of values in each category or to find out information on them?


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/titanic3.csv', index_col = 0)
data = pd.get_dummies(data, columns = ['Embarked'])
print(data[['Name', 'Embarked_C', 'Embarked_Q', 'Embarked_S']].sample(5))

Let's examine one of the possible outputs, specifically one of the possible combinations of five randomly selected rows. You can scroll horizontally through the table to view all the columns:

Explanation:

pd.get_dummies() - this function converts categorical variables into dummy ones (1 or 0);
data - the data frame that you want to use;
columns = ['Embarked'] - columns have categorical variables that you want to transform into dummy ones. Pay attention; it is obligatory to put column names into the list.

Завдання

Your task here is to transform the column 'Sex' into one with dummy variables instead of categorical ones. Then output the sum of the values in each category.

Секція 5. Розділ 6