Course Content

Advanced Techniques in pandas

1. Getting Familiar With Indexing and Selecting Data

Outputting Columns by Title Outputting Rows by Index Selecting Specific Rows and Columns Learning More About Indexation Getting Familiar With lambda Functions Expanding Functionality of the .iloc[] Attribute

2. Dealing With Conditions

Setting Condition Selecting Data Based on Condition Dealing With Several Conditions Making Your Code Beautiful Conditions Quiz

3. Extracting Data

Is Data in ...?Combining Your Knowledge Between Method Extracting Specific Data Finding the Smallest Values of a Column Finding the Largest Values of a Column Finding the Correlation

4. Aggregating Data

Getting Familiar With the .groupby() Method Grouping by Several Columns Complicated Grouping Advanced Grouping Dealing With Pivot Tables Creating a Pivot Table

5. Preprocessing Data

Checking for Missing Values Calculating the Number of Missing Values What Will We Do With the NaN Values?How to Delete Only NaN Values?Filling In the Missing Values Managing Categorical Variables Checking the Column Type Managing an Incorrect Column Renaming the Column

Grouping by Several Columns

Let's add some information on the .groupby() method. You can group by several columns, but the order is crucial in this case. In the previous chapter, we grouped data by the flight number and counted the number of delays. We can make this task complicated by grouping not only by the 'Flight' column, but also by the column 'Airline'. Refresh the information on the dataset and then look at this simple example (the output contains only the first 10 rows):


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/plane', index_col = 0)
data_flights = data[['Flight', 'Delay', 'Airline']].groupby(['Flight', 'Airline']).count()
print(data_flights.head(10))

Explanation:

data[['Flight', 'Delay', 'Airline']] - columns you will work with, including the columns by which you will group;
.groupby(['Flight', 'Airline']) - here, 'Flight' and 'Airline' are arguments of the function .groupby().

Pay attention; if you want to group by several columns, put them into the list - the order is crucial. So, in our case, if rows of the data set have the same value in the column 'Flight', they will relate to one group. Then inside those groups, the function finds other groups for rows with the same value in the column 'Airline'. Then, due to the method .count() that counts the rows, our function will calculate the number of rows in the column 'Delay' that have the same value in the column 'Airline' for each 'Flight' group.

Task

Your task here is to group data by the airport from which the flight started and then by the weekday. Calculate the average time for the groups. Follow the algorithm to manage the task:

Group data:
- Extract the columns 'AirportFrom', 'DayOfWeek', and 'Time' from data (in this order);
- Apply the .groupby() method to the previous columns;
- Within the .groupby() method, put the columns 'AirportFrom' and 'DayOfWeek'; the order is crucial;
- Calculate the mean value of the column 'Time'.
Output the first 10 rows of the data_flights.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 4. Chapter 2

Grouping by Several Columns


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/plane', index_col = 0)
data_flights = data[['Flight', 'Delay', 'Airline']].groupby(['Flight', 'Airline']).count()
print(data_flights.head(10))

Explanation:

data[['Flight', 'Delay', 'Airline']] - columns you will work with, including the columns by which you will group;
.groupby(['Flight', 'Airline']) - here, 'Flight' and 'Airline' are arguments of the function .groupby().

Task

Your task here is to group data by the airport from which the flight started and then by the weekday. Calculate the average time for the groups. Follow the algorithm to manage the task:

Group data:
- Extract the columns 'AirportFrom', 'DayOfWeek', and 'Time' from data (in this order);
- Apply the .groupby() method to the previous columns;
- Within the .groupby() method, put the columns 'AirportFrom' and 'DayOfWeek'; the order is crucial;
- Calculate the mean value of the column 'Time'.
Output the first 10 rows of the data_flights.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 4. Chapter 2

Grouping by Several Columns


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/plane', index_col = 0)
data_flights = data[['Flight', 'Delay', 'Airline']].groupby(['Flight', 'Airline']).count()
print(data_flights.head(10))

Explanation:

data[['Flight', 'Delay', 'Airline']] - columns you will work with, including the columns by which you will group;
.groupby(['Flight', 'Airline']) - here, 'Flight' and 'Airline' are arguments of the function .groupby().

Task

Your task here is to group data by the airport from which the flight started and then by the weekday. Calculate the average time for the groups. Follow the algorithm to manage the task:

Group data:
- Extract the columns 'AirportFrom', 'DayOfWeek', and 'Time' from data (in this order);
- Apply the .groupby() method to the previous columns;
- Within the .groupby() method, put the columns 'AirportFrom' and 'DayOfWeek'; the order is crucial;
- Calculate the mean value of the column 'Time'.
Output the first 10 rows of the data_flights.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!


              1234
            
import pandas as pd
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/4bf24830-59ba-4418-969b-aaf8117d522e/plane', index_col = 0)
data_flights = data[['Flight', 'Delay', 'Airline']].groupby(['Flight', 'Airline']).count()
print(data_flights.head(10))

Explanation:

data[['Flight', 'Delay', 'Airline']] - columns you will work with, including the columns by which you will group;
.groupby(['Flight', 'Airline']) - here, 'Flight' and 'Airline' are arguments of the function .groupby().

Task

Your task here is to group data by the airport from which the flight started and then by the weekday. Calculate the average time for the groups. Follow the algorithm to manage the task:

Group data:
- Extract the columns 'AirportFrom', 'DayOfWeek', and 'Time' from data (in this order);
- Apply the .groupby() method to the previous columns;
- Within the .groupby() method, put the columns 'AirportFrom' and 'DayOfWeek'; the order is crucial;
- Calculate the mean value of the column 'Time'.
Output the first 10 rows of the data_flights.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Section 4. Chapter 2

Switch to desktop for real-world practiceContinue from where you are using one of the options below