Course Content

Cluster Analysis in Python

1. K-Means Algorithm

Exploratory Data Analysis What is a K-Means Algorithm?Defining the Number of Clusters Case 1: Three Distinct Clusters Case 2: Four Distinct Clusters Clustering Weather Data February vs July Average Temperatures Visualizing the Dynamics Across Clusters

2. K-Medoids Algorithm

What is a K-Medoids Algorithm?Cluster Centers Silhouette Scores K-Medoids and the Weather Data Mean Yearly Temperatures Across Clusters Comparing the Dynamics

3. Hierarchical Clustering

What is a Hierarchical Clustering?Dendrograms Setting Parameters: Linkage Rand Index The Weather Data and Linkages Deciding the Number of Clusters Weather Data: Complete and Ward Linkages How Similar are the Results?

4. Spectral Clustering

What is a Spectral Clustering?Peculiarity of Spectral Clustering Setting Parameters: Affinity Defining the Number of Clusters Is 4 the Optimal Number of Clusters?Comparing the Trends Across Clusters

Comparing the Trends Across Clusters

Both cases look correct, or at least there is nothing wrong. Let's compare the dynamics across months for both cases.

You might remember from the previous sections how the algorithms predicted the dynamics. The spectral clustering with 4 clusters will predict the following dynamics.


              12345678910111213141516171819202122232425262728
            
# Import the librarires
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.cluster import SpectralClustering

# Read the data
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/138ab9ad-aa37-4310-873f-0f62abafb038/Cities+weather.csv', index_col = 0)

# Create the model
model = SpectralClustering(n_clusters = 4, affinity = 'nearest_neighbors')

# Fit the data and predict the labels
data['prediction'] = model.fit_predict(data.iloc[:,2:14])

# Extract the list of the columns
col = list(data.columns[2:14])
col.append('prediction')

# Calculate the monthly mean averages for each cluster
d = data[col].groupby('prediction').mean().stack().reset_index()

# Assign new column names
d.columns = ['Group', 'Month', "Temp"]

# Visualize the results
sns.lineplot(x = 'Month', y = "Temp", hue = 'Group', data = d)
plt.show()

Quite an interesting result! The spectral clustering algorithm catches the 'downwards` up to summer dynamics even in the case of 4 clusters. Let's find out what will be the fifth line produced by this algorithm.

Task

Swipe to show code editor

Import SpectralClustering function from sklearn.cluster.
Create SpectralClustering model named model with 5 clusters and 'nearest_neighbors' affinity.
Fit the 3-14 columns of data to model and predict labels. Save them within 'prediction' column of data.
Calculate the mean for each month within the monthly_data variable:

Group the observation of col columns by the 'prediction' column.
Calculate the mean within each group.
Stack the table.
Reset the indices.

Reassign the column names of newly created DataFrame monthly_data to ['Group', 'Month', 'Temp'].
Build seaborn line plot 'Month' vs 'Temp' for each 'Group' value. Display the plot.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 4. Chapter 6

Comparing the Trends Across Clusters

Both cases look correct, or at least there is nothing wrong. Let's compare the dynamics across months for both cases.

You might remember from the previous sections how the algorithms predicted the dynamics. The spectral clustering with 4 clusters will predict the following dynamics.


              12345678910111213141516171819202122232425262728
            
# Import the librarires
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.cluster import SpectralClustering

# Read the data
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/138ab9ad-aa37-4310-873f-0f62abafb038/Cities+weather.csv', index_col = 0)

# Create the model
model = SpectralClustering(n_clusters = 4, affinity = 'nearest_neighbors')

# Fit the data and predict the labels
data['prediction'] = model.fit_predict(data.iloc[:,2:14])

# Extract the list of the columns
col = list(data.columns[2:14])
col.append('prediction')

# Calculate the monthly mean averages for each cluster
d = data[col].groupby('prediction').mean().stack().reset_index()

# Assign new column names
d.columns = ['Group', 'Month', "Temp"]

# Visualize the results
sns.lineplot(x = 'Month', y = "Temp", hue = 'Group', data = d)
plt.show()

Task

Swipe to show code editor

Import SpectralClustering function from sklearn.cluster.
Create SpectralClustering model named model with 5 clusters and 'nearest_neighbors' affinity.
Fit the 3-14 columns of data to model and predict labels. Save them within 'prediction' column of data.
Calculate the mean for each month within the monthly_data variable:

Group the observation of col columns by the 'prediction' column.
Calculate the mean within each group.
Stack the table.
Reset the indices.

Reassign the column names of newly created DataFrame monthly_data to ['Group', 'Month', 'Temp'].
Build seaborn line plot 'Month' vs 'Temp' for each 'Group' value. Display the plot.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Section 4. Chapter 6

Comparing the Trends Across Clusters

Both cases look correct, or at least there is nothing wrong. Let's compare the dynamics across months for both cases.

You might remember from the previous sections how the algorithms predicted the dynamics. The spectral clustering with 4 clusters will predict the following dynamics.


              12345678910111213141516171819202122232425262728
            
# Import the librarires
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.cluster import SpectralClustering

# Read the data
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/138ab9ad-aa37-4310-873f-0f62abafb038/Cities+weather.csv', index_col = 0)

# Create the model
model = SpectralClustering(n_clusters = 4, affinity = 'nearest_neighbors')

# Fit the data and predict the labels
data['prediction'] = model.fit_predict(data.iloc[:,2:14])

# Extract the list of the columns
col = list(data.columns[2:14])
col.append('prediction')

# Calculate the monthly mean averages for each cluster
d = data[col].groupby('prediction').mean().stack().reset_index()

# Assign new column names
d.columns = ['Group', 'Month', "Temp"]

# Visualize the results
sns.lineplot(x = 'Month', y = "Temp", hue = 'Group', data = d)
plt.show()

Task

Swipe to show code editor

Import SpectralClustering function from sklearn.cluster.
Create SpectralClustering model named model with 5 clusters and 'nearest_neighbors' affinity.
Fit the 3-14 columns of data to model and predict labels. Save them within 'prediction' column of data.
Calculate the mean for each month within the monthly_data variable:

Group the observation of col columns by the 'prediction' column.
Calculate the mean within each group.
Stack the table.
Reset the indices.

Reassign the column names of newly created DataFrame monthly_data to ['Group', 'Month', 'Temp'].
Build seaborn line plot 'Month' vs 'Temp' for each 'Group' value. Display the plot.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Everything was clear?

Thanks for your feedback!

Both cases look correct, or at least there is nothing wrong. Let's compare the dynamics across months for both cases.

You might remember from the previous sections how the algorithms predicted the dynamics. The spectral clustering with 4 clusters will predict the following dynamics.


              12345678910111213141516171819202122232425262728
            
# Import the librarires
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.cluster import SpectralClustering

# Read the data
data = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/138ab9ad-aa37-4310-873f-0f62abafb038/Cities+weather.csv', index_col = 0)

# Create the model
model = SpectralClustering(n_clusters = 4, affinity = 'nearest_neighbors')

# Fit the data and predict the labels
data['prediction'] = model.fit_predict(data.iloc[:,2:14])

# Extract the list of the columns
col = list(data.columns[2:14])
col.append('prediction')

# Calculate the monthly mean averages for each cluster
d = data[col].groupby('prediction').mean().stack().reset_index()

# Assign new column names
d.columns = ['Group', 'Month', "Temp"]

# Visualize the results
sns.lineplot(x = 'Month', y = "Temp", hue = 'Group', data = d)
plt.show()

Task

Swipe to show code editor

Import SpectralClustering function from sklearn.cluster.
Create SpectralClustering model named model with 5 clusters and 'nearest_neighbors' affinity.
Fit the 3-14 columns of data to model and predict labels. Save them within 'prediction' column of data.
Calculate the mean for each month within the monthly_data variable:

Group the observation of col columns by the 'prediction' column.
Calculate the mean within each group.
Stack the table.
Reset the indices.

Reassign the column names of newly created DataFrame monthly_data to ['Group', 'Month', 'Temp'].
Build seaborn line plot 'Month' vs 'Temp' for each 'Group' value. Display the plot.

Switch to desktop for real-world practiceContinue from where you are using one of the options below

Section 4. Chapter 6

Switch to desktop for real-world practiceContinue from where you are using one of the options below