Missing values are common in real-world datasets and must be addressed before clustering. We'll cover three basic methods: mean imputation, median imputation, and row removal.

Filling with Mean

This method replaces missing values in a column with the average of its non-missing values. It is simple and maintains the column average.

# First option
df['column_name'].fillna(df['column_name'].mean(), inplace=True)

# Second option
df['column_name'] = df['column_name'].fillna(df['column_name'].mean())

However, it can reduce variance and may not be suitable for skewed data or categorical features.

Filling with Median

This method replaces missing values with the median of the non-missing values in the column. The median is less sensitive to outliers than the mean, making it better for skewed data or data with outliers.

# First option
df['column_name'].fillna(df['column_name'].median(), inplace=True)

# Second option
df['column_name'] = df['column_name'].fillna(df['column_name'].median())

Removing Rows with Missing Values

This method deletes any rows containing missing values. It is simple and introduces no imputed data. However, it can lead to significant data loss and bias if many rows are removed or missingness is not random.

# First option
df.dropna(inplace=True)

# Second option
df = df.dropna()

Choosing the best method depends on your data and analysis goals. The coding file shows practical examples of each technique in more detail.

The code file below provides practical examples of each preprocessing technique covered in this section, including handling missing values: