Overview

Teaching: 20 min
Exercises: 0 min

Questions

• What is Unsupervised Learning in Machine Learning model

Objectives

• Learn how to use K-mean clustering in ML model

7 Unsupervised Learning

• Used when no feature output data
• Often used for clustering data
• Typical method:

  K-means clustering
  Hierarchical clustering
  Ward clustering
  Partition Around Median (PAM)
  

  7.1 K-means clustering

  7.1.1 Explanation of K-means clustering method:

• Given a set of data, we choose K=2 clusters to be splited:

• First select 2 random centroids (denoted as red and blue X)

• Compute the distance between 2 centroid red X and blue X with all the points (for instance using Euclidean distance) and compare with each other. 2 groups are created with shorter distance to 2 centroids

• Now recompute the new centroids of the 2 groups (using mean value of all points in the same groups):

• Compute the distance between 2 new centroids and all the points. We have 2 new groups:

• Repeat the last 2 steps until no more new centroids created. The model reach equilibrium:

7.1.2 Example with K=3

7.1.3 Implementation

library(ggplot2)
library(factoextra)
library(purrr)
data(iris)
ggplot(iris,aes(x=Sepal.Length,y=Petal.Width))+
      geom_point(aes(color=Species))
set.seed(123)
km <- kmeans(iris[,3:4],3,nstart=20)

table(km$cluster,iris$Species)
fviz_cluster(km,data=iris[,3:4])

7.2 How to find optimal K values:

7.2.1 Elbow approach

• Similar to KNN method for supervised learning, for K-means approach, we are able to use Elbow approach to find the optimal K values.
• The Elbow approach ues the Within-Cluster Sum of Square (WSS) to measure the compactness of the clusters:

The optimal K-values can be found from the Elbow using method=”wss”:

fviz_nbclust(iris[,3:4], kmeans, method = "wss")

7.2.2 Gap-Statistics approach

• Developed by Prof. Tibshirani et al in Stanford
• Applied to any clustering method (K-means, Hierarchical)
• Maximize the Gap function:

E*n: expectation under a sample size of n from the reference distribution

library(cluster)
# B is number of Monte Carlo bootstrap samples
gap_stat <- clusGap(iris[,3:4], FUN = kmeans, nstart=20, K.max = 10, B = 50)
fviz_gap_stat(gap_stat)

Key Points

• K-mean