Clustering

Application

• Customer Segmentation
  • cluster customer, can be useful in recommender system
• Data Analysis
  • cluster and analyze each cluster separately
• Dimensionality Reduction Technique
• Anomaly Detection
  • Any instance that has a low affinity to all the clusters is likely to be an anomaly
• Semi-Supervised Learning
  • If only have a few labels, perform clustering and propagate the labels to all the instances in the same cluster
• Search Engines
  • Cluster all images in database, similar images would end up in the same cluster; When a reference image is entered, use trained clustering model to find this image's cluster
• Segment an image

from sklearn.datasets import load_iris
data = load_iris()
X = data.data
y = data.target
data.target_names

array(['setosa', 'versicolor', 'virginica'], dtype='<U10')

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
X = scaler.fit_transform(X)

1. Gaussian Mixture

from sklearn.mixture import GaussianMixture
model = GaussianMixture(n_components=3, random_state=42)
y_pred = model.fit_predict(X)

import numpy as np
mapping = np.array([1, 2, 0])
y_pred = np.array([mapping[cluster_id] for cluster_id in y_pred])

np.sum(y_pred==y) / len(y_pred)

0.03333333333333333

import matplotlib.pyplot as plt
plt.figure(figsize=(9, 3.5))

plt.plot(X[y_pred==0, 2], X[y_pred==0, 3], "yo", label="Cluster 1")
plt.plot(X[y_pred==1, 2], X[y_pred==1, 3], "bs", label="Cluster 2")
plt.plot(X[y_pred==2, 2], X[y_pred==2, 3], "g^", label="Cluster 3")
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper left", fontsize=12)

<matplotlib.legend.Legend at 0x7fe17a1eaeb0>

2. K-Means

Steps

1. Place the centroids randomly
2. Label the instances according to the distance to the centroids
3. Update the centroids
4. Repeat step 2 and 3 until the centroids stop moving

• O(Ikn)
  • k, the number of cluster
  • n, the number of instances
  • I, the number of iteration
  • Complexity can be linear, one of the fastest clustering algorithms
• Scale the input features before running K-Means
• K-Means is guaranteed to converge, may not converge to the right solution, whether it does or not depends on the centroid initialization

Cons

* Prefer clusters of similar sizes
* Does not behave very well when the clusters have varying sizes, different densities, or nonspherical shapes

from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters=3)
y_pred = kmeans.fit_predict(X)

kmeans.labels_, kmeans.cluster_centers_

(array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 2, 2, 2, 1, 2, 2, 2,
        2, 2, 2, 1, 2, 2, 2, 2, 2, 1, 2, 1, 2, 1, 2, 2, 1, 1, 2, 2, 2, 2,
        2, 1, 1, 2, 2, 2, 1, 2, 2, 2, 1, 2, 2, 2, 1, 2, 2, 1], dtype=int32),
 array([[0.19611111, 0.595     , 0.07830508, 0.06083333],
        [0.44125683, 0.30737705, 0.57571548, 0.54918033],
        [0.70726496, 0.4508547 , 0.79704476, 0.82478632]]))

# predict
kmeans.predict(X[10, :].reshape(1, -1))

array([0], dtype=int32)

# distance to each centroid
# nonlinear dimensionality reduction technique
kmeans.transform(X[10, :].reshape(1, -1))

array([[0.15884387, 0.82328741, 1.16116836]])

import matplotlib.pyplot as plt
plt.figure(figsize=(9, 3.5))

plt.plot(X[y_pred==0, 2], X[y_pred==0, 3], "yo", label="Cluster 1")
plt.plot(X[y_pred==1, 2], X[y_pred==1, 3], "bs", label="Cluster 2")
plt.plot(X[y_pred==2, 2], X[y_pred==2, 3], "g^", label="Cluster 3")
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper left", fontsize=12)

<matplotlib.legend.Legend at 0x7fd6eee969d0>

Finding the Optimal Number of Clusters

• silhouette score, the mean silhouette coefficient over all the instances
• silhouette coefficient, $\dfrac{b-a}{max(a, b)}$
  • a, mean distance to the other instances in the same cluster
  • b, mean distance to the instances of the next closest cluster

from sklearn.metrics import silhouette_score
silhouette_score(X, y_pred)

0.5047687565398589

from sklearn.metrics import silhouette_samples
silhouette_coefficients = silhouette_samples(X, y_pred)

Mini-Batch K-Means

• For large datasets that do not fit in memory

from sklearn.cluster import MiniBatchKMeans
minibatch_kmeans = MiniBatchKMeans(n_clusters=3)
n_batches = 3
for X_batch in np.array_split(X, 3):
    minibatch_kmeans.partial_fit(X_batch)

X.tofile('temp.dat', sep='') # save data to a binary file

# manipulate a large array stored in a binary file on disk as if it is entirely in memory
X_mm = np.memmap('temp.dat', dtype='float64', mode='readonly', shape=(150, 4))
batch_size = 3
minibatch_kmeans = MiniBatchKMeans(n_clusters=3, batch_size=batch_size)
minibatch_kmeans.fit(X_mm)

MiniBatchKMeans(batch_size=3, compute_labels=True, init='k-means++',
                init_size=None, max_iter=100, max_no_improvement=10,
                n_clusters=3, n_init=3, random_state=None,
                reassignment_ratio=0.01, tol=0.0, verbose=0)

y_pred = minibatch_kmeans.labels_

plt.figure(figsize=(9, 3.5))

plt.plot(X[y_pred==0, 2], X[y_pred==0, 3], "yo", label="Cluster 1")
plt.plot(X[y_pred==1, 2], X[y_pred==1, 3], "bs", label="Cluster 2")
plt.plot(X[y_pred==2, 2], X[y_pred==2, 3], "g^", label="Cluster 3")
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper left", fontsize=12)

<matplotlib.legend.Legend at 0x7fd6eef7ac90>

Semi-Supervised Learning

• Cluster
• label propagation
• active learning
  • The model is trained with the labeled instances and make predictions on the unlabeled instances
  • The instances for which the model is most uncertain are given to the expert to be labeled
  • iterate this process until the performance improvement stops being worth the labeling effort

3. DBSCAN

• Steps
  • For each instance, count how many instance are located within a small distance $\epsilon$ from it. This regin is called the instances's $\epsilon-neighborhood$
  • If an instance has at least min_samples instance in its $\epsilon-neighborhood$ (including itself), then it is considered a core instance
  • All instances in the neighborhood of a core instance belong to the same cluster. The neighborhood may include other core instances; therefore, a long sequence of neighboring core instances forms a single cluster
  • Any instance that is not a core instance and does not have one in its neighborhood is considered an anomaly
• Prons
  • clusters found by DBSCAN can be any shape
  • Works well if all the clusters are dense enough and if they are well separated by low-density regions
  • robust to outliers
  • Just two hyperparameters
• Cons
  • It can be impssible to capature all the clusters properly if density varies significantly across the clusters
• Complexity
  • CPU, $O(mlogm)$
  • Memory, $O(m^2)$

from sklearn.cluster import DBSCAN
from sklearn.datasets import make_moons

X, y = make_moons(n_samples=1000, noise=0.05)

# choose eps
from sklearn.neighbors import NearestNeighbors
neigh = NearestNeighbors(n_neighbors=2)
nbrs = neigh.fit(X)
distances, indices = nbrs.kneighbors(X)

distances = np.sort(distances, axis=0)
distances = distances[:,1]
plt.plot(distances)

[<matplotlib.lines.Line2D at 0x7fe1b92bfa00>]

# choose eps to be 0.06, 0.05 will create seven clusters
# min_samples depends on domain knowledge
dbscan = DBSCAN(eps=0.05, min_samples=5) 
dbscan.fit(X)

DBSCAN(eps=0.05)

#labels of instances
# -1, considered as anomalies
dbscan.labels_

# indices of the core instances
dbscan.core_sample_indices_.shape
# core instances
dbscan.components_

array([[ 0.92568308,  0.40760832],
       [-1.06801473,  0.23598424],
       [-0.8536932 ,  0.49789903],
       ...,
       [ 0.54786442, -0.41164933],
       [-0.08532256,  0.97242348],
       [ 0.02807615,  1.02441327]])

import matplotlib.pyplot as plt
fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=dbscan.labels_, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=dbscan.labels_)

<matplotlib.collections.PathCollection at 0x7fe1c90e93a0>

#### DBSCAN in sklearn has no predict
#### Users can choose which classification algorithms to use
from sklearn.neighbors import KNeighborsClassifier

knn = KNeighborsClassifier(n_neighbors=50)
knn.fit(dbscan.components_, dbscan.labels_[dbscan.core_sample_indices_])

KNeighborsClassifier(n_neighbors=50)

X_new = np.array([[-0.5, 0], [0, 0.5], [1, -0.1], [2, 1]])
knn.predict(X_new)

array([1, 0, 1, 0])

4. Agglomerative Clustering

• Hierical Clustering (bottom-up)

  • Start with each document being a single cluster
  • Repeatedly joins the two clusters that are most similar
  • Eventually all documents belong to the same cluster.

• Defining closest pair of clusters

  • Ward (default), minimizes the sum of squared differences within all clusters
  • Maximum or Complete-link, distance of the “furthest” points
  • Average-link, average distance between pairs of elements
  • Single-link, distance of the “closest” points

• Pros

  • “rich get richer” behavior leads to uneven cluster sizes
  • single-link algorithms are best for capturing clusters of different sizes and shapes, but it's also sensitive to noise
  • complete link and group average are not affected by noise, but have a bias towards finding global patterns

• Cons

  • only single-link is computationally possible for large datasets, but it doesn't give good results because uses too little information

• Complexity

  • CPU, $O(n^3)$
  • Memory, $O(n^2)$

X, y = make_moons(n_samples=1000, noise=0.05)

from sklearn.cluster import AgglomerativeClustering
clustering = AgglomerativeClustering(linkage='single') # default n_clusters is 2
clustering.fit(X)

AgglomerativeClustering(linkage='single')

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=clustering.labels_, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=clustering.labels_)

<matplotlib.collections.PathCollection at 0x7fe1c89f3a00>

from sklearn.neighbors import kneighbors_graph

# scale nice if provide a m*m sparse distance matrix
# without a connectivity matrix, the algorithm does not scale well
A = kneighbors_graph(X, 30, include_self=False)
clustering = AgglomerativeClustering(linkage='ward', connectivity=A, n_clusters=2)
clustering.fit(X)

AgglomerativeClustering(connectivity=<1000x1000 sparse matrix of type '<class 'numpy.float64'>'
	with 30000 stored elements in Compressed Sparse Row format>)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=clustering.labels_, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=clustering.labels_)

<matplotlib.collections.PathCollection at 0x7fe168e4baf0>

5. BIRCH

• Steps
  • Scan dataset once, build a CF tree in memory
  • Condense the CF tree to a smaller CF tree
  • Global Clustering
  • Clustering Refining (require scan of dataset)
• Parameters
  • threshold, the radius of the subcluster
  • branching_factor, maximum number of CF subclusters in each node
  • n_clusters, number of clusters after the final clustering step
• Prons
  • designed for very large datasets, faster than batch K-Means as long as n is not too large (<20)
• Cons
  • Birch does not scale very well to high dimensional data
  • order dependence, sensitive to skewed input order
  • node may not correspond to natural cluster

from sklearn.cluster import Birch
brc = Birch(n_clusters=2)
brc.fit(X)
labels = brc.predict(X)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=labels, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=labels)

<matplotlib.collections.PathCollection at 0x7fe168f23160>

# mini-batch
brc = Birch(n_clusters=2)
for i in range(10):
    brc.partial_fit(X[0:(i+1)*100, :2])
    
brc.set_params(n_clusters=2)
labels = brc.predict(X)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=labels, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=labels)

<matplotlib.collections.PathCollection at 0x7fd6f09a45d0>

6. Mean-Shift

• Steps

  • Place a circle centered on each instance
  • For each circle, computer the mean of all the instances located within it, shift the circle so that it is centered on the mean
  • Iterates this mean-shifting step until all circles stop moving
  • All the instaces whose circles have settled in the same place are assigned to the same cluster

• Pros

  • aims to discover blobs in a smooth density of samples
  • automatically sets the number of clusters

• Cons

  • prefer to chop clusters into pieces when they have internal density variations
  • not highly scalable, not suited for large datasets

• Complexity

  • CPU, $O(m^2)$

from sklearn.cluster import MeanShift
from sklearn.cluster import estimate_bandwidth

bandwidth = estimate_bandwidth(X, quantile=0.2)

# If bandwidth is not given, the bandwidth is estimated using sklearn.cluster.estimate_bandwidth
clustering = MeanShift(bandwidth=bandwidth)
clustering.fit(X)
labels = clustering.predict(X)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=labels, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=labels)

<matplotlib.collections.PathCollection at 0x7fe1b8f0a370>

Affinity Propagation

• Parameters
  • preference, controls how many exemplars are used
  • damping factor, damps the responsibility and availability messages to avoid numerical oscillations, default damping is 0.5, which means 50% of the calculation for new cluster with each iteration is based on information, and 50% of the calculation is based on current cluster, raise damping, then the info from the other cluster is weighted more heavily, and thus the algorithm will recognize the equality and quickly merge to a single exemplar/cluster
• Pros
  • chooses the number of clusters based on the data provided
• Cons
  • not suited for large datasets
• Complexity
  • CPU, $O(m^2T)$, T is the number of iteration

from sklearn.cluster import AffinityPropagation

clustering = AffinityPropagation(damping = 0.9, preference = -200)
clustering.fit(X)
labels = clustering.predict(X)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=labels, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=labels)

<matplotlib.collections.PathCollection at 0x7fe1b8f0a640>

7. Spectral Clustering

• Steps

  • creates a low-dimensional embedding
  • conduct another clustering algorithm in this low-dimensional space

• Pros

  • Can capture complex cluster structures
  • Can be used to cut graphs
• Cons
  • Does not scale well to large numbers of instances
  • Works well for a small number of clusters, but is not advised for many clusters
  • does not behave well when the clusters have very different sizes

from sklearn.cluster import SpectralClustering
sc = SpectralClustering(n_clusters=2, gamma=20, random_state=42)
sc.fit(X)

SpectralClustering(gamma=20, n_clusters=2, random_state=42)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=sc.labels_, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=sc.labels_)

<matplotlib.collections.PathCollection at 0x7fe1b8afcd00>

8. Ordering Points To Identify the Clustering Structure (OPTICS)

• Steps

  • Keep a sorted seed list, in which each element is a point and its core-distance or reachability-distance
    • core-distance, smallest distance such that o is a core object
    • reachability-distance, distance between core-object o and its points in the $\epsilon$-neighborhood
  • Choose threshold to find the valleys in the plot of ordered points vs distances, the points in each valley contained in a same cluster

• Pros

  • It does not require density parameters
  • Allow for variable density extraction of clusters
  • Better suited for usage on large datasets
• Cons
  • It only produces a cluster ordering
  • It can’t handle high dimensional data
• Complexity
  • CPU, O(nlgn)
  • Memory, O(n)

from sklearn.cluster import OPTICS
clustering = OPTICS(min_samples=30)
clustering.fit(X)

OPTICS(min_samples=30)

fig = plt.figure(figsize=(9, 6))
ax = fig.add_subplot(111)
ax.scatter(X[:, 0], X[:, 1], c=clustering.labels_, marker='o', s=600, cmap="Paired")
ax.scatter(X[:, 0], X[:, 1], marker='*', s=20, c=clustering.labels_)

<matplotlib.collections.PathCollection at 0x7fe188be6580>

Clustering performance evaluation

1. Adjusted Rand index

• measures the similarity of the two assignments
• permute the predicted labels, and get the same score
• adjusted_rand_score is symmetric, swapping the argument does not change the score

• Perfect labeling is scored 1.0, bad have negative or close to 0.0 scores

• Pros

  • Random (uniform) label assignments have a ARI score close to 0.0
  • Bounded range [-1, 1]
  • No assumption is made on the cluster structure
• Cons
  • ARI requires knowledge of the ground truth classes

from sklearn import metrics
labels_true = [0, 0, 0, 1, 1, 1]
labels_pred = [0, 0, 1, 1, 2, 2]

metrics.adjusted_rand_score(labels_true, labels_pred)

0.24242424242424246

2. Mutual Information based scores

• measures the agreement of the two assignments
• permute the predicted labels, and get the same score
• symmetric, swapping the argument does not change the score

• Perfect labeling is scored 1.0, bad have non-positive scores

• Prons

  • Random (uniform) label assignments have a AMI score close to 0.0
  • Upper bound of 1
• Cons
  • MI-based measures require the knowledge of the ground truth classes

labels_true = [0, 0, 0, 1, 1, 1]
labels_pred = [0, 0, 1, 1, 2, 2]

metrics.adjusted_mutual_info_score(labels_true, labels_pred)
metrics.normalized_mutual_info_score(labels_true, labels_pred)
metrics.mutual_info_score(labels_true, labels_pred)

0.4620981203732969

3. Homogeneity, completeness and V-measure

• homogeneity: each cluster contains only members of a single class
• completeness: all members of a given class are assigned to the same cluster

• V-measure: harmonic mean of homogeneity and completeness

  • V-measure is actually equivalent to the mutual information (NMI)

• Prons

  • Bounded scores, 0.0 is as bad as it can be, 1.0 is a perfect score
  • Intuitive interpretation
  • No assumption is made on the cluster structure
• Cons
  • Random labeling won’t yield zero scores especially when the number of clusters is large
  • For smaller sample sizes or larger number of clusters it is safer to use an adjusted index such as the Adjusted Rand Index (ARI)

labels_true = [0, 0, 0, 1, 1, 1]
labels_pred = [0, 0, 1, 1, 2, 2]

metrics.homogeneity_score(labels_true, labels_pred)
metrics.completeness_score(labels_true, labels_pred)
metrics.v_measure_score(labels_true, labels_pred)

0.5158037429793889

4. Fowlkes-Mallows scores

• defined as the geometric mean of the pairwise precision and recall
• The score ranges from 0 to 1. A high value indicates a good similarity between two clusters

• Permute the predicted labels, and get the same score

• Prons

  • Random (uniform) label assignments have a FMI score close to 0.0
  • Upper-bounded at 1
  • No assumption is made on the cluster structure
• Cons
  • Require the knowledge of the ground truth classes

labels_true = [0, 0, 0, 1, 1, 1]
labels_pred = [0, 0, 1, 1, 2, 2]

metrics.fowlkes_mallows_score(labels_true, labels_pred)

0.4714045207910317

5. Silhouette Coefficient

• evaluation is performed using the model itself

• a higher Silhouette Coefficient score relates to a model with better defined clusters

• Prons

  • The score is bounded between -1 for incorrect clustering and +1 for highly dense clustering. Scores around zero indicate overlapping clusters
  • The score is higher when clusters are dense and well separated
• Cons
  • Generally higher for convex clusters than other concepts of clusters

from sklearn import datasets
from sklearn.cluster import KMeans
from sklearn import metrics
X, y = datasets.load_iris(return_X_y=True)

k = 3
kmeans_model = KMeans(n_clusters=k, random_state=1).fit(X)
silhouette_score = metrics.silhouette_score(X, kmeans_model.labels_, metric='euclidean')

from sklearn.metrics import silhouette_samples
y_pred = kmeans_model.labels_
silhouette_coefficients = silhouette_samples(X, y_pred)

import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np

padding = 10
pos = 0
for i in range(k):
    coeffs = silhouette_coefficients[y_pred == i]
    coeffs.sort()
    
    color = mpl.cm.Spectral(i/k)
    plt.fill_betweenx(np.arange(pos, pos+len(coeffs)), 0, coeffs, facecolor = color, edgecolor = color, alpha = 0.7)
    pos += len(coeffs) + padding
plt.axvline(x=silhouette_score, color = 'red', linestyle = "--")

<matplotlib.lines.Line2D at 0x7fe0b96c8610>

6. Calinski-Harabasz Index

• Evaluation is performed using the model itself
• A higher Calinski-Harabasz score relates to a model with better defined clusters

• The index is the ratio of the sum of between-clusters dispersion and of inter-cluster dispersion for all clusters

• Prons

  • The score is higher when clusters are dense and well separated, which relates to a standard concept of a cluster
  • The score is fast to compute
• Cons
  • Generally higher for convex clusters than other concepts of clusters

metrics.calinski_harabasz_score(X, kmeans_model.labels_)

561.62775662962

7. Davies-Bouldin Index

• A lower Davies-Bouldin index relates to a model with better separation between the clusters

• Zero is the lowest possible score. Values closer to zero indicate a better partition

• Prons

  • The computation of Davies-Bouldin is simpler than that of Silhouette score
  • The index is computed only quantities and features inherent to the dataset
• Cons
  • Generally higher for convex clusters than other concepts of clusters
  • The usage of centroid distance limits the distance metric to Euclidean space

metrics.davies_bouldin_score(X, kmeans_model.labels_)

0.6619715465007528

8. Contingency Matrix

• Prons
  • Allows to examine the spread of each true cluster across predicted clusters and vice versa
  • The contingency table calculated is typically utilized in the calculation of a similarity statistic between the two clusterings
• Cons
  • Contingency matrix is easy to interpret for a small number of clusters, but becomes very hard to interpret for a large number of clusters
  • It doesn’t give a single metric to use as an objective for clustering optimisation

from sklearn.metrics.cluster import contingency_matrix
labels_true = [0, 0, 0, 1, 1, 1]
labels_pred = [0, 0, 1, 1, 2, 2]

contingency_matrix(labels_true, labels_pred)

array([[2, 1, 0],
       [0, 1, 2]])

Reference

• BIRCH Clustering Algorithm Example In Python
• DBSCAN Python Example: The Optimal Value For Epsilon (EPS)
• Review on Density Based Clustering Algorithms for Big Data