Comparing Histograms

from scipy import stats, special
import numpy as np
import matplotlib.pyplot as plt

sample_1 = stats.uniform.rvs(size=100)
sample_2 = stats.uniform.rvs(size=100)

fig, ax = plt.subplots()
ax.plot(range(100), sample_1)
ax.plot(range(100), sample_2)

[<matplotlib.lines.Line2D at 0x7f7f8840c040>]

Minkowski-form distance

$$D(X, Y) = (\sum_{i}|X_{i} - Y_{i}|^{p})^{1/p}$$

• p = 1, Manhattan distance

• p = 2, Euclidian Distance

• Lower distance, higher similarity

# Euclidian distance
from scipy.spatial import distance
distance.euclidean(sample_1, sample_2)
# or
distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'euclidean')
# or 
distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'minkowski', p=2.)

# Standardized Euclidean distance
distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'seuclidean', V=None)

array([[14.14213562]])

# Manhattan distance
distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'minkowski', p=1.)
# or
distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'cityblock')

array([[34.0490069]])

Cosine distance

$$d = 1- \frac{u\cdot v}{\|u\|\|v\|} = 1 - \frac{\sum_{i=1}^{n}A_{i}B_{i}}{\sqrt{\sum_{i=1}^{n}A_{i}^{2}}\sqrt{\sum_{i=1}^{n}B_{i}^{2}}}$$

• cosine_distance = 1 - cosine_similarity
• measures the cosine of the angle between two vectors projected in a multi-dimensional space
• Lower distance, higher similarity

distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'cosine')

array([[0.2620535]])

Correlation distance

$$d = 1- \frac{(u-\bar{u})(v-\bar{v})}{\|u-\bar{u}\|_{2}\|v-\bar{v}\|_{2}}$$

• correlation_distance = 1 - correlation

distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'correlation')

array([[1.04351762]])

Chebyshev distance

$$d = max_{i}|u_{i} - v_{i}|$$

• maximum norm-1 distance between their respective elements

distance.cdist(sample_1.reshape(1, -1), sample_2.reshape(1, -1), 'chebyshev')

array([[0.97902329]])

Mahalanobis distance

$$\sqrt{(u-v)(C^{-1})(u-v)^{T}}$$

• C is the covariance matrix, VI is $C^{-1}$
• measures distance relative to the centroid

distance.cdist(sample_1, sample_2, 'mahalanobis', VI=None) # each sample need at least 100 records

Kolmogorov-Smirnov distance (K-S)

• D statistic is simply the maximum of D– and D+, smaller D, higher similarity
• Greater p-value, higer similarity

stats.kstest(sample_1, sample_2)

KstestResult(statistic=0.1, pvalue=0.7020569828664881)

Mutual information

• Equal to the Kullback-Leibler divergence
• A measure of the similarity between two labels

from sklearn import metrics
metrics.mutual_info_score(sample_1, sample_2)

/opt/anaconda3/lib/python3.8/site-packages/sklearn/metrics/cluster/_supervised.py:58: UserWarning: Clustering metrics expects discrete values but received continuous values for label, and continuous values for target
  warnings.warn(msg, UserWarning)

4.605170185988092

Earth Mover's distance

• minimum amount of work required to transform one sample into another

from scipy.stats import wasserstein_distance
wasserstein_distance(sample_1, sample_2)

0.02437976189308293

Reference

• Image Similarity and 
the Earth Mover’s Distance
• Distance at Scipy