Multilabel and Multioutput Algorithms ¶
1. Multilabel Classification¶
- Labeling each sample with m labels from n_classes possible classes
- m can be 0 to n_classes inclusive
- Comparable to running n_classes binary classification tasks, a binary output is assigned to each class, for every sample
1.1 Multioutput Classifier¶
- Some classifiers in sklearn support multilabel, such as KNeighborsClassifier, RandomForestClassifier, do not need use multioutput classifier
- If a classifier do not support multilabel, such as LogisticRegression, need use multioutput classifier
In [8]:
from sklearn.datasets import load_digits
digits = load_digits()
X = digits.data
y = digits.target
In [11]:
from sklearn.model_selection import train_test_split
train_X, test_X, train_Y, test_Y = train_test_split(X, y, test_size=0.2, random_state=42, stratify = y)
In [14]:
import numpy as np
y_train_large = (train_Y >= 7)
y_train_odd = (train_Y % 2 == 1)
y_multilabel = np.c_[y_train_large, y_train_odd] # create multiple labels
In [29]:
# by default, KNeighborsClassifier in sklearn supports multilabel
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(random_state=1)
model.fit(train_X, y_multilabel)
y_pred = model.predict(test_X) # 360*2
In [34]:
from sklearn.multioutput import MultiOutputClassifier
forest = RandomForestClassifier(random_state=1)
model = MultiOutputClassifier(forest, n_jobs=-1)
model.fit(train_X, y_multilabel)
y_pred = model.predict(test_X) # 360*2
In [38]:
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(random_state=0)
model = MultiOutputClassifier(lr, n_jobs=-1)
model.fit(train_X, y_multilabel)
y_pred = model.predict(test_X) # 360*2
1.2 Classifier Chain¶
- Combine a number of binary classifiers into a single multi-label model
- Each model makes a prediction in the order specified by the chain using all of the available features provided to the model plus the predictions of models that are earlier in the chain
In [49]:
from sklearn.multioutput import ClassifierChain
model = RandomForestClassifier(random_state=1)
chain = ClassifierChain(model, order='random', random_state=0)
chain.fit(train_X, y_multilabel)
y_pred = chain.predict(test_X) # 360*2
2. Multioutput Regression¶
- predicts multiple numerical properties for each sample
2.1 Multioutput Regressor¶
In [63]:
from sklearn.datasets import make_regression
X, y = make_regression(n_samples=10, n_targets=3, random_state=1)
In [57]:
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.multioutput import MultiOutputRegressor
# MultiOutputRegressor, fits one regressor per target it can not take advantage of correlations between targets
model = MultiOutputRegressor(GradientBoostingRegressor(random_state=0))
model.fit(X, y)
model.predict(X)
Out[57]:
2.2 Regressor Chain¶
- Combining a number of regressions into a single multi-target model
- Each model makes a prediction in the order specified by the chain using all of the available features provided to the model plus the predictions of models that are earlier in the chain.
In [74]:
from sklearn.multioutput import RegressorChain
from sklearn.linear_model import LogisticRegression
model = GradientBoostingRegressor()
chain = RegressorChain(base_estimator=model)
chain.fit(X, y)
chain.predict(X)
Out[74]:
3. Multiclass-multioutput Classification¶
- Multitask classification
- Labels each sample with a set of non-binary properties
- All classifiers handling multiclass-multioutput support the multilabel classification task as a special case
- Currently no Multiclass-multioutput classification class defined in sklearn like MultiOutputClassifier and MultiOutputRegressor
In [75]:
digits = load_digits()
X = digits.data
y = digits.target
train_X, test_X, train_Y, test_Y = train_test_split(X, y, test_size=0.2, random_state=42, stratify = y)
In [86]:
noise = np.random.randint(0, 3, (len(train_X), 64))
X_train_mod = train_X + noise
noise = np.random.randint(0, 3, (len(test_X), 64))
X_test_mod = test_X + noise
y_train_mod = train_X
y_test_mod = test_X
In [87]:
import matplotlib
import matplotlib.pyplot as plt
plt.imshow(X_test_mod[100].reshape(8, 8), cmap = matplotlib.cm.binary, interpolation='nearest')
Out[87]:
In [95]:
# by default, KNeighborsClassifier in sklearn supports multiclass-multioutput classification
knn_clf = KNeighborsClassifier()
knn_clf.fit(X_train_mod, y_train_mod)
clean_digit = knn_clf.predict([X_test_mod[100]])
In [96]:
plt.imshow(clean_digit.reshape(8, 8), cmap = matplotlib.cm.binary, interpolation='nearest')
Out[96]:
