Elastic Net

$$J(\theta) = MSE(\theta) + r\alpha \Sigma_{i=1}^{n}|\theta_{i}| +\dfrac{1-r}{2}\alpha\Sigma_{i=1}^{n} \theta_{i}^{2}$$
  • r = 0, is equivalent to Ridge Regression
  • r = 1, is equivalent to Lasso Regression
  • Ridge is default
  • Use Lasso or Elastic Net if you suspect that only a few features are useful

Create Random Dataset

In [1]:
import numpy as np

X = 2*np.random.rand(100, 1)
Y = 4 + 3 * X + np.random.randn(100, 1)

Train Model with Ridge Regression

In [19]:
from sklearn.linear_model import ElasticNet

model = ElasticNet(alpha = 0.1, l1_ratio=0.5)

model.fit(X, Y)
Out[19]:
ElasticNet(alpha=0.1, copy_X=True, fit_intercept=True, l1_ratio=0.5,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)
In [20]:
model.intercept_, model.coef_
Out[20]:
(array([4.22975567]), array([2.65643358]))
In [21]:
X_new = np.linspace(0, 2, 100).reshape(-1, 1)
Y_predict = model.predict(X_new)
In [22]:
import matplotlib.pyplot as plt

fig, ax = plt.subplots();

ax.scatter(X, Y);
ax.plot(X_new, Y_predict, 'r')
Out[22]:
[<matplotlib.lines.Line2D at 0x1a1583b2d0>]