Define the objective in business terms

How will your solution be used?

What are the current solutions/workarounds (if any)?

How should you frame this problem (supervised/unsupervised, online/offline, etc.)?

How should performance be measured?

Is the performance measure aligned with the business objective?

What would be the minimum performance needed to reach the business objective?

What are comparable problem? Can you reuse experience or tools

How would you solve the problem manually?

List hte assumptions you (for others) have made so far.

Verify assumptions if possible.

List the data you need and how much you need.

Find and document where you can get that data.

Check how much space it will take.

Check legal obligations, and get authorization if necessary.

Get access authorizations.

Create a workspace (with enough storage space).

Get the data.

Convert the data to a format you can easily manipulate (without changing the data itself).

Ensure sensitive information is deleted or protected (e.g., anonymized).

Check the size and type of data (time series, sample, geographical, etc.).

Sample a test set, put it aside, and never look at it (no data snooping).

Create a copy of the data for exploration (sampling it down to a manageable size if necessary).

Create a Jupyter notebook to keep a record of your data exploration.

Study each attribute and its characteristics:

• Name
• Type (categorical, int/float, bounded/unbounded, text, structured, etc.)
• % of missing values
• Noisiness and type of noise (stochastic, outliers, rounding errors, etc.)
• Possibly useful for the task?
• Type of distribution (Gaussina, uniform, logarithmic, etc.)

For supervised learning tasks, identify the target attribute(s).

Visualize the data.

Study the correlations between attributes.

Study how you would solve the problem manually.

Identify the promising transformations you may want to apply.

Identtify extra data that would be useful (go back to "Get the Data").

Document what you have learned.

Work on copies of the data (keeyp the original dataset intact)

Write functions for all data transformations you apply, for five reasons:

• So you can easily prepare the data the next time you get a fresh dataset
• So you can apply these transformations in future projects
• To clean and prepare the test set
• To clearn and prepare new data instance once your solution is live
• To make it easy to treat your preparation choices as hyperparameters

1. Data clearning
   • Fix or remove outliers
   • Fill in missing values or drop their rows (or columns)
2. Feature selection
   • Drop the attributes that provide no useful information for the task
3. Feature engineering, where appropirate:
   • Discretize continuous features
   • Decompose features (e.g., categorical, date/time, etc.)
   • Add promising transformations of features
   • Agregate features into promising new features
4. Feature scaling: standardize or normalize features

1. Train many quick and dirty models from different categories (e.g., linear, naive Bayes, SVM, Random Forests, neural net, etc.) using standard parameters
2. Measure and compare their performance
   • For each model, use N-fold cross-validation and compute the mean and standard deviation of the performance measure on the N folds
3. Analyze the most significant vairables for each algorithm
4. Analyze the types of errors the models make
   • What data would a human have used to avoid these errors
5. Have a quick round of feature selection and engineering
6. Have one or two more quick iterations of the five previous steps
7. Short-lsit the top three to five most promising models, preferring models that make different types of errors

1. Fine-tune the hyperparameters using crossing-validation
   • Treat your data transformation choices as hyperparameters, especially when you are not sure about them
   • Unless there are very few hyperparameter values to explore, prefer random search over grid search. If training is very long, you may prefer a Bayesian optimization approach
2. Try Ensemble methods. Combining your best models will often perform better than runing them individually
3. Once you are confident about your final model, measure its performance on the test set to estimate the generalization error

1. Document what you have done
2. Create a nice presentation, make sure you highlight the big picture first
3. Explain why your solution achieves the business objective
4. Don't forget to present interesting points you noticed along the way
   • Describe what worked and what did not
   • List your assumptions and your system's limitations
5. Ensure your key findings are communicated through beautiful visualizations or easy-to-remember statements

1. Get your solution ready for production (plug into production data inputs, write unit test, etc.)
2. Write monitoring code to check your system's live performance at regular itervals and trigger alerts when it drops
   • Beware of slow degradation too: models tend to "rot" as data evolves
   • Measuring performance may require a human pipeline
   • Also monitor your inputs's quality. This is particularly important for online learning systems
3. Retrain your models on a regular basis on fresh data

Hands-On Machine Learning with Scikit-Learn & TensorFlow, Appendix B