Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems

1

Siam Mapped: A History of the Geo-Body of a Nation University of Hawaii Press Thongchai Winichakul Year: 1994 Language: english File: PDF, 4.84 MB

2

The Sacred Seduction: Forbidden Truths—Love, Sex & Seduction Mindbuilt Design Studio Rumio Year: 2015 Language: english File: AZW3 , 1.98 MB

SECOND EDITION

Hands-on Machine Learning with
Scikit-Learn, Keras, and
TensorFlow

Concepts, Tools, and Techniques to
Build Intelligent Systems

Aurélien Géron

Beijing

Boston Farnham Sebastopol

Tokyo

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow
by Aurélien Géron
Copyright © 2019 Aurélien Géron. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional
sales department: 800-998-9938 or corporate@oreilly.com.

Editor: Nicole Tache
Interior Designer: David Futato
June 2019:

Cover Designer: Karen Montgomery
Illustrator: Rebecca Demarest

Second Edition

Revision History for the Early Release
2018-11-05: First Release
2019-01-24: Second Release
2019-03-07: Third Release
2019-03-29: Fourth Release
2019-04-22: Fifth Release
See http://oreilly.com/catalog/errata.csp?isbn=9781492032649 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Hands-on Machine Learning with
Scikit-Learn, Keras, and TensorFlow, the cover image, and related trade dress are trademarks of O’Reilly
Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the author disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.

978-1-492-03264-9
[LSI]

Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Part I.

The Fundamentals of Machine Learning

1. The Machine Learning Landscape. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
What Is Machine Learning?
Why Use Machine Learning?
Types of Machine Learning Systems
Supervised/Unsupervised Learning
Batch and Online Learning
Instance-Based Versus Model-Based Learning
Main Challenges of Machine Learning
Insufficient Quantity of Training Data
Nonrepresentative Training Data
Poor-Quality Data
Irrelevant Features
Overfitting the Training Data
Underfitting the Training Data
Stepping Back
Testing and Validating
Hyperparameter Tuning and Model Selection
Data Mismatch
Exercises

4
4
8
8
15
18
24
24
26
27
27
28
30
30
31
32
33
34

2. End-to-End Machine Learning Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Working with Real Data
Look at the Big Picture

38
39
iii

Frame the Problem
Select a Performance Measure
Check the Assumptions
Get the Data
Create the Workspace
Download the Data
Take a Quick Look at the Data Structure
Create a Test Set
Discover and Visualize the Data to Gain Insights
Visualizing Geographical Data
Looking for Correlations
Experimenting with Attribute Combinations
Prepare the Data for Machine Learning Algorithms
Data Cleaning
Handling Text and Categorical Attributes
Custom Transformers
Feature Scaling
Transformation Pipelines
Select and Train a Model
Training and Evaluating on the Training Set
Better Evaluation Using Cross-Validation
Fine-Tune Your Model
Grid Search
Randomized Search
Ensemble Methods
Analyze the Best Models and Their Errors
Evaluate Your System on the Test Set
Launch, Monitor, and Maintain Your System
Try It Out!
Exercises

39
42
45
45
45
49
50
54
58
59
62
65
66
67
69
71
72
73
75
75
76
79
79
81
82
82
83
84
85
85

3. Classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
MNIST
Training a Binary Classifier
Performance Measures
Measuring Accuracy Using Cross-Validation
Confusion Matrix
Precision and Recall
Precision/Recall Tradeoff
The ROC Curve
Multiclass Classification
Error Analysis

iv

|

Table of Contents

87
90
90
91
92
94
95
99
102
104

Multilabel Classification
Multioutput Classification
Exercises

108
109
110

4. Training Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Linear Regression
The Normal Equation
Computational Complexity
Gradient Descent
Batch Gradient Descent
Stochastic Gradient Descent
Mini-batch Gradient Descent
Polynomial Regression
Learning Curves
Regularized Linear Models
Ridge Regression
Lasso Regression
Elastic Net
Early Stopping
Logistic Regression
Estimating Probabilities
Training and Cost Function
Decision Boundaries
Softmax Regression
Exercises

114
116
119
119
123
126
129
130
132
136
137
139
142
142
144
144
145
146
149
153

5. Support Vector Machines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Linear SVM Classification
Soft Margin Classification
Nonlinear SVM Classification
Polynomial Kernel
Adding Similarity Features
Gaussian RBF Kernel
Computational Complexity
SVM Regression
Under the Hood
Decision Function and Predictions
Training Objective
Quadratic Programming
The Dual Problem
Kernelized SVM
Online SVMs

155
156
159
160
161
162
163
164
166
166
167
169
170
171
174

Table of Contents

|

v

Exercises

175

6. Decision Trees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Training and Visualizing a Decision Tree
Making Predictions
Estimating Class Probabilities
The CART Training Algorithm
Computational Complexity
Gini Impurity or Entropy?
Regularization Hyperparameters
Regression
Instability
Exercises

177
179
181
182
183
183
184
185
188
189

7. Ensemble Learning and Random Forests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Voting Classifiers
Bagging and Pasting
Bagging and Pasting in Scikit-Learn
Out-of-Bag Evaluation
Random Patches and Random Subspaces
Random Forests
Extra-Trees
Feature Importance
Boosting
AdaBoost
Gradient Boosting
Stacking
Exercises

192
195
196
197
198
199
200
200
201
202
205
210
213

8. Dimensionality Reduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
The Curse of Dimensionality
Main Approaches for Dimensionality Reduction
Projection
Manifold Learning
PCA
Preserving the Variance
Principal Components
Projecting Down to d Dimensions
Using Scikit-Learn
Explained Variance Ratio
Choosing the Right Number of Dimensions
PCA for Compression

vi

| Table of Contents

216
218
218
220
222
222
223
224
224
225
225
226

Randomized PCA
Incremental PCA
Kernel PCA
Selecting a Kernel and Tuning Hyperparameters
LLE
Other Dimensionality Reduction Techniques
Exercises

227
227
228
229
232
234
235

9. Unsupervised Learning Techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
Clustering
K-Means
Limits of K-Means
Using clustering for image segmentation
Using Clustering for Preprocessing
Using Clustering for Semi-Supervised Learning
DBSCAN
Other Clustering Algorithms
Gaussian Mixtures
Anomaly Detection using Gaussian Mixtures
Selecting the Number of Clusters
Bayesian Gaussian Mixture Models
Other Anomaly Detection and Novelty Detection Algorithms

238
240
250
251
252
254
256
259
260
266
267
270
274

Part II. Neural Networks and Deep Learning
10. Introduction to Artificial Neural Networks with Keras. . . . . . . . . . . . . . . . . . . . . . . . . . 277
From Biological to Artificial Neurons
Biological Neurons
Logical Computations with Neurons
The Perceptron
Multi-Layer Perceptron and Backpropagation
Regression MLPs
Classification MLPs
Implementing MLPs with Keras
Installing TensorFlow 2
Building an Image Classifier Using the Sequential API
Building a Regression MLP Using the Sequential API
Building Complex Models Using the Functional API
Building Dynamic Models Using the Subclassing API
Saving and Restoring a Model
Using Callbacks

278
279
281
281
286
289
290
292
293
294
303
304
309
311
311

Table of Contents

|

vii

Visualization Using TensorBoard
Fine-Tuning Neural Network Hyperparameters
Number of Hidden Layers
Number of Neurons per Hidden Layer
Learning Rate, Batch Size and Other Hyperparameters
Exercises

313
315
319
320
320
322

11. Training Deep Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
Vanishing/Exploding Gradients Problems
Glorot and He Initialization
Nonsaturating Activation Functions
Batch Normalization
Gradient Clipping
Reusing Pretrained Layers
Transfer Learning With Keras
Unsupervised Pretraining
Pretraining on an Auxiliary Task
Faster Optimizers
Momentum Optimization
Nesterov Accelerated Gradient
AdaGrad
RMSProp
Adam and Nadam Optimization
Learning Rate Scheduling
Avoiding Overfitting Through Regularization
ℓ1 and ℓ2 Regularization
Dropout
Monte-Carlo (MC) Dropout
Max-Norm Regularization
Summary and Practical Guidelines
Exercises

326
327
329
333
338
339
341
343
344
344
345
346
347
349
349
352
356
356
357
360
362
363
364

12. Custom Models and Training with TensorFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
A Quick Tour of TensorFlow
Using TensorFlow like NumPy
Tensors and Operations
Tensors and NumPy
Type Conversions
Variables
Other Data Structures
Customizing Models and Training Algorithms
Custom Loss Functions

viii

|

Table of Contents

368
371
371
373
374
374
375
376
376

Saving and Loading Models That Contain Custom Components
Custom Activation Functions, Initializers, Regularizers, and Constraints
Custom Metrics
Custom Layers
Custom Models
Losses and Metrics Based on Model Internals
Computing Gradients Using Autodiff
Custom Training Loops
TensorFlow Functions and Graphs
Autograph and Tracing
TF Function Rules

377
379
380
383
386
388
389
393
396
398
400

13. Loading and Preprocessing Data with TensorFlow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
The Data API
Chaining Transformations
Shuffling the Data
Preprocessing the Data
Putting Everything Together
Prefetching
Using the Dataset With tf.keras
The TFRecord Format
Compressed TFRecord Files
A Brief Introduction to Protocol Buffers
TensorFlow Protobufs
Loading and Parsing Examples
Handling Lists of Lists Using the SequenceExample Protobuf
The Features API
Categorical Features
Crossed Categorical Features
Encoding Categorical Features Using One-Hot Vectors
Encoding Categorical Features Using Embeddings
Using Feature Columns for Parsing
Using Feature Columns in Your Models
TF Transform
The TensorFlow Datasets (TFDS) Project

404
405
406
409
410
411
413
414
415
415
416
418
419
420
421
421
422
423
426
426
428
429

14. Deep Computer Vision Using Convolutional Neural Networks. . . . . . . . . . . . . . . . . . . 431
The Architecture of the Visual Cortex
Convolutional Layer
Filters
Stacking Multiple Feature Maps
TensorFlow Implementation

432
434
436
437
439

Table of Contents

|

ix

Memory Requirements
Pooling Layer
TensorFlow Implementation
CNN Architectures
LeNet-5
AlexNet
GoogLeNet
VGGNet
ResNet
Xception
SENet
Implementing a ResNet-34 CNN Using Keras
Using Pretrained Models From Keras
Pretrained Models for Transfer Learning
Classification and Localization
Object Detection
Fully Convolutional Networks (FCNs)
You Only Look Once (YOLO)
Semantic Segmentation
Exercises

x

|

Table of Contents

441
442
444
446
449
450
452
456
457
459
461
464
465
467
469
471
473
475
478
482

Preface

The Machine Learning Tsunami
In 2006, Geoffrey Hinton et al. published a paper1 showing how to train a deep neural
network capable of recognizing handwritten digits with state-of-the-art precision
(>98%). They branded this technique “Deep Learning.” Training a deep neural net
was widely considered impossible at the time,2 and most researchers had abandoned
the idea since the 1990s. This paper revived the interest of the scientific community
and before long many new papers demonstrated that Deep Learning was not only
possible, but capable of mind-blowing achievements that no other Machine Learning
(ML) technique could hope to match (with the help of tremendous computing power
and great amounts of data). This enthusiasm soon extended to many other areas of
Machine Learning.
Fast-forward 10 years and Machine Learning has conquered the industry: it is now at
the heart of much of the magic in today’s high-tech products, ranking your web
search results, powering your smartphone’s speech recognition, recommending vid‐
eos, and beating the world champion at the game of Go. Before you know it, it will be
driving your car.

Machine Learning in Your Projects
So naturally you are excited about Machine Learning and you would love to join the
party!
Perhaps you would like to give your homemade robot a brain of its own? Make it rec‐
ognize faces? Or learn to walk around?

1 Available on Hinton’s home page at http://www.cs.toronto.edu/~hinton/.
2 Despite the fact that Yann Lecun’s deep convolutional neural networks had worked well for image recognition

since the 1990s, although they were not as general purpose.

xi

Or maybe your company has tons of data (user logs, financial data, production data,
machine sensor data, hotline stats, HR reports, etc.), and more than likely you could
unearth some hidden gems if you just knew where to look; for example:
• Segment customers and find the best marketing strategy for each group
• Recommend products for each client based on what similar clients bought
• Detect which transactions are likely to be fraudulent
• Forecast next year’s revenue
• And more
Whatever the reason, you have decided to learn Machine Learning and implement it
in your projects. Great idea!

Objective and Approach
This book assumes that you know close to nothing about Machine Learning. Its goal
is to give you the concepts, the intuitions, and the tools you need to actually imple‐
ment programs capable of learning from data.
We will cover a large number of techniques, from the simplest and most commonly
used (such as linear regression) to some of the Deep Learning techniques that regu‐
larly win competitions.
Rather than implementing our own toy versions of each algorithm, we will be using
actual production-ready Python frameworks:
• Scikit-Learn is very easy to use, yet it implements many Machine Learning algo‐
rithms efficiently, so it makes for a great entry point to learn Machine Learning.
• TensorFlow is a more complex library for distributed numerical computation. It
makes it possible to train and run very large neural networks efficiently by dis‐
tributing the computations across potentially hundreds of multi-GPU servers.
TensorFlow was created at Google and supports many of their large-scale
Machine Learning applications. It was open sourced in November 2015.
• Keras is a high level Deep Learning API that makes it very simple to train and
run neural networks. It can run on top of either TensorFlow, Theano or Micro‐
soft Cognitive Toolkit (formerly known as CNTK). TensorFlow comes with its
own implementation of this API, called tf.keras, which provides support for some
advanced TensorFlow features (e.g., to efficiently load data).
The book favors a hands-on approach, growing an intuitive understanding of
Machine Learning through concrete working examples and just a little bit of theory.
While you can read this book without picking up your laptop, we highly recommend

xii

|

Preface

you experiment with the code examples available online as Jupyter notebooks at
https://github.com/ageron/handson-ml2.

Prerequisites
This book assumes that you have some Python programming experience and that you
are familiar with Python’s main scientific libraries, in particular NumPy, Pandas, and
Matplotlib.
Also, if you care about what’s under the hood you should have a reasonable under‐
standing of college-level math as well (calculus, linear algebra, probabilities, and sta‐
tistics).
If you don’t know Python yet, http://learnpython.org/ is a great place to start. The offi‐
cial tutorial on python.org is also quite good.
If you have never used Jupyter, Chapter 2 will guide you through installation and the
basics: it is a great tool to have in your toolbox.
If you are not familiar with Python’s scientific libraries, the provided Jupyter note‐
books include a few tutorials. There is also a quick math tutorial for linear algebra.

Roadmap
This book is organized in two parts. Part I, The Fundamentals of Machine Learning,
covers the following topics:
• What is Machine Learning? What problems does it try to solve? What are the
main categories and fundamental concepts of Machine Learning systems?
• The main steps in a typical Machine Learning project.
• Learning by fitting a model to data.
• Optimizing a cost function.
• Handling, cleaning, and preparing data.
• Selecting and engineering features.
• Selecting a model and tuning hyperparameters using cross-validation.
• The main challenges of Machine Learning, in particular underfitting and overfit‐
ting (the bias/variance tradeoff).
• Reducing the dimensionality of the training data to fight the curse of dimension‐
ality.
• Other unsupervised learning techniques, including clustering, density estimation
and anomaly detection.

Preface

|

xiii

• The most common learning algorithms: Linear and Polynomial Regression,
Logistic Regression, k-Nearest Neighbors, Support Vector Machines, Decision
Trees, Random Forests, and Ensemble methods.

xiv

|

Preface

Part II, Neural Networks and Deep Learning, covers the following topics:
• What are neural nets? What are they good for?
• Building and training neural nets using TensorFlow and Keras.
• The most important neural net architectures: feedforward neural nets, convolu‐
tional nets, recurrent nets, long short-term memory (LSTM) nets, autoencoders
and generative adversarial networks (GANs).
• Techniques for training deep neural nets.
• Scaling neural networks for large datasets.
• Learning strategies with Reinforcement Learning.
• Handling uncertainty with Bayesian Deep Learning.
The first part is based mostly on Scikit-Learn while the second part uses TensorFlow
and Keras.
Don’t jump into deep waters too hastily: while Deep Learning is no
doubt one of the most exciting areas in Machine Learning, you
should master the fundamentals first. Moreover, most problems
can be solved quite well using simpler techniques such as Random
Forests and Ensemble methods (discussed in Part I). Deep Learn‐
ing is best suited for complex problems such as image recognition,
speech recognition, or natural language processing, provided you
have enough data, computing power, and patience.

Other Resources
Many resources are available to learn about Machine Learning. Andrew Ng’s ML
course on Coursera and Geoffrey Hinton’s course on neural networks and Deep
Learning are amazing, although they both require a significant time investment
(think months).
There are also many interesting websites about Machine Learning, including of
course Scikit-Learn’s exceptional User Guide. You may also enjoy Dataquest, which
provides very nice interactive tutorials, and ML blogs such as those listed on Quora.
Finally, the Deep Learning website has a good list of resources to learn more.
Of course there are also many other introductory books about Machine Learning, in
particular:
• Joel Grus, Data Science from Scratch (O’Reilly). This book presents the funda‐
mentals of Machine Learning, and implements some of the main algorithms in
pure Python (from scratch, as the name suggests).

Preface

|

xv

• Stephen Marsland, Machine Learning: An Algorithmic Perspective (Chapman and
Hall). This book is a great introduction to Machine Learning, covering a wide
range of topics in depth, with code examples in Python (also from scratch, but
using NumPy).
• Sebastian Raschka, Python Machine Learning (Packt Publishing). Also a great
introduction to Machine Learning, this book leverages Python open source libra‐
ries (Pylearn 2 and Theano).
• François Chollet, Deep Learning with Python (Manning). A very practical book
that covers a large range of topics in a clear and concise way, as you might expect
from the author of the excellent Keras library. It favors code examples over math‐
ematical theory.
• Yaser S. Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin, Learning from
Data (AMLBook). A rather theoretical approach to ML, this book provides deep
insights, in particular on the bias/variance tradeoff (see Chapter 4).
• Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 3rd
Edition (Pearson). This is a great (and huge) book covering an incredible amount
of topics, including Machine Learning. It helps put ML into perspective.
Finally, a great way to learn is to join ML competition websites such as Kaggle.com
this will allow you to practice your skills on real-world problems, with help and
insights from some of the best ML professionals out there.

Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width

Used for program listings, as well as within paragraphs to refer to program ele‐
ments such as variable or function names, databases, data types, environment
variables, statements and keywords.
Constant width bold

Shows commands or other text that should be typed literally by the user.
Constant width italic

Shows text that should be replaced with user-supplied values or by values deter‐
mined by context.

xvi

|

Preface

This element signifies a tip or suggestion.

This element signifies a general note.

This element indicates a warning or caution.

Code Examples
Supplemental material (code examples, exercises, etc.) is available for download at
https://github.com/ageron/handson-ml2. It is mostly composed of Jupyter notebooks.
Some of the code examples in the book leave out some repetitive sections, or details
that are obvious or unrelated to Machine Learning. This keeps the focus on the
important parts of the code, and it saves space to cover more topics. However, if you
want the full code examples, they are all available in the Jupyter notebooks.
Note that when the code examples display some outputs, then these code examples
are shown with Python prompts (>>> and ...), as in a Python shell, to clearly distin‐
guish the code from the outputs. For example, this code defines the square() func‐
tion then it computes and displays the square of 3:
>>> def square(x):
...
return x ** 2
...
>>> result = square(3)
>>> result
9

When code does not display anything, prompts are not used. However, the result may
sometimes be shown as a comment like this:
def square(x):
return x ** 2
result = square(3)

# result is 9

Preface

|

xvii

Using Code Examples
This book is here to help you get your job done. In general, if example code is offered
with this book, you may use it in your programs and documentation. You do not
need to contact us for permission unless you’re reproducing a significant portion of
the code. For example, writing a program that uses several chunks of code from this
book does not require permission. Selling or distributing a CD-ROM of examples
from O’Reilly books does require permission. Answering a question by citing this
book and quoting example code does not require permission. Incorporating a signifi‐
cant amount of example code from this book into your product’s documentation does
require permission.
We appreciate, but do not require, attribution. An attribution usually includes the
title, author, publisher, and ISBN. For example: “Hands-On Machine Learning with
Scikit-Learn, Keras and TensorFlow by Aurélien Géron (O’Reilly). Copyright 2019
Aurélien Géron, 978-1-492-03264-9.” If you feel your use of code examples falls out‐
side fair use or the permission given above, feel free to contact us at permis‐
sions@oreilly.com.

O’Reilly Safari
Safari (formerly Safari Books Online) is a membership-based
training and reference platform for enterprise, government,
educators, and individuals.
Members have access to thousands of books, training videos, Learning Paths, interac‐
tive tutorials, and curated playlists from over 250 publishers, including O’Reilly
Media, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Profes‐
sional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press,
John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe
Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and
Course Technology, among others.
For more information, please visit http://oreilly.com/safari.

How to Contact Us
Please address comments and questions concerning this book to the publisher:
O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
xviii

|

Preface

707-829-0515 (international or local)
707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at http://bit.ly/hands-on-machine-learningwith-scikit-learn-and-tensorflow or https://homl.info/oreilly.
To comment or ask technical questions about this book, send email to bookques‐
tions@oreilly.com.
For more information about our books, courses, conferences, and news, see our web‐
site at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia

Changes in the Second Edition
This second edition has five main objectives:
1. Cover additional topics: additional unsupervised learning techniques (including
clustering, anomaly detection, density estimation and mixture models), addi‐
tional techniques for training deep nets (including self-normalized networks),
additional computer vision techniques (including the Xception, SENet, object
detection with YOLO, and semantic segmentation using R-CNN), handling
sequences using CNNs (including WaveNet), natural language processing using
RNNs, CNNs and Transformers, generative adversarial networks, deploying Ten‐
sorFlow models, and more.
2. Update the book to mention some of the latest results from Deep Learning
research.
3. Migrate all TensorFlow chapters to TensorFlow 2, and use TensorFlow’s imple‐
mentation of the Keras API (called tf.keras) whenever possible, to simplify the
code examples.
4. Update the code examples to use the latest version of Scikit-Learn, NumPy, Pan‐
das, Matplotlib and other libraries.
5. Clarify some sections and fix some errors, thanks to plenty of great feedback
from readers.
Some chapters were added, others were rewritten and a few were reordered. Table P-1
shows the mapping between the 1st edition chapters and the 2nd edition chapters:

Preface

|

xix

Table P-1. Chapter mapping between 1st and 2nd edition
1st Ed. chapter 2nd Ed. Chapter % Changes
1
1
<10%

2nd Ed. Title
The Machine Learning Landscape

2

2

<10%

End-to-End Machine Learning Project

3

3

<10%

Classification

4

4

<10%

Training Models

5

5

<10%

Support Vector Machines

6

6

<10%

Decision Trees

7

7

<10%

Ensemble Learning and Random Forests

8

8

<10%

Dimensionality Reduction

N/A

9

100% new

Unsupervised Learning Techniques

10

10

~75%

Introduction to Artificial Neural Networks with Keras

11

11

~50%

Training Deep Neural Networks

9

12

100% rewritten Custom Models and Training with TensorFlow

Part of 12

13

100% rewritten Loading and Preprocessing Data with TensorFlow

13

14

~50%

Deep Computer Vision Using Convolutional Neural Networks

Part of 14

15

~75%

Processing Sequences Using RNNs and CNNs

Part of 14

16

~90%

Natural Language Processing with RNNs and Attention

15

17

~75%

Autoencoders and GANs

16

18

~75%

Reinforcement Learning

Part of 12

19

100% rewritten Deploying your TensorFlow Models

More specifically, here are the main changes for each 2nd edition chapter (other than
clarifications, corrections and code updates):
• Chapter 1
— Added a section on handling mismatch between the training set and the vali‐
dation & test sets.
• Chapter 2
— Added how to compute a confidence interval.
— Improved the installation instructions (e.g., for Windows).
— Introduced the upgraded OneHotEncoder and the new ColumnTransformer.
• Chapter 4
— Explained the need for training instances to be Independent and Identically
Distributed (IID).
• Chapter 7
— Added a short section about XGBoost.

xx

|

Preface

• Chapter 9 – new chapter including:
— Clustering with K-Means, how to choose the number of clusters, how to use it
for dimensionality reduction, semi-supervised learning, image segmentation,
and more.
— The DBSCAN clustering algorithm and an overview of other clustering algo‐
rithms available in Scikit-Learn.
— Gaussian mixture models, the Expectation-Maximization (EM) algorithm,
Bayesian variational inference, and how mixture models can be used for clus‐
tering, density estimation, anomaly detection and novelty detection.
— Overview of other anomaly detection and novelty detection algorithms.
• Chapter 10 (mostly new)
— Added an introduction to the Keras API, including all its APIs (Sequential,
Functional and Subclassing), persistence and callbacks (including the Tensor
Board callback).
• Chapter 11 (many changes)
— Introduced self-normalizing nets, the SELU activation function and Alpha
Dropout.
— Introduced self-supervised learning.
— Added Nadam optimization.
— Added Monte-Carlo Dropout.
— Added a note about the risks of adaptive optimization methods.
— Updated the practical guidelines.
• Chapter 12 – completely rewritten chapter, including:
— A tour of TensorFlow 2
— TensorFlow’s lower-level Python API
— Writing custom loss functions, metrics, layers, models
— Using auto-differentiation and creating custom training algorithms.
— TensorFlow Functions and graphs (including tracing and autograph).
• Chapter 13 – new chapter, including:
— The Data API
— Loading/Storing data efficiently using TFRecords
— The Features API (including an introduction to embeddings).
— An overview of TF Transform and TF Datasets
— Moved the low-level implementation of the neural network to the exercises.

Preface

|

xxi

— Removed details about queues and readers that are now superseded by the
Data API.
• Chapter 14
— Added Xception and SENet architectures.
— Added a Keras implementation of ResNet-34.
— Showed how to use pretrained models using Keras.
— Added an end-to-end transfer learning example.
— Added classification and localization.
— Introduced Fully Convolutional Networks (FCNs).
— Introduced object detection using the YOLO architecture.
— Introduced semantic segmentation using R-CNN.
• Chapter 15
— Added an introduction to Wavenet.
— Moved the Encoder–Decoder architecture and Bidirectional RNNs to Chapter
16.
• Chapter 16
— Explained how to use the Data API to handle sequential data.
— Showed an end-to-end example of text generation using a Character RNN,
using both a stateless and a stateful RNN.
— Showed an end-to-end example of sentiment analysis using an LSTM.
— Explained masking in Keras.
— Showed how to reuse pretrained embeddings using TF Hub.
— Showed how to build an Encoder–Decoder for Neural Machine Translation
using TensorFlow Addons/seq2seq.
— Introduced beam search.
— Explained attention mechanisms.
— Added a short overview of visual attention and a note on explainability.
— Introduced the fully attention-based Transformer architecture, including posi‐
tional embeddings and multi-head attention.
— Added an overview of recent language models (2018).
• Chapters 17, 18 and 19: coming soon.

xxii

|

Preface

Acknowledgments
Never in my wildest dreams did I imagine that the first edition of this book would get
such a large audience. I received so many messages from readers, many asking ques‐
tions, some kindly pointing out errata, and most sending me encouraging words. I
cannot express how grateful I am to all these readers for their tremendous support.
Thank you all so very much! Please do not hesitate to file issues on github if you find
errors in the code examples (or just to ask questions), or to submit errata if you find
errors in the text. Some readers also shared how this book helped them get their first
job, or how it helped them solve a concrete problem they were working on: I find
such feedback incredibly motivating. If you find this book helpful, I would love it if
you could share your story with me, either privately (e.g., via LinkedIn) or publicly
(e.g., in an Amazon review).
I am also incredibly thankful to all the amazing people who took time out of their
busy lives to review my book with such care. In particular, I would like to thank Fran‐
çois Chollet for reviewing all the chapters based on Keras & TensorFlow, and giving
me some great, in-depth feedback. Since Keras is one of the main additions to this 2nd
edition, having its author review the book was invaluable. I highly recommend Fran‐
çois’s excellent book Deep Learning with Python3: it has the conciseness, clarity and
depth of the Keras library itself. Big thanks as well to Ankur Patel, who reviewed
every chapter of this 2nd edition and gave me excellent feedback.
This book also benefited from plenty of help from members of the TensorFlow team,
in particular Martin Wicke, who tirelessly answered dozens of my questions and dis‐
patched the rest to the right people, including Alexandre Passos, Allen Lavoie, André
Susano Pinto, Anna Revinskaya, Anthony Platanios, Clemens Mewald, Dan Moldo‐
van, Daniel Dobson, Dustin Tran, Edd Wilder-James, Goldie Gadde, Jiri Simsa, Kar‐
mel Allison, Nick Felt, Paige Bailey, Pete Warden (who also reviewed the 1st edition),
Ryan Sepassi, Sandeep Gupta, Sean Morgan, Todd Wang, Tom O’Malley, William
Chargin, and Yuefeng Zhou, all of whom were tremendously helpful. A huge thank
you to all of you, and to all other members of the TensorFlow team. Not just for your
help, but also for making such a great library.
Big thanks to Haesun Park, who gave me plenty of excellent feedback and caught sev‐
eral errors while he was writing the Korean translation of the 1st edition of this book.
He also translated the Jupyter notebooks to Korean, not to mention TensorFlow’s
documentation. I do not speak Korean, but judging by the quality of his feedback, all
his translations must be truly excellent! Moreover, he kindly contributed some of the
solutions to the exercises in this book.

3 “Deep Learning with Python,” François Chollet (2017).

Preface

|

xxiii

Many thanks as well to O’Reilly’s fantastic staff, in particular Nicole Tache, who gave
me insightful feedback, always cheerful, encouraging, and helpful: I could not dream
of a better editor. Big thanks to Michele Cronin as well, who was very helpful (and
patient) at the start of this 2nd edition. Thanks to Marie Beaugureau, Ben Lorica, Mike
Loukides, and Laurel Ruma for believing in this project and helping me define its
scope. Thanks to Matt Hacker and all of the Atlas team for answering all my technical
questions regarding formatting, asciidoc, and LaTeX, and thanks to Rachel Mona‐
ghan, Nick Adams, and all of the production team for their final review and their
hundreds of corrections.
I would also like to thank my former Google colleagues, in particular the YouTube
video classification team, for teaching me so much about Machine Learning. I could
never have started the first edition without them. Special thanks to my personal ML
gurus: Clément Courbet, Julien Dubois, Mathias Kende, Daniel Kitachewsky, James
Pack, Alexander Pak, Anosh Raj, Vitor Sessak, Wiktor Tomczak, Ingrid von Glehn,
Rich Washington, and everyone I worked with at YouTube and in the amazing Goo‐
gle research teams in Mountain View. All these people are just as nice and helpful as
they are bright, and that’s saying a lot.
I will never forget the kind people who reviewed the 1st edition of this book, including
David Andrzejewski, Eddy Hung, Grégoire Mesnil, Iain Smears, Ingrid von Glehn,
Justin Francis, Karim Matrah, Lukas Biewald, Michel Tessier, Salim Sémaoune, Vin‐
cent Guilbeau and of course my dear brother Sylvain.
Last but not least, I am infinitely grateful to my beloved wife, Emmanuelle, and to our
three wonderful children, Alexandre, Rémi, and Gabrielle, for encouraging me to
work hard on this book, as well as for their insatiable curiosity: explaining some of
the most difficult concepts in this book to my wife and children helped me clarify my
thoughts and directly improved many parts of this book. Plus, they keep bringing me
cookies and coffee! What more can one dream of?

xxiv

|

Preface

PART I

The Fundamentals of
Machine Learning

CHAPTER 1

The Machine Learning Landscape

With Early Release ebooks, you get books in their earliest form—
the author’s raw and unedited content as he or she writes—so you
can take advantage of these technologies long before the official
release of these titles. The following will be Chapter 1 in the final
release of the book.

When most people hear “Machine Learning,” they picture a robot: a dependable but‐
ler or a deadly Terminator depending on who you ask. But Machine Learning is not
just a futuristic fantasy, it’s already here. In fact, it has been around for decades in
some specialized applications, such as Optical Character Recognition (OCR). But the
first ML application that really became mainstream, improving the lives of hundreds
of millions of people, took over the world back in the 1990s: it was the spam filter.
Not exactly a self-aware Skynet, but it does technically qualify as Machine Learning
(it has actually learned so well that you seldom need to flag an email as spam any‐
more). It was followed by hundreds of ML applications that now quietly power hun‐
dreds of products and features that you use regularly, from better recommendations
to voice search.
Where does Machine Learning start and where does it end? What exactly does it
mean for a machine to learn something? If I download a copy of Wikipedia, has my
computer really “learned” something? Is it suddenly smarter? In this chapter we will
start by clarifying what Machine Learning is and why you may want to use it.
Then, before we set out to explore the Machine Learning continent, we will take a
look at the map and learn about the main regions and the most notable landmarks:
supervised versus unsupervised learning, online versus batch learning, instancebased versus model-based learning. Then we will look at the workflow of a typical ML
project, discuss the main challenges you may face, and cover how to evaluate and
fine-tune a Machine Learning system.
3

This chapter introduces a lot of fundamental concepts (and jargon) that every data
scientist should know by heart. It will be a high-level overview (the only chapter
without much code), all rather simple, but you should make sure everything is
crystal-clear to you before continuing to the rest of the book. So grab a coffee and let’s
get started!
If you already know all the Machine Learning basics, you may want
to skip directly to Chapter 2. If you are not sure, try to answer all
the questions listed at the end of the chapter before moving on.

What Is Machine Learning?
Machine Learning is the science (and art) of programming computers so they can
learn from data.
Here is a slightly more general definition:
[Machine Learning is the] field of study that gives computers the ability to learn
without being explicitly programmed.
—Arthur Samuel, 1959

And a more engineering-oriented one:
A computer program is said to learn from experience E with respect to some task T
and some performance measure P, if its performance on T, as measured by P, improves
with experience E.
—Tom Mitchell, 1997

For example, your spam filter is a Machine Learning program that can learn to flag
spam given examples of spam emails (e.g., flagged by users) and examples of regular
(nonspam, also called “ham”) emails. The examples that the system uses to learn are
called the training set. Each training example is called a training instance (or sample).
In this case, the task T is to flag spam for new emails, the experience E is the training
data, and the performance measure P needs to be defined; for example, you can use
the ratio of correctly classified emails. This particular performance measure is called
accuracy and it is often used in classification tasks.
If you just download a copy of Wikipedia, your computer has a lot more data, but it is
not suddenly better at any task. Thus, it is not Machine Learning.

Why Use Machine Learning?
Consider how you would write a spam filter using traditional programming techni‐
ques (Figure 1-1):
4

|

Chapter 1: The Machine Learning Landscape

1. First you would look at what spam typically looks like. You might notice that
some words or phrases (such as “4U,” “credit card,” “free,” and “amazing”) tend to
come up a lot in the subject. Perhaps you would also notice a few other patterns
in the sender’s name, the email’s body, and so on.
2. You would write a detection algorithm for each of the patterns that you noticed,
and your program would flag emails as spam if a number of these patterns are
detected.
3. You would test your program, and repeat steps 1 and 2 until it is good enough.

Figure 1-1. The traditional approach
Since the problem is not trivial, your program will likely become a long list of com‐
plex rules—pretty hard to maintain.
In contrast, a spam filter based on Machine Learning techniques automatically learns
which words and phrases are good predictors of spam by detecting unusually fre‐
quent patterns of words in the spam examples compared to the ham examples
(Figure 1-2). The program is much shorter, easier to maintain, and most likely more
accurate.

Why Use Machine Learning?

|

5

Figure 1-2. Machine Learning approach
Moreover, if spammers notice that all their emails containing “4U” are blocked, they
might start writing “For U” instead. A spam filter using traditional programming
techniques would need to be updated to flag “For U” emails. If spammers keep work‐
ing around your spam filter, you will need to keep writing new rules forever.
In contrast, a spam filter based on Machine Learning techniques automatically noti‐
ces that “For U” has become unusually frequent in spam flagged by users, and it starts
flagging them without your intervention (Figure 1-3).

Figure 1-3. Automatically adapting to change
Another area where Machine Learning shines is for problems that either are too com‐
plex for traditional approaches or have no known algorithm. For example, consider
speech recognition: say you want to start simple and write a program capable of dis‐
tinguishing the words “one” and “two.” You might notice that the word “two” starts
with a high-pitch sound (“T”), so you could hardcode an algorithm that measures
high-pitch sound intensity and use that to distinguish ones and twos. Obviously this
technique will not scale to thousands of words spoken by millions of very different
6

|

Chapter 1: The Machine Learning Landscape

people in noisy environments and in dozens of languages. The best solution (at least
today) is to write an algorithm that learns by itself, given many example recordings
for each word.
Finally, Machine Learning can help humans learn (Figure 1-4): ML algorithms can be
inspected to see what they have learned (although for some algorithms this can be
tricky). For instance, once the spam filter has been trained on enough spam, it can
easily be inspected to reveal the list of words and combinations of words that it
believes are the best predictors of spam. Sometimes this will reveal unsuspected cor‐
relations or new trends, and thereby lead to a better understanding of the problem.
Applying ML techniques to dig into large amounts of data can help discover patterns
that were not immediately apparent. This is called data mining.

Figure 1-4. Machine Learning can help humans learn
To summarize, Machine Learning is great for:
• Problems for which existing solutions require a lot of hand-tuning or long lists of
rules: one Machine Learning algorithm can often simplify code and perform bet‐
ter.
• Complex problems for which there is no good solution at all using a traditional
approach: the best Machine Learning techniques can find a solution.
• Fluctuating environments: a Machine Learning system can adapt to new data.
• Getting insights about complex problems and large amounts of data.

Why Use Machine Learning?

|

7

Types of Machine Learning Systems
There are so many different types of Machine Learning systems that it is useful to
classify them in broad categories based on:
• Whether or not they are trained with human supervision (supervised, unsuper‐
vised, semisupervised, and Reinforcement Learning)
• Whether or not they can learn incrementally on the fly (online versus batch
learning)
• Whether they work by simply comparing new data points to known data points,
or instead detect patterns in the training data and build a predictive model, much
like scientists do (instance-based versus model-based learning)
These criteria are not exclusive; you can combine them in any way you like. For
example, a state-of-the-art spam filter may learn on the fly using a deep neural net‐
work model trained using examples of spam and ham; this makes it an online, modelbased, supervised learning system.
Let’s look at each of these criteria a bit more closely.

Supervised/Unsupervised Learning
Machine Learning systems can be classified according to the amount and type of
supervision they get during training. There are four major categories: supervised
learning, unsupervised learning, semisupervised learning, and Reinforcement Learn‐
ing.

Supervised learning
In supervised learning, the training data you feed to the algorithm includes the desired
solutions, called labels (Figure 1-5).

Figure 1-5. A labeled training set for supervised learning (e.g., spam classification)

8

|

Chapter 1: The Machine Learning Landscape

A typical supervised learning task is classification. The spam filter is a good example
of this: it is trained with many example emails along with their class (spam or ham),
and it must learn how to classify new emails.
Another typical task is to predict a target numeric value, such as the price of a car,
given a set of features (mileage, age, brand, etc.) called predictors. This sort of task is
called regression (Figure 1-6).1 To train the system, you need to give it many examples
of cars, including both their predictors and their labels (i.e., their prices).
In Machine Learning an attribute is a data type (e.g., “Mileage”),
while a feature has several meanings depending on the context, but
generally means an attribute plus its value (e.g., “Mileage =
15,000”). Many people use the words attribute and feature inter‐
changeably, though.

Figure 1-6. Regression
Note that some regression algorithms can be used for classification as well, and vice
versa. For example, Logistic Regression is commonly used for classification, as it can
output a value that corresponds to the probability of belonging to a given class (e.g.,
20% chance of being spam).

1 Fun fact: this odd-sounding name is a statistics term introduced by Francis Galton while he was studying the

fact that the children of tall people tend to be shorter than their parents. Since children were shorter, he called
this regression to the mean. This name was then applied to the methods he used to analyze correlations
between variables.

Types of Machine Learning Systems

|

9

Here are some of the most important supervised learning algorithms (covered in this
book):
• k-Nearest Neighbors
• Linear Regression
• Logistic Regression
• Support Vector Machines (SVMs)
• Decision Trees and Random Forests
• Neural networks2

Unsupervised learning
In unsupervised learning, as you might guess, the training data is unlabeled
(Figure 1-7). The system tries to learn without a teacher.

Figure 1-7. An unlabeled training set for unsupervised learning
Here are some of the most important unsupervised learning algorithms (most of
these are covered in Chapter 8 and Chapter 9):
• Clustering
— K-Means
— DBSCAN
— Hierarchical Cluster Analysis (HCA)
• Anomaly detection and novelty detection
— One-class SVM
— Isolation Forest
2 Some neural network architectures can be unsupervised, such as autoencoders and restricted Boltzmann

machines. They can also be semisupervised, such as in deep belief networks and unsupervised pretraining.

10

| Chapter 1: The Machine Learning Landscape

• Visualization and dimensionality reduction
— Principal Component Analysis (PCA)
— Kernel PCA
— Locally-Linear Embedding (LLE)
— t-distributed Stochastic Neighbor Embedding (t-SNE)
• Association rule learning
— Apriori
— Eclat
For example, say you have a lot of data about your blog’s visitors. You may want to
run a clustering algorithm to try to detect groups of similar visitors (Figure 1-8). At
no point do you tell the algorithm which group a visitor belongs to: it finds those
connections without your help. For example, it might notice that 40% of your visitors
are males who love comic books and generally read your blog in the evening, while
20% are young sci-fi lovers who visit during the weekends, and so on. If you use a
hierarchical clustering algorithm, it may also subdivide each group into smaller
groups. This may help you target your posts for each group.

Figure 1-8. Clustering
Visualization algorithms are also good examples of unsupervised learning algorithms:
you feed them a lot of complex and unlabeled data, and they output a 2D or 3D rep‐
resentation of your data that can easily be plotted (Figure 1-9). These algorithms try
to preserve as much structure as they can (e.g., trying to keep separate clusters in the
input space from overlapping in the visualization), so you can understand how the
data is organized and perhaps identify unsuspected patterns.

Types of Machine Learning Systems

|

11

Figure 1-9. Example of a t-SNE visualization highlighting semantic clusters3
A related task is dimensionality reduction, in which the goal is to simplify the data
without losing too much information. One way to do this is to merge several correla‐
ted features into one. For example, a car’s mileage may be very correlated with its age,
so the dimensionality reduction algorithm will merge them into one feature that rep‐
resents the car’s wear and tear. This is called feature extraction.
It is often a good idea to try to reduce the dimension of your train‐
ing data using a dimensionality reduction algorithm before you
feed it to another Machine Learning algorithm (such as a super‐
vised learning algorithm). It will run much faster, the data will take
up less disk and memory space, and in some cases it may also per‐
form better.

Yet another important unsupervised task is anomaly detection—for example, detect‐
ing unusual credit card transactions to prevent fraud, catching manufacturing defects,
or automatically removing outliers from a dataset before feeding it to another learn‐
ing algorithm. The system is shown mostly normal instances during training, so it
learns to recognize them and when it sees a new instance it can tell whether it looks

3 Notice how animals are rather well separated from vehicles, how horses are close to deer but far from birds,

and so on. Figure reproduced with permission from Socher, Ganjoo, Manning, and Ng (2013), “T-SNE visual‐
ization of the semantic word space.”

12

|

Chapter 1: The Machine Learning Landscape

like a normal one or whether it is likely an anomaly (see Figure 1-10). A very similar
task is novelty detection: the difference is that novelty detection algorithms expect to
see only normal data during training, while anomaly detection algorithms are usually
more tolerant, they can often perform well even with a small percentage of outliers in
the training set.

Figure 1-10. Anomaly detection
Finally, another common unsupervised task is association rule learning, in which the
goal is to dig into large amounts of data and discover interesting relations between
attributes. For example, suppose you own a supermarket. Running an association rule
on your sales logs may reveal that people who purchase barbecue sauce and potato
chips also tend to buy steak. Thus, you may want to place these items close to each
other.

Semisupervised learning
Some algorithms can deal with partially labeled training data, usually a lot of unla‐
beled data and a little bit of labeled data. This is called semisupervised learning
(Figure 1-11).
Some photo-hosting services, such as Google Photos, are good examples of this. Once
you upload all your family photos to the service, it automatically recognizes that the
same person A shows up in photos 1, 5, and 11, while another person B shows up in
photos 2, 5, and 7. This is the unsupervised part of the algorithm (clustering). Now all
the system needs is for you to tell it who these people are. Just one label per person,4
and it is able to name everyone in every photo, which is useful for searching photos.

4 That’s when the system works perfectly. In practice it often creates a few clusters per person, and sometimes

mixes up two people who look alike, so you need to provide a few labels per person and manually clean up
some clusters.

Types of Machine Learning Systems

|

13

Figure 1-11. Semisupervised learning
Most semisupervised learning algorithms are combinations of unsupervised and
supervised algorithms. For example, deep belief networks (DBNs) are based on unsu‐
pervised components called restricted Boltzmann machines (RBMs) stacked on top of
one another. RBMs are trained sequentially in an unsupervised manner, and then the
whole system is fine-tuned using supervised learning techniques.

Reinforcement Learning
Reinforcement Learning is a very different beast. The learning system, called an agent
in this context, can observe the environment, select and perform actions, and get
rewards in return (or penalties in the form of negative rewards, as in Figure 1-12). It
must then learn by itself what is the best strategy, called a policy, to get the most
reward over time. A policy defines what action the agent should choose when it is in a
given situation.

14

| Chapter 1: The Machine Learning Landscape

Figure 1-12. Reinforcement Learning
For example, many robots implement Reinforcement Learning algorithms to learn
how to walk. DeepMind’s AlphaGo program is also a good example of Reinforcement
Learning: it made the headlines in May 2017 when it beat the world champion Ke Jie
at the game of Go. It learned its winning policy by analyzing millions of games, and
then playing many games against itself. Note that learning was turned off during the
games against the champion; AlphaGo was just applying the policy it had learned.

Batch and Online Learning
Another criterion used to classify Machine Learning systems is whether or not the
system can learn incrementally from a stream of incoming data.

Batch learning
In batch learning, the system is incapable of learning incrementally: it must be trained
using all the available data. This will generally take a lot of time and computing
resources, so it is typically done offline. First the system is trained, and then it is
launched into production and runs without learning anymore; it just applies what it
has learned. This is called offline learning.
If you want a batch learning system to know about new data (such as a new type of
spam), you need to train a new version of the system from scratch on the full dataset
(not just the new data, but also the old data), then stop the old system and replace it
with the new one.
Fortunately, the whole process of training, evaluating, and launching a Machine
Learning system can be automated fairly easily (as shown in Figure 1-3), so even a
Types of Machine Learning Systems

|

15

batch learning system can adapt to change. Simply update the data and train a new
version of the system from scratch as often as needed.
This solution is simple and often works fine, but training using the full set of data can
take many hours, so you would typically train a new system only every 24 hours or
even just weekly. If your system needs to adapt to rapidly changing data (e.g., to pre‐
dict stock prices), then you need a more reactive solution.
Also, training on the full set of data requires a lot of computing resources (CPU,
memory space, disk space, disk I/O, network I/O, etc.). If you have a lot of data and
you automate your system to train from scratch every day, it will end up costing you a
lot of money. If the amount of data is huge, it may even be impossible to use a batch
learning algorithm.
Finally, if your system needs to be able to learn autonomously and it has limited
resources (e.g., a smartphone application or a rover on Mars), then carrying around
large amounts of training data and taking up a lot of resources to train for hours
every day is a showstopper.
Fortunately, a better option in all these cases is to use algorithms that are capable of
learning incrementally.

Online learning
In online learning, you train the system incrementally by feeding it data instances
sequentially, either individually or by small groups called mini-batches. Each learning
step is fast and cheap, so the system can learn about new data on the fly, as it arrives
(see Figure 1-13).

Figure 1-13. Online learning
Online learning is great for systems that receive data as a continuous flow (e.g., stock
prices) and need to adapt to change rapidly or autonomously. It is also a good option
16

| Chapter 1: The Machine Learning Landscape

if you have limited computing resources: once an online learning system has learned
about new data instances, it does not need them anymore, so you can discard them
(unless you want to be able to roll back to a previous state and “replay” the data). This
can save a huge amount of space.
Online learning algorithms can also be used to train systems on huge datasets that
cannot fit in one machine’s main memory (this is called out-of-core learning). The
algorithm loads part of the data, runs a training step on that data, and repeats the
process until it has run on all of the data (see Figure 1-14).
Out-of-core learning is usually done offline (i.e., not on the live
system), so online learning can be a confusing name. Think of it as
incremental learning.

Figure 1-14. Using online learning to handle huge datasets
One important parameter of online learning systems is how fast they should adapt to
changing data: this is called the learning rate. If you set a high learning rate, then your
system will rapidly adapt to new data, but it will also tend to quickly forget the old
data (you don’t want a spam filter to flag only the latest kinds of spam it was shown).
Conversely, if you set a low learning rate, the system will have more inertia; that is, it
will learn more slowly, but it will also be less sensitive to noise in the new data or to
sequences of nonrepresentative data points (outliers).
A big challenge with online learning is that if bad data is fed to the system, the sys‐
tem’s performance will gradually decline. If we are talking about a live system, your
clients will notice. For example, bad data could come from a malfunctioning sensor
on a robot, or from someone spamming a search engine to try to rank high in search
Types of Machine Learning Systems

|

17

results. To reduce this risk, you need to monitor your system closely and promptly
switch learning off (and possibly revert to a previously working state) if you detect a
drop in performance. You may also want to monitor the input data and react to
abnormal data (e.g., using an anomaly detection algorithm).

Instance-Based Versus Model-Based Learning
One more way to categorize Machine Learning systems is by how they generalize.
Most Machine Learning tasks are about making predictions. This means that given a
number of training examples, the system needs to be able to generalize to examples it
has never seen before. Having a good performance measure on the training data is
good, but insufficient; the true goal is to perform well on new instances.
There are two main approaches to generalization: instance-based learning and
model-based learning.

Instance-based learning
Possibly the most trivial form of learning is simply to learn by heart. If you were to
create a spam filter this way, it would just flag all emails that are identical to emails
that have already been flagged by users—not the worst solution, but certainly not the
best.
Instead of just flagging emails that are identical to known spam emails, your spam
filter could be programmed to also flag emails that are very similar to known spam
emails. This requires a measure of similarity between two emails. A (very basic) simi‐
larity measure between two emails could be to count the number of words they have
in common. The system would flag an email as spam if it has many words in com‐
mon with a known spam email.
This is called instance-based learning: the system learns the examples by heart, then
generalizes to new cases by comparing them to the learned examples (or a subset of
them), using a similarity measure. For example, in Figure 1-15 the new instance
would be classified as a triangle because the majority of the most similar instances
belong to that class.

18

|

Chapter 1: The Machine Learning Landscape

Figure 1-15. Instance-based learning

Model-based learning
Another way to generalize from a set of examples is to build a model of these exam‐
ples, then use that model to make predictions. This is called model-based learning
(Figure 1-16).

Figure 1-16. Model-based learning
For example, suppose you want to know if money makes people happy, so you down‐
load the Better Life Index data from the OECD’s website as well as stats about GDP
per capita from the IMF’s website. Then you join the tables and sort by GDP per cap‐
ita. Table 1-1 shows an excerpt of what you get.

Types of Machine Learning Systems

|

19

Table 1-1. Does money make people happier?
Country
Hungary

GDP per capita (USD) Life satisfaction
12,240
4.9

Korea

27,195

5.8

France

37,675

6.5

Australia

50,962

7.3

United States 55,805

7.2

Let’s plot the data for a few random countries (Figure 1-17).

Figure 1-17. Do you see a trend here?
There does seem to be a trend here! Although the data is noisy (i.e., partly random), it
looks like life satisfaction goes up more or less linearly as the country’s GDP per cap‐
ita increases. So you decide to model life satisfaction as a linear function of GDP per
capita. This step is called model selection: you selected a linear model of life satisfac‐
tion with just one attribute, GDP per capita (Equation 1-1).
Equation 1-1. A simple linear model
life_satisfaction = θ0 + θ1 × GDP_per_capita

This model has two model parameters, θ0 and θ1.5 By tweaking these parameters, you
can make your model represent any linear function, as shown in Figure 1-18.

5 By convention, the Greek letter θ (theta) is frequently used to represent model parameters.

20

|

Chapter 1: The Machine Learning Landscape

Figure 1-18. A few possible linear models
Before you can use your model, you need to define the parameter values θ0 and θ1.
How can you know which values will make your model perform best? To answer this
question, you need to specify a performance measure. You can either define a utility
function (or fitness function) that measures how good your model is, or you can define
a cost function that measures how bad it is. For linear regression problems, people
typically use a cost function that measures the distance between the linear model’s
predictions and the training examples; the objective is to minimize this distance.
This is where the Linear Regression algorithm comes in: you feed it your training
examples and it finds the parameters that make the linear model fit best to your data.
This is called training the model. In our case the algorithm finds that the optimal
parameter values are θ0 = 4.85 and θ1 = 4.91 × 10–5.
Now the model fits the training data as closely as possible (for a linear model), as you
can see in Figure 1-19.

Figure 1-19. The linear model that fits the training data best

Types of Machine Learning Systems

|

21

You are finally ready to run the model to make predictions. For example, say you
want to know how happy Cypriots are, and the OECD data does not have the answer.
Fortunately, you can use your model to make a good prediction: you look up Cyprus’s
GDP per capita, find $22,587, and then apply your model and find that life satisfac‐
tion is likely to be somewhere around 4.85 + 22,587 × 4.91 × 10-5 = 5.96.
To whet your appetite, Example 1-1 shows the Python code that loads the data, pre‐
pares it,6 creates a scatterplot for visualization, and then trains a linear model and
makes a prediction.7
Example 1-1. Training and running a linear model using Scikit-Learn
import
import
import
import

matplotlib.pyplot as plt
numpy as np
pandas as pd
sklearn.linear_model

# Load the data
oecd_bli = pd.read_csv("oecd_bli_2015.csv", thousands=',')
gdp_per_capita = pd.read_csv("gdp_per_capita.csv",thousands=',',delimiter='\t',
encoding='latin1', na_values="n/a")
# Prepare the data
country_stats = prepare_country_stats(oecd_bli, gdp_per_capita)
X = np.c_[country_stats["GDP per capita"]]
y = np.c_[country_stats["Life satisfaction"]]
# Visualize the data
country_stats.plot(kind='scatter', x="GDP per capita", y='Life satisfaction')
plt.show()
# Select a linear model
model = sklearn.linear_model.LinearRegression()
# Train the model
model.fit(X, y)
# Make a prediction for Cyprus
X_new = [[22587]] # Cyprus' GDP per capita
print(model.predict(X_new)) # outputs [[ 5.96242338]]

6 The prepare_country_stats() function’s definition is not shown here (see this chapter’s Jupyter notebook if

you want all the gory details). It’s just boring Pandas code that joins the life satisfaction data from the OECD
with the GDP per capita data from the IMF.

7 It’s okay if you don’t understand all the code yet; we will present Scikit-Learn in the following chapters.

22

| Chapter 1: The Machine Learning Landscape

If you had used an instance-based learning algorithm instead, you
would have found that Slovenia has the closest GDP per capita to
that of Cyprus ($20,732), and since the OECD data tells us that
Slovenians’ life satisfaction is 5.7, you would have predicted a life
satisfaction of 5.7 for Cyprus. If you zoom out a bit and look at the
two next closest countries, you will find Portugal and Spain with
life satisfactions of 5.1 and 6.5, respectively. Averaging these three
values, you get 5.77, which is pretty close to your model-based pre‐
diction. This simple algorithm is called k-Nearest Neighbors regres‐
sion (in this example, k = 3).
Replacing the Linear Regression model with k-Nearest Neighbors
regression in the previous code is as simple as replacing these two
lines:
import sklearn.linear_model
model = sklearn.linear_model.LinearRegression()

with these two:
import sklearn.neighbors
model = sklearn.neighbors.KNeighborsRegressor(n_neighbors=3)

If all went well, your model will make good predictions. If not, you may need to use
more attributes (employment rate, health, air pollution, etc.), get more or better qual‐
ity training data, or perhaps select a more powerful model (e.g., a Polynomial Regres‐
sion model).
In summary:
• You studied the data.
• You selected a model.
• You trained it on the training data (i.e., the learning algorithm searched for the
model parameter values that minimize a cost function).
• Finally, you applied the model to make predictions on new cases (this is called
inference), hoping that this model will generalize well.
This is what a typical Machine Learning project looks like. In Chapter 2 you will
experience this first-hand by going through an end-to-end project.
We have covered a lot of ground so far: you now know what Machine Learning is
really about, why it is useful, what some of the most common categories of ML sys‐
tems are, and what a typical project workflow looks like. Now let’s look at what can go
wrong in learning and prevent you from making accurate predictions.

Types of Machine Learning Systems

|

23

Main Challenges of Machine Learning
In short, since your main task is to select a learning algorithm and train it on some
data, the two things that can go wrong are “bad algorithm” and “bad data.” Let’s start
with examples of bad data.

Insufficient Quantity of Training Data
For a toddler to learn what an apple is, all it takes is for you to point to an apple and
say “apple” (possibly repeating this procedure a few times). Now the child is able to
recognize apples in all sorts of colors and shapes. Genius.
Machine Learning is not quite there yet; it takes a lot of data for most Machine Learn‐
ing algorithms to work properly. Even for very simple problems you typically need
thousands of examples, and for complex problems such as image or speech recogni‐
tion you may need millions of examples (unless you can reuse parts of an existing
model).

24

|

Chapter 1: The Machine Learning Landscape

The Unreasonable Effectiveness of Data
In a famous paper published in 2001, Microsoft researchers Michele Banko and Eric
Brill showed that very different Machine Learning algorithms, including fairly simple
ones, performed almost identically well on a complex problem of natural language
disambiguation8 once they were given enough data (as you can see in Figure 1-20).

Figure 1-20. The importance of data versus algorithms9
As the authors put it: “these results suggest that we may want to reconsider the tradeoff between spending time and money on algorithm development versus spending it
on corpus development.”
The idea that data matters more than algorithms for complex problems was further
popularized by Peter Norvig et al. in a paper titled “The Unreasonable Effectiveness
of Data” published in 2009.10 It should be noted, however, that small- and mediumsized datasets are still very common, and it is not always easy or cheap to get extra
training data, so don’t abandon algorithms just yet.

8 For example, knowing whether to write “to,” “two,” or “too” depending on the context.
9 Figure reproduced with permission from Banko and Brill (2001), “Learning Curves for Confusion Set Disam‐

biguation.”

10 “The Unreasonable Effectiveness of Data,” Peter Norvig et al. (2009).

Main Challenges of Machine Learning

|

25

Nonrepresentative Training Data
In order to generalize well, it is crucial that your training data be representative of the
new cases you want to generalize to. This is true whether you use instance-based
learning or model-based learning.
For example, the set of countries we used earlier for training the linear model was not
perfectly representative; a few countries were missing. Figure 1-21 shows what the
data looks like when you add the missing countries.

Figure 1-21. A more representative training sample
If you train a linear model on this data, you get the solid line, while the old model is
represented by the dotted line. As you can see, not only does adding a few missing
countries significantly alter the model, but it makes it clear that such a simple linear
model is probably never going to work well. It seems that very rich countries are not
happier than moderately rich countries (in fact they seem unhappier), and conversely
some poor countries seem happier than many rich countries.
By using a nonrepresentative training set, we trained a model that is unlikely to make
accurate predictions, especially for very poor and very rich countries.
It is crucial to use a training set that is representative of the cases you want to general‐
ize to. This is often harder than it sounds: if the sample is too small, you will have
sampling noise (i.e., nonrepresentative data as a result of chance), but even very large
samples can be nonrepresentative if the sampling method is flawed. This is called
sampling bias.

A Famous Example of Sampling Bias
Perhaps the most famous example of sampling bias happened during the US presi‐
dential election in 1936, which pitted Landon against Roosevelt: the Literary Digest
conducted a very large poll, sending mail to about 10 million people. It got 2.4 million
answers, and predicted with high confidence that Landon would get 57% of the votes.
26

|

Chapter 1: The Machine Learning Landscape

Instead, Roosevelt won with 62% of the votes. The flaw was in the Literary Digest’s
sampling method:
• First, to obtain the addresses to send the polls to, the Literary Digest used tele‐
phone directories, lists of magazine subscribers, club membership lists, and the
like. All of these lists tend to favor wealthier people, who are more likely to vote
Republican (hence Landon).
• Second, less than 25% of the people who received the poll answered. Again, this
introduces a sampling bias, by ruling out people who don’t care much about poli‐
tics, people who don’t like the Literary Digest, and other key groups. This is a spe‐
cial type of sampling bias called nonresponse bias.
Here is another example: say you want to build a system to recognize funk music vid‐
eos. One way to build your training set is to search “funk music” on YouTube and use
the resulting videos. But this assumes that YouTube’s search engine returns a set of
videos that are representative of all the funk music videos on YouTube. In reality, the
search results are likely to be biased toward popular artists (and if you live in Brazil
you will get a lot of “funk carioca” videos, which sound nothing like James Brown).
On the other hand, how else can you get a large training set?

Poor-Quality Data
Obviously, if your training data is full of errors, outliers, and noise (e.g., due to poorquality measurements), it will make it harder for the system to detect the underlying
patterns, so your system is less likely to perform well. It is often well worth the effort
to spend time cleaning up your training data. The truth is, most data scientists spend
a significant part of their time doing just that. For example:
• If some instances are clearly outliers, it may help to simply discard them or try to
fix the errors manually.
• If some instances are missing a few features (e.g., 5% of your customers did not
specify their age), you must decide whether you want to ignore this attribute alto‐
gether, ignore these instances, fill in the missing values (e.g., with the median
age), or train one model with the feature and one model without it, and so on.

Irrelevant Features
As the saying goes: garbage in, garbage out. Your system will only be capable of learn‐
ing if the training data contains enough relevant features and not too many irrelevant
ones. A critical part of the success of a Machine Learning project is coming up with a
good set of features to train on. This process, called feature engineering, involves:

Main Challenges of Machine Learning

|

27

• Feature selection: selecting the most useful features to train on among existing
features.
• Feature extraction: combining existing features to produce a more useful one (as
we saw earlier, dimensionality reduction algorithms can help).
• Creating new features by gathering new data.
Now that we have looked at many examples of bad data, let’s look at a couple of exam‐
ples of bad algorithms.

Overfitting the Training Data
Say you are visiting a foreign country and the taxi driver rips you off. You might be
tempted to say that all taxi drivers in that country are thieves. Overgeneralizing is
something that we humans do all too often, and unfortunately machines can fall into
the same trap if we are not careful. In Machine Learning this is called overfitting: it
means that the model performs well on the training data, but it does not generalize
well.
Figure 1-22 shows an example of a high-degree polynomial life satisfaction model
that strongly overfits the training data. Even though it performs much better on the
training data than the simple linear model, would you really trust its predictions?

Figure 1-22. Overfitting the training data
Complex models such as deep neural networks can detect subtle patterns in the data,
but if the training set is noisy, or if it is too small (which introduces sampling noise),
then the model is likely to detect patterns in the noise itself. Obviously these patterns
will not generalize to new instances. For example, say you feed your life satisfaction
model many more attributes, including uninformative ones such as the country’s
name. In that case, a complex model may detect patterns like the fact that all coun‐
tries in the training data with a w in their name have a life satisfaction greater than 7:
New Zealand (7.3), Norway (7.4), Sweden (7.2), and Switzerland (7.5). How confident

28

|

Chapter 1: The Machine Learning Landscape

are you that the W-satisfaction rule generalizes to Rwanda or Zimbabwe? Obviously
this pattern occurred in the training data by pure chance, but the model has no way
to tell whether a pattern is real or simply the result of noise in the data.
Overfitting happens when the model is too complex relative to the
amount and noisiness of the training data. The possible solutions
are:
• To simplify the model by selecting one with fewer parameters
(e.g., a linear model rather than a high-degree polynomial
model), by reducing the number of attributes in the training
data or by constraining the model
• To gather more training data
• To reduce the noise in the training data (e.g., fix data errors
and remove outliers)

Constraining a model to make it simpler and reduce the risk of overfitting is called
regularization. For example, the linear model we defined earlier has two parameters,
θ0 and θ1. This gives the learning algorithm two degrees of freedom to adapt the model
to the training data: it can tweak both the height (θ0) and the slope (θ1) of the line. If
we forced θ1 = 0, the algorithm would have only one degree of freedom and would
have a much harder time fitting the data properly: all it could do is move the line up
or down to get as close as possible to the training instances, so it would end up
around the mean. A very simple model indeed! If we allow the algorithm to modify θ1
but we force it to keep it small, then the learning algorithm will effectively have some‐
where in between one and two degrees of freedom. It will produce a simpler model
than with two degrees of freedom, but more complex than with just one. You want to
find the right balance between fitting the training data perfectly and keeping the
model simple enough to ensure that it will generalize well.
Figure 1-23 shows three models: the dotted line represents the original model that
was trained with a few countries missing, the dashed line is our second model trained
with all countries, and the solid line is a linear model trained with the same data as
the first model but with a regularization constraint. You can see that regularization
forced the model to have a smaller slope, which fits a bit less the training data that the
model was trained on, but actually allows it to generalize better to new examples.

Main Challenges of Machine Learning

|

29

Figure 1-23. Regularization reduces the risk of overfitting
The amount of regularization to apply during learning can be controlled by a hyper‐
parameter. A hyperparameter is a parameter of a learning algorithm (not of the
model). As such, it is not affected by the learning algorithm itself; it must be set prior
to training and remains constant during training. If you set the regularization hyper‐
parameter to a very large value, you will get an almost flat model (a slope close to
zero); the learning algorithm will almost certainly not overfit the training data, but it
will be less likely to find a good solution. Tuning hyperparameters is an important
part of building a Machine Learning system (you will see a detailed example in the
next chapter).

Underfitting the Training Data
As you might guess, underfitting is the opposite of overfitting: it occurs when your
model is too simple to learn the underlying structure of the data. For example, a lin‐
ear model of life satisfaction is prone to underfit; reality is just more complex than
the model, so its predictions are bound to be inaccurate, even on the training exam‐
ples.
The main options to fix this problem are:
• Selecting a more powerful model, with more parameters
• Feeding better features to the learning algorithm (feature engineering)
• Reducing the constraints on the model (e.g., reducing the regularization hyper‐
parameter)

Stepping Back
By now you already know a lot about Machine Learning. However, we went through
so many concepts that you may be feeling a little lost, so let’s step back and look at the
big picture:
30

|

Chapter 1: The Machine Learning Landscape

• Machine Learning is about making machines get better at some task by learning
from data, instead of having to explicitly code rules.
• There are many different types of ML systems: supervised or not, batch or online,
instance-based or model-based, and so on.
• In a ML project you gather data in a training set, and you feed the training set to
a learning algorithm. If the algorithm is model-based it tunes some parameters to
fit the model to the training set (i.e., to make good predictions on the training set
itself), and then hopefully it will be able to make good predictions on new cases
as well. If the algorithm is instance-based, it just learns the examples by heart and
generalizes to new instances by comparing them to the learned instances using a
similarity measure.
• The system will not perform well if your training set is too small, or if the data is
not representative, noisy, or polluted with irrelevant features (garbage in, garbage
out). Lastly, your model needs to be neither too simple (in which case it will
underfit) nor too complex (in which case it will overfit).
There’s just one last important topic to cover: once you have trained a model, you
don’t want to just “hope” it generalizes to new cases. You want to evaluate it, and finetune it if necessary. Let’s see how.

Testing and Validating
The only way to know how well a model will generalize to new cases is to actually try
it out on new cases. One way to do that is to put your model in production and moni‐
tor how well it performs. This works well, but if your model is horribly bad, your
users will complain—not the best idea.
A better option is to split your data into two sets: the training set and the test set. As
these names imply, you train your model using the training set, and you test it using
the test set. The error rate on new cases is called the generalization error (or out-ofsample error), and by evaluating your model on the test set, you get an estimate of this
error. This value tells you how well your model will perform on instances it has never
seen before.
If the training error is low (i.e., your model makes few mistakes on the training set)
but the generalization error is high, it means that your model is overfitting the train‐
ing data.
It is common to use 80% of the data for training and hold out 20%
for testing. However, this depends on the size of the dataset: if it
contains 10 million instances, then holding out 1% means your test
set will contain 100,000 instances: that’s probably more than
enough to get a good estimate of the generalization error.
Testing and Validating

|

31

Hyperparameter Tuning and Model Selection
So evaluating a model is simple enough: just use a test set. Now suppose you are hesi‐
tating between two models (say a linear model and a polynomial model): how can
you decide? One option is to train both and compare how well they generalize using
the test set.
Now suppose that the linear model generalizes better, but you want to apply some
regularization to avoid overfitting. The question is: how do you choose the value of
the regularization hyperparameter? One option is to train 100 different models using
100 different values for this hyperparameter. Suppose you find the best hyperparame‐
ter value that produces a model with the lowest generalization error, say just 5% error.
So you launch this model into production, but unfortunately it does not perform as
well as expected and produces 15% errors. What just happened?
The problem is that you measured the generalization error multiple times on the test
set, and you adapted the model and hyperparameters to produce the best model for
that particular set. This means that the model is unlikely to perform as well on new
data.
A common solution to this problem is called holdout validation: you simply hold out
part of the training set to evaluate several candidate models and select the best one.
The new heldout set is called the validation set (or sometimes the development set, or
dev set). More specifically, you train multiple models with various hyperparameters
on the reduced training set (i.e., the full training set minus the validation set), and
you select the model that performs best on the validation set. After this holdout vali‐
dation process, you train the best model on the full training set (including the valida‐
tion set), and this gives you the final model. Lastly, you evaluate this final model on
the test set to get an estimate of the generalization error.
This solution usually works quite well. However, if the validation set is too small, then
model evaluations will be imprecise: you may end up selecting a suboptimal model by
mistake. Conversely, if the validation set is too large, then the remaining training set
will be much smaller than the full training set. Why is this bad? Well, since the final
model will be trained on the full training set, it is not ideal to compare candidate
models trained on a much smaller training set. It would be like selecting the fastest
sprinter to participate in a marathon. One way to solve this problem is to perform
repeated cross-validation, using many small validation sets. Each model is evaluated
once per validation set, after it is trained on the rest of the data. By averaging out all
the evaluations of a model, we get a much more accurate measure of its performance.
However, there is a drawback: the training time is multiplied by the number of valida‐
tion sets.

32

|

Chapter 1: The Machine Learning Landscape

Data Mismatch
In some cases, it is easy to get a large amount of data for training, but it is not per‐
fectly representative of the data that will be used in production. For example, suppose
you want to create a mobile app to take pictures of flowers and automatically deter‐
mine their species. You can easily download millions of pictures of flowers on the
web, but they won’t be perfectly representative of the pictures that will actually be
taken using the app on a mobile device. Perhaps you only have 10,000 representative
pictures (i.e., actually taken with the app). In this case, the most important rule to
remember is that the validation set and the test must be as representative as possible
of the data you expect to use in production, so they should be composed exclusively
of representative pictures: you can shuffle them and put half in the validation set, and
half in the test set (making sure that no duplicates or near-duplicates end up in both
sets). After training your model on the web pictures, if you observe that the perfor‐
mance of your model on the validation set is disappointing, you will not know
whether this is because your model has overfit the training set, or whether this is just
due to the mismatch between the web pictures and the mobile app pictures. One sol‐
ution is to hold out part of the training pictures (from the web) in yet another set that
Andrew Ng calls the train-dev set. After the model is trained (on the training set, not
on the train-dev set), you can evaluate it on the train-dev set: if it performs well, then
the model is not overfitting the training set, so if performs poorly on the validation
set, the problem must come from the data mismatch. You can try to tackle this prob‐
lem by preprocessing the web images to make them look more like the pictures that
will be taken by the mobile app, and then retraining the model. Conversely, if the
model performs poorly on the train-dev set, then the model must have overfit the
training set, so you should try to simplify or regularize the model, get more training
data and clean up the training data, as discussed earlier.

No Free Lunch Theorem
A model is a simplified version of the observations. The simplifications are meant to
discard the superfluous details that are unlikely to generalize to new instances. How‐
ever, to decide what data to discard and what data to keep, you must make assump‐
tions. For example, a linear model makes the assumption that the data is
fundamentally linear and that the distance between the instances and the straight line
is just noise, which can safely be ignored.
In a famous 1996 paper,11 David Wolpert demonstrated that if you make absolutely
no assumption about the data, then there is no reason to prefer one model over any
other. This is called the No Free Lunch (NFL) theorem. For some datasets the best

11 “The Lack of A Priori Distinctions Between Learning Algorithms,” D. Wolpert (1996).

Testing and Validating

|

33

model is a linear model, while for other datasets it is a neural network. There is no
model that is a priori guaranteed to work better (hence the name of the theorem). The
only way to know for sure which model is best is to evaluate them all. Since this is not
possible, in practice you make some reasonable assumptions about the data and you
evaluate only a few reasonable models. For example, for simple tasks you may evalu‐
ate linear models with various levels of regularization, and for a complex problem you
may evaluate various neural networks.

Exercises
In this chapter we have covered some of the most important concepts in Machine
Learning. In the next chapters we will dive deeper and write more code, but before we
do, make sure you know how to answer the following questions:
1. How would you define Machine Learning?
2. Can you name four types of problems where it shines?
3. What is a labeled training set?
4. What are the two most common supervised tasks?
5. Can you name four common unsupervised tasks?
6. What type of Machine Learning algorithm would you use to allow a robot to
walk in various unknown terrains?
7. What type of algorithm would you use to segment your customers into multiple
groups?
8. Would you frame the problem of spam detection as a supervised learning prob‐
lem or an unsupervised learning problem?
9. What is an online learning system?
10. What is out-of-core learning?
11. What type of learning algorithm relies on a similarity measure to make predic‐
tions?
12. What is the difference between a model parameter and a learning algorithm’s
hyperparameter?
13. What do model-based learning algorithms search for? What is the most common
strategy they use to succeed? How do they make predictions?
14. Can you name four of the main challenges in Machine Learning?
15. If your model performs great on the training data but generalizes poorly to new
instances, what is happening? Can you name three possible solutions?
16. What is a test set and why would you want to use it?
34

|

Chapter 1: The Machine Learning Landscape

17. What is the purpose of a validation set?
18. What can go wrong if you tune hyperparameters using the test set?
19. What is repeated cross-validation and why would you prefer it to using a single
validation set?
Solutions to these exercises are available in ???.

Exercises

|

35

CHAPTER 2

End-to-End Machine Learning Project

With Early Release ebooks, you get books in their earliest form—
the author’s raw and unedited content as he or she writes—so you
can take advantage of these technologies long before the official
release of these titles. The following will be Chapter 2 in the final
release of the book.

In this chapter, you will go through an example project end to end, pretending to be a
recently hired data scientist in a real estate company.1 Here are the main steps you will
go through:
1. Look at the big picture.
2. Get the data.
3. Discover and visualize the data to gain insights.
4. Prepare the data for Machine Learning algorithms.
5. Select a model and train it.
6. Fine-tune your model.
7. Present your solution.
8. Launch, monitor, and maintain your system.

1 The example project is completely fictitious; the goal is just to illustrate the main steps of a Machine Learning

project, not to learn anything about the real estate business.

37

Working with Real Data
When you are learning about Machine Learning it is best to actually experiment with
real-world data, not just artificial datasets. Fortunately, there are thousands of open
datasets to choose from, ranging across all sorts of domains. Here are a few places
you can look to get data:
• Popular open data repositories:
— UC Irvine Machine Learning Repository
— Kaggle datasets
— Amazon’s AWS datasets
• Meta portals (they list open data repositories):
— http://dataportals.org/
— http://opendatamonitor.eu/
— http://quandl.com/
• Other pages listing many popular open data repositories:
— Wikipedia’s list of Machine Learning datasets
— Quora.com question
— Datasets subreddit
In this chapter we chose the California Housing Prices dataset from the StatLib repos‐
itory2 (see Figure 2-1). This dataset was based on data from the 1990 California cen‐
sus. It is not exactly recent (you could still afford a nice house in the Bay Area at the
time), but it has many qualities for learning, so we will pretend it is recent data. We
also added a categorical attribute and removed a few features for teaching purposes.

2 The original dataset appeared in R. Kelley Pace and Ronald Barry, “Sparse Spatial Autoregressions,” Statistics

& Probability Letters 33, no. 3 (1997): 291–297.

38

|

Chapter 2: End-to-End Machine Learning Project

Figure 2-1. California housing prices

Look at the Big Picture
Welcome to Machine Learning Housing Corporation! The first task you are asked to
perform is to build a model of housing prices in California using the California cen‐
sus data. This data has metrics such as the population, median income, median hous‐
ing price, and so on for each block group in California. Block groups are the smallest
geographical unit for which the US Census Bureau publishes sample data (a block
group typically has a population of 600 to 3,000 people). We will just call them “dis‐
tricts” for short.
Your model should learn from this data and be able to predict the median housing
price in any district, given all the other metrics.
Since you are a well-organized data scientist, the first thing you do
is to pull out your Machine Learning project checklist. You can
start with the one in ???; it should work reasonably well for most
Machine Learning projects but make sure to adapt it to your needs.
In this chapter we will go through many checklist items, but we will
also skip a few, either because they are self-explanatory or because
they will be discussed in later chapters.

Frame the Problem
The first question to ask your boss is what exactly is the business objective; building a
model is probably not the end goal. How does the company expect to use and benefit

Look at the Big Picture

|

39

from this model? This is important because it will determine how you frame the
problem, what algorithms you will select, what performance measure you will use to
evaluate your model, and how much effort you should spend tweaking it.
Your boss answers that your model’s output (a prediction of a district’s median hous‐
ing price) will be fed to another Machine Learning system (see Figure 2-2), along
with many other signals.3 This downstream system will determine whether it is worth
investing in a given area or not. Getting this right is critical, as it directly affects reve‐
nue.

Figure 2-2. A Machine Learning pipeline for real estate investments

Pipelines
A sequence of data processing components is called a data pipeline. Pipelines are very
common in Machine Learning systems, since there is a lot of data to manipulate and
many data transformations to apply.
Components typically run asynchronously. Each component pulls in a large amount
of data, processes it, and spits out the result in another data store, and then some time
later the next component in the pipeline pulls this data and spits out its own output,
and so on. Each component is fairly self-contained: the interface between components
is simply the data store. This makes the system quite simple to grasp (with the help of
a data flow graph), and different teams can focus on different components. Moreover,
if a component breaks down, the downstream components can often continue to run
normally (at least for a while) by just using the last output from the broken compo‐
nent. This makes the architecture quite robust.

3 A piece of information fed to a Machine Learning system is often called a signal in reference to Shannon’s

information theory: you want a high signal/noise ratio.

40

|

Chapter 2: End-to-End Machine Learning Project

On the other hand, a broken component can go unnoticed for some time if proper
monitoring is not implemented. The data gets stale and the overall system’s perfor‐
mance drops.

The next question to ask is what the current solution looks like (if any). It will often
give you a reference performance, as well as insights on how to solve the problem.
Your boss answers that the district housing prices are currently estimated manually
by experts: a team gathers up-to-date information about a district, and when they
cannot get the median housing price, they estimate it using complex rules.
This is costly and time-consuming, and their estimates are not great; in cases where
they manage to find out the actual median housing price, they often realize that their
estimates were off by more than 20%. This is why the company thinks that it would
be useful to train a model to predict a district’s median housing price given other data
about that district. The census data looks like a great dataset to exploit for this pur‐
pose, since it includes the median housing prices of thousands of districts, as well as
other data.
Okay, with all this information you are now ready to start designing your system.
First, you need to frame the problem: is it supervised, unsupervised, or Reinforce‐
ment Learning? Is it a classification task, a regression task, or something else? Should
you use batch learning or online learning techniques? Before you read on, pause and
try to answer these questions for yourself.
Have you found the answers? Let’s see: it is clearly a typical supervised learning task
since you are given labeled training examples (each instance comes with the expected
output, i.e., the district’s median housing price). Moreover, it is also a typical regres‐
sion task, since you are asked to predict a value. More specifically, this is a multiple
regression problem since the system will use multiple features to make a prediction (it
will use the district’s population, the median income, etc.). It is also a univariate
regression problem since we are only trying to predict a single value for each district.
If we were trying to predict multiple values per district, it would be a multivariate
regression problem. Finally, there is no continuous flow of data coming in the system,
there is no particular need to adjust to changing data rapidly, and the data is small
enough to fit in memory, so plain batch learning should do just fine.
If the data was huge, you could either split your batch learning
work across multiple servers (using the MapReduce technique), or
you could use an online learning technique instead.

Look at the Big Picture

|

41

Select a Performance Measure
Your next step is to select a performance measure. A typical performance measure for
regression problems is the Root Mean Square Error (RMSE). It gives an idea of how
much error the system typically makes in its predictions, with a higher weight for
large errors. Equation 2-1 shows the mathematical formula to compute the RMSE.
Equation 2-1. Root Mean Square Error (RMSE)
RMSE X, h =

42

|

1 m
hxi −yi
mi∑
=1

2

Chapter 2: End-to-End Machine Learning Project

Notations
This equation introduces several very common Machine Learning notations that we
will use throughout this book:
• m is the number of instances in the dataset you are measuring the RMSE on.
— For example, if you are evaluating the RMSE on a validation set of 2,000 dis‐
tricts, then m = 2,000.
• x(i) is a vector of all the feature values (excluding the label) of the ith instance in
the dataset, and y(i) is its label (the desired output value for that instance).
— For example, if the first district in the dataset is located at longitude –118.29°,
latitude 33.91°, and it has 1,416 inhabitants with a median income of $38,372,
and the median house value is $156,400 (ignoring the other features for now),
then:

x1

−118 . 29
33 . 91
=
1, 416
38, 372

and:
y 1 = 156, 400
• X is a matrix containing all the feature values (excluding labels) of all instances in
the dataset. There is one row per instance and the ith row is equal to the transpose
of x(i), noted (x(i))T.4
— For example, if the first district is as just described, then the matrix X looks
like this:

X=

x1

T

x2

T

=

⋮
x 1999

T

x 2000

T

−118 . 29 33 . 91 1, 416 38, 372
⋮
⋮
⋮
⋮

4 Recall that the transpose operator flips a column vector into a row vector (and vice versa).

Look at the Big Picture

|

43

• h is your system’s