42.111. Neural Networks for Classification with TensorFlow#

42.111.1. Getting Started: Binary Classifier#

We will first practice building neural networks for binary classifier. In binary classification, we have two classes.

We will use a classical cancer dataset to predict if a given patient has a malignant or benign based on their medical information. We will get it from sklearn datasets. You can read more about the dataset.

The dataset contains two labels: malignant, benign.

42.111.1.1. Getting the data#

We will get the data from sklearn datasets.

import numpy as np
import pandas as pd
import sklearn
import seaborn as sns
import matplotlib.pyplot as plt

import tensorflow as tf
from tensorflow import keras

from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()

# the dataset contain the following features

list(data.feature_names)

# the dataset contain the following labels

data.target_names

# Getting features and labels 

X = data.data
y = data.target

# the features and labels are numpy array 

type(X)

# To quickly look in data we can get the dataframe from X

data_df = pd.DataFrame(X, columns=data.feature_names)

42.111.2. Taking a look in the data#

# Looking from the head 

data_df.head()

# Getting the basic information

data_df.info()

# Getting the basic stats

data_df.describe().transpose()

42.111.3. Preparing the Data#

The data from sklearn is reasonably cleaned. Let’s split the data into train and test sets, and we will follow with scaling the values to be between 0 and 1.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.2, shuffle=True, random_state=42)

After splitting the data into training and testing sets, let’s see the number of examples in each set.

print('The number of training samples: {}\nThe number of testing samples: {}'.format(X_train.shape[0], X_test.shape[0]))

# Scaling the features to be between 0 and 1.

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()

X_train_scaled = scaler.fit_transform(X_train)

Let’s also scale the test set. We do not fit the scaler on the test set. We only transform it.

X_test_scaled = scaler.transform(X_test)

We are now ready to create, compile and train the model.

42.111.4. Creating, Compiling and Training a Model#

In TensorFlow, creating a model is only putting together an empty graphs. We are going to use Sequential API to stack the layers, from the input to output.

In model compilaton, it’s where we specify the optimizer and loss function. Loss function is there for calculating the difference between the predictions and the actual output, and optimizer is there for reducing the loss.

Also, if we are interested in tracking other metrics during training, we can specify them in metric.

# Creating a model 

from keras.callbacks import EarlyStopping, ModelCheckpoint

# Getting the input shape

input_shape = X_train_scaled.shape[1:]

model_1 = tf.keras.models.Sequential([
                                      
              # The first layer has 30 neurons(or units)                    
              tf.keras.layers.Dense(units=30, input_shape=input_shape, activation='relu'),

              # The second layer has 25 neurons 

              tf.keras.layers.Dense(units=15, activation='relu'),

              # The third layer has 1 neuron and activation of sigmoid. 
              # Because of sigmoid, the output of this layer will be a value bwteen 0 and 1
              tf.keras.layers.Dense(1, activation='sigmoid')                     
]) 

# Compiling the model 

model_1.compile(optimizer='sgd',
              loss='binary_crossentropy',
              metrics='accuracy')

After the model is created and compiled, it’s time to teach it. It’s time to train it on the data.

# By setting validation_split=0.15, I am allocating 15% of the dataset to be used for evaluating the model during the training
# Model training returns model history(accuracy, loss, epochs...) 

history = model_1.fit(X_train_scaled, y_train, epochs=60, validation_split=0.15)

I trained for 60 epochs. That was quick.

Let’s visualize accuracy and loss to actually see how the model did. It is always easy to notice performance on graph than looking on models training progress above.

‼️ If you retrain again, it will continue where it left. So, for example, if you train for 30 epochs, and you rerun the cell, it will train for same more epochs again.

42.111.5. Visualizing the Results#

Visualizing the model results after training is always a good way to learn what you can do to improve the performance.

Let’s get a Pandas dataframe containing training loss and accuracy, and validation loss and accuracy.

# Getting the dataframe of loss and accuracies on both training and validation

loss_acc_metrics_df = pd.DataFrame(history.history)
loss_acc_metrics_df.plot(figsize=(10,5))

This is really impressive. Seems that for only 60 epochs, the training accuracy was up to 90% while validation accuracy was 84%.

This is not bad considering that we have only 455 training samples, and also 15% of such samples are allocated to the validation set. The validation accuracy can be increased by increaing the validation samples.

Let’s evaluate the model on the test set.

42.111.6. Evaluating the Model#

Quite often, you will want to test your model on the data that it never saw. This data is normally called test set and in more applied practice, you will only feed the test to the model after you have done your best to improve it.

Let’s now evaluate the model on the test set. One thing to note here is that the test set must be preprocessed the same way we preprocessed the training set. The training set was rescaled and that was applied to the test set.

If this is not obeyed, you would not know why you’re having poor results. Just look up on the next next cell how poor the accuracy will be if I evaluate the model on unscaled data when I trained it on scaled data.

# if you need a model trained, you can use this cell
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/best_model_1.h5"
model_path = get_file("best_model_1.h5", model_url)
model_1 = load_model(model_path)

# Evaluating a model on unseen data: test set

model_eval = model_1.evaluate(X_test_scaled, y_test)

# Printing the loss and accuracy 

print('Test loss: {}\nTest accuracy:{}'.format(model_eval[0],model_eval[1]))

# ‼DON'T DO THIS!! X_test is not scaled. The results will be awful

model_1.evaluate(X_test, y_test)

It’s very suprising how the model did on the test data. It achieved 93%.

Accuracy is one classification metric, but there are more metrics such as f1 score, recall, and precision. The easiest way to get these metrics is to use classification_report function provided by Scikit-Learn.

# Getting the prediction

predictions = model_1.predict(X_test_scaled)

predictions[:15]

If you look at the predictions above, they are probabilities (value between 0 and 1).

Round function will return the closest integer. For example, for a prediction of 0.3, it will be 0. If the prediction is 0.6, the closest integer is 1.

# Rounding the predictions to 0 and 1

predictions = tf.round(predictions)

 # Display the first 15 preds values 
 
predictions[:15]

Great, the predictions are now rounded up to either 0 or 1.

# Getting the confusion matrix

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(y_test, predictions)

# Plotting confusion matrix 

plt.figure(figsize=(6,6))
sns.heatmap(cm, square=True, annot=True, fmt='d', cbar=True,
                        xticklabels=['0: Malignant', '1:Benign'],
                        yticklabels=['0: Malignant', '1:Benign'])
plt.ylabel('Actual label')
plt.xlabel('Predicted label');

Let’s interpret the confusion matrix:

First off, the rows represent the actual classes and the columns represent the predicted classes.

# Classification report: F1 score, Recall, Precision

from sklearn.metrics import classification_report

print(classification_report(y_test, predictions))

Here are notes about these metrics: Accuracy is the ratio of the correct predicted samples over the total samples. Precision is the ratio of correct predicted positive samples over the total positive predictions. Recall is the ration of correct predicted positive samples over total positive samples. F1 score is the harmonic mean of precision and recall.

42.111.7. Going Beyond Binary Classifier to Multiclass Classifier: 10 Fashions Classifier#

So far, we have built a neural network for regression(in previous labs) and binary classification. And we have only been working with structured datasets(datasets in tabular format).

Can the same neural networks we used be able to recognize images? In this next practice, we will turn the page to image classification. We will build a neural network for recognizing 10 different fashions and along the way, we will learn other things such as stopping the training upon a given condition is met, and using TensorBoard to visualize model.

That is going to be cool! Let’s get started!

42.111.8. Getting the Fashion data#

Let’s get the dataset from Keras.

import tensorflow as tf

fashion_mnist = tf.keras.datasets.fashion_mnist

(fashion_train, fashion_train_label), (fashion_test, fashion_test_label) = fashion_mnist.load_data()

42.111.9. Looking in the Fashion Data#

As always, it is a best practice to peep into the images to see how they like.

Let’s display the pixels values of a given image, image, and its corresponding label.

index = 10

# Get the pixels

fashion_train[index]

# A list of label names

class_names = ['T-shirt/top', 'Trouser','Pullover','Dress','Coat','Sandal','Shirt','Sneaker','Bag','Ankle boot']

# Show the image

plt.imshow(fashion_train[index])

# Display the label

image_label = fashion_train_label[index]
print('This type of fashion is: {}({})'.format(class_names[image_label], image_label))

The fashions with the label 0 is T-shirt/top. Normally, the pixels of image range from 0 to 255. If you can look back where we displayed the pixels, you will see that they vary from 0 to 255.

We can also visualize some random images.

import random

plt.figure(figsize=(6,6))

for index in range(6):

  ax = plt.subplot(2,3, index+1)
  random_index = random.choice(range(len(fashion_train)))
  plt.imshow(fashion_train[random_index])
  plt.title(class_names[fashion_train_label[random_index]])

You can rerun the above cell to display different fashions.

Another important thing to look at when working with images is to see their size.

This is important because later when we will be creating a model, we have to specify the input shape and such shape is same as the shape of the images. Each image is 28*28, but let’s verify that.

# Getting the image shape

print('The shape of the whole training dataset:{}'.format(fashion_train[0].shape))
print('The shape of the first(and other)image:{}'.format(fashion_train[0].shape))

Now that we know the dataset that we are working with, let us do some few proprocessing before building a model.

42.111.10. Preparing the Data#

In many cases, real world images datasets are not that clean like fashion mnist.

You may have to correct images that were incorrectly labeled, or you have labels in texts that need to be converted to numbers(most machine learning models accept numeric input), or scale the pixels values.

The latter is what we are going to do. It is inarguable that scaling the images pixels to value between 0 and 1 increase the performance of the neural network, and hence the results. Let’s do it!!

As we have seen, the pixels range from 0 to 255. So we will divide both training and test set by 255.0.

# Scaling the image pixels to be between 0 and 1

fashion_train = fashion_train/255.0

fashion_test = fashion_test/255.0

We are now ready to build a neural network.

42.111.11. Creating, Compiling, and Training a Model#

There are few points to note before creating a model:

When working with images, the shape of the input images has to be correctly provided. This is a common error done by many people, including me (before I learned it).
This is a multiclass classification, which is different to the binary classifier we did earlier. The difference will be reflected in the choice of output activation function, number of output neurons, and the loss function.
That said, we will use softmax as activation in the last layer, 10 neurons or units because we have 10 fashions, and the loss will be SparseCategoricalCrossentropy because the labels are pure integer. If the labels were in one hot format, we would use CategoricalCrossentropy. Learn more about Keras losses . Documentation is always the top source when learning all the possibilities of any framework. And also, Keras doc is beautifuly organized. Not to mention that the keras API is also well designed as well.

Let’s now create a model.

# Creating a model

fashion_classifier = tf.keras.models.Sequential([
                                                 
        # Flattening layer will convert array of pixels into one dimensional column array                                        
       tf.keras.layers.Flatten(),
       tf.keras.layers.Dense(units=64, activation='relu'),
       tf.keras.layers.Dense(units=32, activation='relu'),
       tf.keras.layers.Dense(units=10, activation='softmax')                           

])

# Compiling a model: Specifying a loss and optimization function

fashion_classifier.compile(optimizer='adam',
                           loss='sparse_categorical_crossentropy',
                           metrics=['accuracy']
                           
)

Now that we built and compiled a model, we can train it.

In order to train a model, we must have an input data and output(labels). We train the model to get the relationship between the input and output. Such relationship is what we tend to call rules. So, in other words, we provide the data and the answers to a model to get the rules.

# Training a model
# Allocating 15% of training data to validation set

fashion_classifier.fit(fashion_train, fashion_train_label, epochs=20, validation_split=0.15)

This was fast. When using Google Colab, you can speed up the training by changing the runtime typeto GPU. You can head over Runtime in the menu bar

But also, training mnist for 20 epochs is not slow that we would need to activate GPU. We will take an advantage of GPU in later labs.

42.111.12. Visualizing the Model Results#

Let’s visualize the model results to see how training went.

# Getting the dataframe of loss and accuracies on both training and validation

loss_acc_metrics_df = pd.DataFrame(fashion_classifier.history.history)

# Plotting the loss and accuracy

loss_acc_metrics_df.plot(figsize=(10,5))

At the end of the training, the accuracy is about 92% while validation accuracy being 88% or so. That’s not bad considering that we built a simple model and trained for only 20 epochs.

Let’s see how the model performs on unseed data: test set.

42.111.13. Model Evaluation#

# if you need a model trained, you can use this cell
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/best_model_2.h5"
model_path = get_file("best_model_2.h5", model_url)
fashion_classifier = load_model(model_path)

# Evaluating the model on unseen data

eval = fashion_classifier.evaluate(fashion_test, fashion_test_label)

# Printing the loss and accuracy 

print('Test loss: {}\nTest accuracy:{}'.format(eval[0],eval[1]))

The fashion classifier that we built is 88% confident at recognizing unseen fashion.

We could also find other Classification metrics based on True/False positives & negatives such as precision, recall, but since we already saw how to compute them in the binary classifier, let’s see other interesting things: Controlling the training using callbacks and using TensorBoard.

42.111.14. Controlling Training with Callbacks#

We can use Callbacks functions to control the training.

Take an example: we can stop training when the model is lo longer showing significant improvements on validation set. Or we can terminate training when a certain condition is met.

42.111.14.1. Implementing Callbacks#

There are various functionalities available in Keras Callbacks.

Let’s start with how to use ModelCheckpoint to save the model when the performance on the validation set is best so far. By saving the best model on the validation set, we avoid things like overfitting which is a common issue in machine learning model training, neural network specifically. We also train for less time.

I will rebuild a same model again.

# Creating a same model as used before

def classifier():

  model = tf.keras.models.Sequential([
                                                 
        # Flattening layer will convert array of pixels into one dimensional column array                                        
       tf.keras.layers.Flatten(),
       tf.keras.layers.Dense(units=64, activation='relu'),
       tf.keras.layers.Dense(units=32, activation='relu'),
       tf.keras.layers.Dense(units=10, activation='softmax')                           

  ])

# Compiling a model: Specifying a loss and optimization function

  model.compile(optimizer='adam',
                           loss='sparse_categorical_crossentropy',
                           metrics=['accuracy']
                           
  )

  return model

# Defining callbacks 

from keras.callbacks import ModelCheckpoint

callbacks = ModelCheckpoint('fashion_classifier.h5', save_best_only=True)

The callbacks defined above is passed into the model fit.

# Controlling training with callbacks

# Get the model

fashion_classifier_2 = classifier()

fashion_classifier_2.fit(fashion_train, fashion_train_label, epochs=20, validation_split=0.15, callbacks=[callbacks])

# if you need a model trained, you can use this cell
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/fashion_classifier.h5"
model_path = get_file("fashion_classifier.h5", model_url)
fashion_classifier_21 = load_model(model_path)

# Evaluate the model on the test data
train_loss, train_accuracy = fashion_classifier_21.evaluate(fashion_train, fashion_train_label)

# Print the test loss and accuracy
print('Train loss:', train_loss)
print('Train accuracy:', train_accuracy)

Another easier way to control the model training is to use EarlyStopping.

By using early stopping, the training will be stopped when a monitored metric is no longer improving for a given number of consecutive epochs.

The metric to be monitored during the training is val_accuracy in our example, and it is assigned to monitor argument. The patience represent the number of consecutive epochs of which the training will stop when there is no significant improvements in val_accuracy.

from keras.callbacks import EarlyStopping

early_stop = EarlyStopping(monitor='val_accuracy', patience=4, restore_best_weights=True)

Let’s train for 100 epochs to be able to notice if the training stops as soon as there is no improvements in the validation set for 4 epochs in row.

# Stopping training early

# Getting the model

fashion_classifier_2 = classifier()

history = fashion_classifier_2.fit(fashion_train, fashion_train_label, epochs=100, validation_split=0.15, callbacks=[early_stop])

# if you need a model trained, you can use this cell
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/fashion_classifier_2.h5"
model_path = get_file("fashion_classifier_2.h5", model_url)
fashion_classifier_22 = load_model(model_path)

# Evaluate the model on the test data
train_loss, train_accuracy = fashion_classifier_22.evaluate(fashion_train, fashion_train_label)

# Print the test loss and accuracy
print('Train loss:', train_loss)
print('Train accuracy:', train_accuracy)

Early stopping can potentially save our time and resources.

One last thing before going to custom callbacks, we can combine both ModelCheckpoint(for saving the best model on validation set on the course of training) and EarlyStopping for saving us time.

# Combining Early stopping and Model Check point

# Getting the model 

fashion_classifier_2 = classifier()

history = fashion_classifier_2.fit(fashion_train, fashion_train_label, epochs=100, validation_split=0.15, callbacks=[callbacks, early_stop])

# if you need a model trained, you can use this cell
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/fashion_classifier_3.h5"
model_path = get_file("fashion_classifier_3.h5", model_url)
fashion_classifier_23 = load_model(model_path)

# Evaluate the model on the test data
train_loss, train_accuracy = fashion_classifier_23.evaluate(fashion_train, fashion_train_label)

# Print the test loss and accuracy
print('Train loss:', train_loss)
print('Train accuracy:', train_accuracy)

42.111.14.2. Custom Callback#

Keras offers various functions for implementing custom callbacks that are very handy when you want to control the model training with a little bit of customization.

You can do certain actions on almost every step of the training. Let’s stop the training when the accuracy is 95%.

# Custom callbacks

class callback(tf.keras.callbacks.Callback):

  def on_epoch_end(self, epoch, logs={}):

    if (logs.get('accuracy') > 0.95):

      print('\n Training is cancelled at an accuracy of 95%')
      self.model.stop_training = True

# Call callbacks

custom_callback = callback()

# Implementing custom ballback

# Getting the model 

fashion_classifier_2 = classifier()

history = fashion_classifier_2.fit(fashion_train, fashion_train_label, epochs=100, validation_split=0.15, callbacks=[custom_callback])

# if you need a model trained, you can use this cell
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/fashion_classifier_4.h5"
model_path = get_file("fashion_classifier_4.h5", model_url)
fashion_classifier_24 = load_model(model_path)

# Evaluate the model on the test data
train_loss, train_accuracy = fashion_classifier_24.evaluate(fashion_train, fashion_train_label)

# Print the test loss and accuracy
print('Train loss:', train_loss)
print('Train accuracy:', train_accuracy)

42.111.15. Using TensorBoard for Model Visualization#

Tensorboard is incredible tool used by many people (and not just only TensorFlow developers) to experiment with machine learning.

With TensorBoard, you can:

Track and visualize the loss and accuracy
Visualize the model graphs and operations
Display images and other types of data
View the histograms of weights and biases.

We first have to load the TensorBoard extension as follows.

# Load the Tensorboard notebook extension
# And import datetime

%load_ext tensorboard

And we clear all logs from all runs we did before.

!rm -rf ./logs/

Le’s get the model from the function classifier defined in previous cells.

# Getting the model 

fashion_classifier = classifier()

Let’s create a Keras callback.

# Create a callback

tfboard_callback = tf.keras.callbacks.TensorBoard(log_dir="logs")

Now, train the model and provide the tfboard_callback to callback argument.

fashion_classifier.fit(fashion_train, fashion_train_label, epochs=20, validation_split=0.15, callbacks=[tfboard_callback])

Now that the training is over, we can start the TensorBoard.

import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.utils import get_file
model_url = "https://static-1300131294.cos.ap-shanghai.myqcloud.com/data/deep-learning/nn/fashion_classifier_vis.h5"
model_path = get_file("fashion_classifier_vis.h5", model_url)
fashion_classifier_23 = load_model(model_path)

# Evaluate the model on the test data
train_loss, train_accuracy = fashion_classifier_23.evaluate(fashion_train, fashion_train_label)

# Print the test loss and accuracy
print('Train loss:', train_loss)
print('Train accuracy:', train_accuracy)

%tensorboard --logdir logs

As you can see, TensorBoard is very useful. The fact that you can use it to visualize the performance metrics, model graphs, and datasets as well.

42.111.16. Acknowledgments#

Thanks to Jean de Dieu Nyandwi for creating the open-source course machine learning complete . It inspires the majority of the content in this chapter.

Ocademy Open Machine Learning Book

Neural Networks for Classification with TensorFlow

Contents