It’s a major challenge to keep up with the continuous changes to the Keras/TensorFlow neural code library (and the PyTorch library too). I recently upgraded my Keras installation to version 2.6 and so I’m going through all my standard examples to bring them up to date with the inevitable changes.

I was using a new desktop machine and so I had to install TensorFlow 2.6 (which contains Keras 2.6). I ran into unexpected problems when the “wrapt” sub-library refused to build correctly (the installation process builds a .whl file for wrapt instead of using a pre-built .whl file). I found a hack online that suggested issuing the command SET WRAPT_INSTALL_EXTENSIONS=false, before the command pip install tensorflow, and that magically worked. Somehow.

One of the standard examples is the MNIST image dataset. There are 70,000 simple images (60,000 training images and 10,000 test images). Each image has 28×28 pixels and is a handwritten digit from ‘0’ to ‘9’. Each pixel value is between 0 and 255.

I used the built-in MNIST dataset from Keras but I could have loaded the raw MNIST data using np.loadtxt() or a similar function.

I used the Model() approach by defining separate layers and then passing the first and last layer to the Model() constructor. An alternative design is to use the Sequential() approach. I have no strong preference between Model() and Sequential() — it’s just syntax.

To save time, I only used 2 training epochs with a batch size of 100. In a non-demo scenario I’d use more epochs, but then have to watch for over-fitting.

After the model was trained, I set up a fake 28×28 image with one vertical bar, one horizontal bar, and one diagonal bar. The trained model predicted the fake image is a ‘5’ with pseudo-probability = 1.0000. Many machine learning systems aren’t very good at distinguishing between authentic and fake items. This has generated interest in ML systems that output a prediction plus a confidence score of some type.

Good fun!

Left: A clever pseudo Chanel bag made from a paper grocery bag and a chain from a hardware store. Center: A whimsical pseudo Louis Vuitton bag, complete with misspelling. Right: A serious attempt at a Coach bag, but I don’t think it will fool many people.

Code below.

# mnist_tfk.py
# MNIST using CNN 
# Keras 2.6.0 in TensorFlow 2.6.0 ("_tfk")
# Anaconda3-2020.02  Python 3.7.6  Windows 10

import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'  # suppress warn

import numpy as np
import tensorflow as tf
from tensorflow import keras as K
import matplotlib.pyplot as plt

def main():
  # 0. get started
  print("\nBegin MNIST using Keras %s " % K.__version__)
  np.random.seed(1)
  tf.random.set_seed(1)

  # 1. load data
  print("\nLoading train and test data ")
  (train_x, train_y), \
  (test_x, test_y) = K.datasets.mnist.load_data()
  train_x = train_x.reshape(60_000, 28, 28, 1)
  test_x = test_x.reshape(10_000, 28, 28, 1)
  train_x = train_x.astype(np.float32)
  test_x = test_x.astype(np.float32)
  train_x /= 255
  test_x /= 255
  train_y = K.utils.to_categorical(train_y, 10)
  test_y = K.utils.to_categorical(test_y, 10)

  # 2. define model
  print("\nCreating network with two Convolution, \
two Dropout, two Dense layers ")
  g_init = K.initializers.glorot_uniform(seed=1)
  opt = K.optimizers.Adam(learning_rate=0.01)

  x = K.layers.Input(shape=(28,28,1))
  con1 = K.layers.Conv2D(filters=32,
    kernel_size=(3,3), kernel_initializer=g_init,
    activation='relu', padding='valid')(x)
  con2 = K.layers.Conv2D(filters=64,
    kernel_size=(3,3), kernel_initializer=g_init,
    activation='relu', padding='valid')(con1)
  mp1 = K.layers.MaxPooling2D(pool_size=(2,2))(con2)
  do1 = K.layers.Dropout(0.25)(mp1)
  z = K.layers.Flatten()(do1)
  fc1 = K.layers.Dense(units=128,
    kernel_initializer=g_init, activation='relu')(z)
  do2 = K.layers.Dropout(0.5)(fc1)
  fc2 = K.layers.Dense(units=10,
    kernel_initializer=g_init, activation='softmax')(do2)

  model = K.models.Model(x, fc2)

  model.compile(loss='categorical_crossentropy',
    optimizer=opt, metrics=['accuracy'])
  
  # 3. train model
  bat_size= 100
  max_epochs = 2
  print("\nStarting training with batch size = %d " % bat_size)
  model.fit(train_x, train_y, batch_size=bat_size,
    epochs=max_epochs, verbose=1)
  print("Training finished ")

  # 4. evaluate model
  eval = model.evaluate(test_x, test_y, verbose=0)
  loss = eval[0]
  acc = eval[1] * 100
  print("\nTest data: loss = %0.4f \
 accuracy = %0.2f%%" % (loss, acc))

  # 5. save model
  print("\nSaving MNIST model to disk ")
  # mp = ".\\Models\\mnist_model.h5"
  # model.save(mp)

  # 6. use model
  print("\nMaking prediction for fake image: ")
  # np.set_printoptions(precision=4, suppress=True)
  np.set_printoptions(formatter={'float': '{: 0.4f}'.format})

  x = np.zeros(shape=(28,28), dtype=np.float32)
  for row in range(5,23):
    x[row][9] = 180  # vertical line
  for rc in range(9,19):
    x[rc][rc] = 250  # diagonal
  for col in range(5,15):  
    x[14][col] = 200  # horizontal

  plt.imshow(x, cmap=plt.get_cmap('gray_r'))
  plt.show()

  x = x.reshape(1, 28, 28, 1)
  pred_probs = model.predict(x)
  print("\nPrediction probabilities: ")
  print(pred_probs)
  
  pred_digit = np.argmax(pred_probs)
  print("\nPredicted digit: ")
  print(pred_digit)

  print("\nEnd MNIST demo ")

if __name__ == "__main__":
  main()