This is the first part of a two parts tutorial. In this part we create a TensorFlow model and save the trained model. In the second part we will take our saved model upload it to Machine Learning Foundation and use it. Our model will be a three-layer convolutional neuronal network classifying cifar-10 images.
You have installed TensorFlow, best the GPU version, at least version 1.11.0. If you have installed both the CPU as well as GPU version, it can happen that Python loads the CPU version and the network is not executed on GPU. In that case uninstall the CPU version.
This requires that you also have installed one of the following Python versions: 3.4, 3.5 or 3.6. At the time this tutorial was written TensorFlow does not support Python 3.7.
You have a global SAP Cloud Platform (SCP) account and have created a Space that contains an instance of service ml-foundation including a service key.
We want to build an images classification model that is using the CIFAR-10 dataset. This dataset covers 60000 color images of 32x32 pixels, which have been categories in 10 classes. The classes are 'Airplane', 'Automobile', 'Bird', 'Cat', 'Deer', 'Dog', 'Frog', 'Horse', 'Ship' and 'Truck'. You can download the images from here: cifar-10-python.tar.gz. Extract the images to a new folder that cloud be called cifar10 blow your home directory. We want to use this folder to store all the files we create throughout the whole tutorial.
In this tutorial we will create four files separating the different tasks that need to be fulfilled to build, run and save a TensorFlow model:
As mentioned, we will store all our python files in the same folder we have stored the images in.
This first file, we want to call data_manager.py and store it in the same folder as the cifar10 images, will contain two classes that shall support us reading and converting the image data as well as providing them to the model.
First and of special interest CifarImageProvider. This class loads the images and labels from file system and converts them:
This is already a lot of code and most is just some utility functionality. But we should have a look at the load function. As the name already intends it reads the images from file system, but it does something important in addition, it converts the images. In the original format (NCHW) the three colors were separated, so we have three layers 32x32 each. This is transposed so we have a 32x32x3 tensor (NHWC) or each pixel has now all the color information. Finally, the integer value from 0 to 255 are converter into floats between 0 and 1 as deep networks tent to work better then. This is important to remember, as this is the format we must provide the image data to our model and this will be the format an image needs to have for inference.
This is done by the following line:
The second class, we create in data_manager.py, is just a small façade, so we can better deal with training and test data:
Having done that, it becomes time for a small test. We want to load the images and have a look at the statistic:
Open a console, navigate to your cifar10 folder and execute python cifar10.py command. After the program as finish you should see:
Number of images train: 50000
Number of labels train: 50000
Number of images test: 10000
Number of labels test: 10000
The next file, model_elements.py, will contain some helper functions to let us build up our model. This is quite useful as we want to build up a CNN having three identical layers, using batch normalization, average pooling and parameterized rectified linear unit as activation function. As the later one, at least till TensorFlow 1.11, is not part of TensorFlow, we must create it in on our own.
BYOM uses TensorFlow serving. This expects a model provided in a special format. This format can be created using a Saved Model Builder. TensorFlow Saved Model provides three types of APIs called Classify, Regress and Predict. The request implementation, which we will use in the second part, for the different APIs differ. PredictRequest on the one side expects values encoded via TensorProto. ClassificationRequest and RegressionRequest on the other side expect the data as Example. We just look at Predict, as it does not require to preprocess data during training and we get back all the information we need.
We start with creating tf_serving.py, which contains two helper functions supporting us in saving our trained model.
The first step we need to make is to create a folder to save our model in:
The saved model builder requires to create this folder by its own, so you can’t overwrite a model. To ensure we get always a new one we add a timestamp as last folder.
Next, we define the PREDICT API that we will use for inference.
The helper function has two parameter input_layer and prediction. The first one is a tensor that is the entry point in our TensorFlow graph. It describes how the model expect the input and where to inject the input into the graph. The second one describes the output of the model, so on the one hand how the output looks like and on the other hand it gives the node in our graph from which the output shall be taken. We will see that when we build your model.
The API definition itself is called signature. As we build a prediction API our signature has one input and one output parameter. The names, 'images' and 'scores', are given by us. We cloud have chosen whatever we want.
Now we have all parts we need to build our model. For this we create cifar10.py. The script will be described in four blocks. Giving the change to explain each.
As mentioned we build a CNN for image classification. This model shall have three convolutional layers and one fully connected one. In the first block we start with declaring some hyperparameter:
Out of the hyperparameter the following shall be explained:
In addition, we define the path below the home directory to the folder the model shall be saved in.
In the next block, we create two functions to inform us about the progress during the training. First one to determine the test accuracy and second one to have a progress indicator.
Now we can build up the model:
Nothing too exciting if you are familiar with TensorFlow. Only three things shall be mentioned here:
Finally, we have code to train and save the trained model.
We want to have a look at three statements:
Now we have all the python code is there to train our model. Your folder should now contain 12 files:
Let’s run cifar10.py. We can do that e.g. by opening a console, navigate to the folder you create the python files in and call e.g. python cifar10.py. But be aware that this will run a while. If you are in a hurry or you are not using GPU you should halve the values of the hyperparameter C1, C2 and C3 and may reduce the number of epochs. If the python program has finished, you may see something like this:
Both need to be combined in a .zip file, which we want to call cifar10.zip.
Prerequisites:
You have installed TensorFlow, best the GPU version, at least version 1.11.0. If you have installed both the CPU as well as GPU version, it can happen that Python loads the CPU version and the network is not executed on GPU. In that case uninstall the CPU version.
This requires that you also have installed one of the following Python versions: 3.4, 3.5 or 3.6. At the time this tutorial was written TensorFlow does not support Python 3.7.
You have a global SAP Cloud Platform (SCP) account and have created a Space that contains an instance of service ml-foundation including a service key.
Target Reader Group
This tutorial shall give developers some insides how to use TensorFlow serving API and how to deploy and run an own model on SCP.
Even so a TensorFlow model is build up from scratch you won’t learn anything about neuronal networks or why certain values for hyperparameter where chosen.
Build up a TensorFlow Model
Get the Data
We want to build an images classification model that is using the CIFAR-10 dataset. This dataset covers 60000 color images of 32x32 pixels, which have been categories in 10 classes. The classes are 'Airplane', 'Automobile', 'Bird', 'Cat', 'Deer', 'Dog', 'Frog', 'Horse', 'Ship' and 'Truck'. You can download the images from here: cifar-10-python.tar.gz. Extract the images to a new folder that cloud be called cifar10 blow your home directory. We want to use this folder to store all the files we create throughout the whole tutorial.
Create the Python Programs
In this tutorial we will create four files separating the different tasks that need to be fulfilled to build, run and save a TensorFlow model:
- Read and manage the images.
- Provide some helper functions to define the model.
- Provide some helper functions to save the model.
- Describe, train and save the model.
As mentioned, we will store all our python files in the same folder we have stored the images in.
Read Images
This first file, we want to call data_manager.py and store it in the same folder as the cifar10 images, will contain two classes that shall support us reading and converting the image data as well as providing them to the model.
First and of special interest CifarImageProvider. This class loads the images and labels from file system and converts them:
import numpy as np
import time
import pickle
class CifarImageProvider(object):
def __init__(self, source_files, batch_size 😞
self._source = source_files
self._i = 0
self._batch_size = batch_size
self.images = None
self.labels = None
self._start_time = 0.0
def _unpickle(self, file):
with open(file, 'rb') as fo:
directory = pickle.load(fo, encoding="bytes")
return directory
def _one_hot(self, vec, vals=10):
n = len(vec)
out = np.zeros((n, vals))
out[range(n), vec] = 1
return out
def load(self):
data = [self._unpickle(f) for f in self._source]
images = np.vstack([d[b"data"] for d in data])
n = len(images)
self.images = images.reshape(n,3,32,32).transpose(0,2,3,1).astype(float)/255
self.labels = self._one_hot(np.hstack([d[b"labels"] for d in data]), 10)
return self
def next_batch(self):
x,y = self.images[self._i:self._i + self._batch_size], self.labels[self._i:self._i + self._batch_size]
self._i = (self._i + self._batch_size) % len(self.images)
return x,y
def new_epoch(self):
self._start_time = time.time()
self._i = 0
def end_epoch(self):
return time.time() - self._start_time
This is already a lot of code and most is just some utility functionality. But we should have a look at the load function. As the name already intends it reads the images from file system, but it does something important in addition, it converts the images. In the original format (NCHW) the three colors were separated, so we have three layers 32x32 each. This is transposed so we have a 32x32x3 tensor (NHWC) or each pixel has now all the color information. Finally, the integer value from 0 to 255 are converter into floats between 0 and 1 as deep networks tent to work better then. This is important to remember, as this is the format we must provide the image data to our model and this will be the format an image needs to have for inference.
This is done by the following line:
self.images = images.reshape(n,3,32,32).transpose(0,2,3,1).astype(float)/255 The second class, we create in data_manager.py, is just a small façade, so we can better deal with training and test data:
class CifarImageManager(object):
BATCH_SIZE = 100
def __init__(self, batch_size=BATCH_SIZE 😞
self.train = CifarImageProvider(["data_batch_{}".format(i) for i in range(1,6)], batch_size=batch_size).load()
self.test = CifarImageProvider(["test_batch"], batch_size=batch_size).load()
def print_statistics(self):
print("Number of images train: {}".format(len(self.train.images)))
print("Number of labels train: {}".format(len(self.train.labels)))
print("Number of images test: {}".format(len(self.test.images)))
print("Number of labels test: {}".format(len(self.test.labels)))
def train_len(self):
return len(self.train.images)
def test_len(self):
return len(self.test.images)
Having done that, it becomes time for a small test. We want to load the images and have a look at the statistic:
imageManager = CifarImageManager()
imageManager.print_statistics()
Open a console, navigate to your cifar10 folder and execute python cifar10.py command. After the program as finish you should see:
Number of images train: 50000
Number of labels train: 50000
Number of images test: 10000
Number of labels test: 10000
Creating Model Elements
The next file, model_elements.py, will contain some helper functions to let us build up our model. This is quite useful as we want to build up a CNN having three identical layers, using batch normalization, average pooling and parameterized rectified linear unit as activation function. As the later one, at least till TensorFlow 1.11, is not part of TensorFlow, we must create it in on our own.
import tensorflow as tf
def weight_variable(shape):
return tf.Variable(tf.truncated_normal(shape, stddev = 0.1))
def bias_variable(shape):
return tf.Variable(tf.constant(0.1, shape = shape))
def pool_2x2(x):
return tf.nn.avg_pool(x, ksize = [1, 2, 2, 1], strides = [1, 2, 2, 1], padding = 'SAME')
def conv_layer_bn(input_layer, shape, training, activation = tf.nn.relu6):
'''
Convolutional layer with batch normalization and average pooling
Args:
input_layer: input tensor
shape: target shape respectively filter. It has for dimensions:
filter_height, filter_width, in_channels, out_channels
training: Indicator if the model is used in training or
inference, needed for batch normalization
activation: Activation function to be used
'''
with tf.name_scope("CNN-layer"):
W = weight_variable(shape)
b = bias_variable([shape[3]])
cnn = tf.nn.conv2d(input_layer, W, strides = [1, 1, 1, 1], padding = 'SAME') + b
bn = tf.layers.batch_normalization(cnn, training = training, momentum = 0.9)
return pool_2x2(activation(bn))
def full_layer(input_layer, size):
'''
Full connected layer. Main purpose here is to convert the output of the CNN layer,
the so-called feature vector, into the classes
Args:
input_layer: input tensor
size: number of output neurons
'''
in_size = int(input_layer.get_shape()[1])
W = weight_variable([in_size, size])
b = bias_variable([size])
return tf.matmul(input_layer, W) + b
def lrelu6(x, alpha = 0.2, name = "LeakyReLU"):
'''
Leaky Rectified Linear Unit with fixed saturation of 6. This activation
function is not part of tensorflow as of now.
Args:
x: tensor
alpha: slope of the negative part
'''
return tf.maximum(alpha * x, tf.minimum(6.0, x), name = name)
def prelu6(x, name = "ParameterizedReLU"):
'''
Parametric Rectified Linear Unit with fixed saturation of 6.
This activation function is not part of tensorflow as of now.
prelu learns the slope of the negative part
Args:
x: tensor
'''
with tf.variable_scope(name_or_scope = None, default_name = "prelu"):
alpha = tf.get_variable("prelu", shape = x.get_shape()[-1],
dtype = x.dtype, initializer = tf.constant_initializer(0.1))
return lrelu6(x, alpha, name = name)
Preparing Model Saving
BYOM uses TensorFlow serving. This expects a model provided in a special format. This format can be created using a Saved Model Builder. TensorFlow Saved Model provides three types of APIs called Classify, Regress and Predict. The request implementation, which we will use in the second part, for the different APIs differ. PredictRequest on the one side expects values encoded via TensorProto. ClassificationRequest and RegressionRequest on the other side expect the data as Example. We just look at Predict, as it does not require to preprocess data during training and we get back all the information we need.
We start with creating tf_serving.py, which contains two helper functions supporting us in saving our trained model.
The first step we need to make is to create a folder to save our model in:
#---------------------------------------------------------------------
# Prepare Saving
#---------------------------------------------------------------------
from pathlib import Path
from tensorflow.saved_model import utils
from tensorflow.saved_model import signature_def_utils
from tensorflow.saved_model import signature_constants
import datetime
def build_path(prefix):
path = Path.joinpath(Path.home(), prefix, datetime.datetime.utcnow().strftime('%Y-%m-%dT%H%M%S') )
print("Model saved to {}".format(path))
return str(path)
The saved model builder requires to create this folder by its own, so you can’t overwrite a model. To ensure we get always a new one we add a timestamp as last folder.
Next, we define the PREDICT API that we will use for inference.
def predict_signature(input_layer, prediction):
tensor_info_x = utils.build_tensor_info(input_layer)
tensor_info_y = utils.build_tensor_info(prediction)
return signature_def_utils.build_signature_def(
inputs = {'images' : tensor_info_x},
outputs = {'scores' : tensor_info_y},
method_name = signature_constants.PREDICT_METHOD_NAME)
The helper function has two parameter input_layer and prediction. The first one is a tensor that is the entry point in our TensorFlow graph. It describes how the model expect the input and where to inject the input into the graph. The second one describes the output of the model, so on the one hand how the output looks like and on the other hand it gives the node in our graph from which the output shall be taken. We will see that when we build your model.
The API definition itself is called signature. As we build a prediction API our signature has one input and one output parameter. The names, 'images' and 'scores', are given by us. We cloud have chosen whatever we want.
Building Up the Model
Now we have all parts we need to build our model. For this we create cifar10.py. The script will be described in four blocks. Giving the change to explain each.
As mentioned we build a CNN for image classification. This model shall have three convolutional layers and one fully connected one. In the first block we start with declaring some hyperparameter:
Out of the hyperparameter the following shall be explained:
#---------------------------------------------------------------------
# Build TensorFlow Model
#---------------------------------------------------------------------
import numpy as np
import tensorflow as tf
import tf_serving as tfs
import model_elements as cifar
from tensorflow.saved_model import builder as saved_builder
from tensorflow.saved_model import tag_constants
from data_manager import CifarImageManager
from pprint import pprint
HYPER_PARAMETER= {
'TF_VERSION' : tf.__version__,
'BATCH_SIZE' : 50,
'NO_EPOCHS' : 50,
'LEARNING_RATE' : 0.00075,
'KEEP_PROB' : 0.5,
'C1' : 64,
'C2' : 128,
'C3' : 256
}
EXPORT_DIR = 'tutorial'
pprint(HYPER_PARAMETER)
- NO_EPOCHS give the number of epochs used for training.
- C1, C2 and C3 give the number of filters per convolutional layer.
In addition, we define the path below the home directory to the folder the model shall be saved in.
In the next block, we create two functions to inform us about the progress during the training. First one to determine the test accuracy and second one to have a progress indicator.
def test(sess, epoche=0, end='\n'):
run_time = cifarManager.train.end_epoch()
X = cifarManager.test.images.reshape(10,1000,32,32,3)
Y = cifarManager.test.labels.reshape(10,1000,10)
acc = np.mean([sess.run(accuracy, feed_dict={x: X[i], y_: Y[i], keep_prob: 1.0}) for i in range(10)])
print("{} : {:.4}% : {}".format(epoche +1, acc*100, round(run_time, 0)), end=end)
def indicator(n):
s = '[ ] '.replace(' ', '.', n) + str(n) + '%'
print('\r', s, end='')
Now we can build up the model:
cifarManager = CifarImageManager(HYPER_PARAMETER['BATCH_SIZE'])
x = tf.placeholder(tf.float32, shape=[None,32,32,3], name='x')
y_ = tf.placeholder(tf.float32, shape=[None, 10], name='y_')
training = tf.placeholder_with_default(False, shape=(), name='training')
keep_prob = tf.placeholder_with_default(1.0, shape=(), name='keep_prob')
'''
Three convolutional layer
'''
conv1 = cifar.conv_layer_bn(x, shape=[3,3,3,HYPER_PARAMETER['C1']], training=training, activation=cifar.prelu6)
conv2 = cifar.conv_layer_bn(conv1, shape=[3,3,HYPER_PARAMETER['C1'],HYPER_PARAMETER['C2']], training=training, activation=cifar.prelu6)
conv3 = cifar.conv_layer_bn(conv2, shape=[3,3,HYPER_PARAMETER['C2'],HYPER_PARAMETER['C3']], training=training, activation=cifar.prelu6)
'''
Convert output of cnn-layer into feature vector, plus drop-out to prevent overfitting
'''
conv3_flat = tf.reshape(conv3, [-1, 4*4*HYPER_PARAMETER['C3']])
conv3_drop = tf.nn.dropout(conv3_flat, keep_prob=keep_prob)
'''
Convert feature vector into on hot encoding representing the 10 classes
'''
full_1 = cifar.prelu6(cifar.full_layer(conv3_drop, 512))
full1_drop = tf.nn.dropout(full_1, keep_prob=keep_prob)
prediction = cifar.full_layer(full1_drop, 10)
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=prediction, labels=y_))
train_step = tf.train.AdamOptimizer(HYPER_PARAMETER['LEARNING_RATE']).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(prediction, 1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
Nothing too exciting if you are familiar with TensorFlow. Only three things shall be mentioned here:
- Give your variables and placeholder proper names. Otherwise, if something went wrong during deployment or interference, you will get problems finding the error.
- The shape of the placeholder x and y_ start with None. This is important, as it let TensorFlow accept any batch size including batch size one, which we need if we want to infer later in production single images.
- During inference we do not want to provide values for the placeholder training and keep_probe. So, we use placeholder_with_default As default value we choose the one we want to have during inference.
Finally, we have code to train and save the trained model.
extra_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
builder = saved_builder.SavedModelBuilder(tfs.build_path(EXPORT_DIR))
with tf.Session() as sess:
cifar.sess = sess
sess.run(tf.global_variables_initializer())
steps = cifarManager.train_len() // HYPER_PARAMETER['BATCH_SIZE']
for j in range(HYPER_PARAMETER['NO_EPOCHS']):
cifarManager.train.new_epoch()
for i in range(steps):
batch = cifarManager.train.next_batch()
sess.run([train_step, extra_update_ops], feed_dict={training : True, x:batch[0], y_:batch[1], keep_prob: 0.5})
indicator(i * 100 // steps)
print(" ", end='')
test(sess, j, end='')
print()
builder.add_meta_graph_and_variables(sess,
[tag_constants.SERVING],
signature_def_map={
'predict_images' : tfs.predict_signature(x, prediction),
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY : tfs.predict_signature(x, prediction) })
builder.save()
We want to have a look at three statements:
- builder = saved_builder.SavedModelBuilder(tfs.build_path(EXPORT_DIR)): This statement gives us the Saved Model Builder.
- add_meta_graph_and_variables( …: This is maybe the most interesting statement. We provide the session that is running, so the saver knows the graph and has access to all current Variable values. We tag our model to be foreseen for serving and provide a list of signature definitions. You should notice that we give every signature a name and that we can define a default signature, for the scope of this tutorial they are the same.
- save(): Finally save our trained model.
Now we have all the python code is there to train our model. Your folder should now contain 12 files:
Let’s run cifar10.py. We can do that e.g. by opening a console, navigate to the folder you create the python files in and call e.g. python cifar10.py. But be aware that this will run a while. If you are in a hurry or you are not using GPU you should halve the values of the hyperparameter C1, C2 and C3 and may reduce the number of epochs. If the python program has finished, you may see something like this:
That is, we achieved ~85% accuracy for the test data and it took round about 22 seconds per epoch. In case you have halved C1, C2 and C3 it is likely that you get the following:
There is a last thing that we need to do, we have to go to the folder the TensorFlow model was saved to. Here you should find two things. A file that contains the model and a folder for the variables:
Both need to be combined in a .zip file, which we want to call cifar10.zip.
That’s it for this tutorial. In the second part, as mentioned, we will upload our model to MLF deploy it and infer some examples.
- SAP Managed Tags:
2 Comments
