Simple CNN basic Model from Scratch

Convolutional Neural Network:    CNN

We will first build a CNN basic Model from Scratch  with three convolutional layers, and max pooling to automatically extract features from our images and reduce the resolution of the output convolutional feature map. We assume that you know enough about CNN, so we will not cover theoretical details. Feel free to check out my book or any other resources on the web explaining convolutional neural networks! Now let us use Keras to build our CNN model architecture.

CNN basic Model from Scratch

Here we import all required and basic tensorflow libraries .

import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np

from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Flatten
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.utils import normalize, to_categorical
from tensorflow.keras.layers import Dropout
from tensorflow.keras.optimizers import SGD, RMSprop

CNN basic Model from Scratch

Let us here we extract data available from cifar for this  tutorial. It having already  split data into X-train, X-test, Y-train, and Y-test .  If you have whole dataset then you can check Transfer learning using Mobile Net on Custom Data  , load and split the data then continue from here. It will be easy for you.

(X_train, y_train), (X_test, y_test) = cifar10.load_data()
print("The size of training dataset is: ", X_train.shape)
print("The size of testing dataset is: ", X_test.shape)
print("The size of training dataset y is: ", y_train.shape)
print("The size of testing dataset y is: ", y_test.shape)

CNN basic Model from Scratch

This few line of code help to visualized the X-train data.

#view few images 
for i in range(9):
    plt.subplot(330 + 1 + i)
    plt.imshow(X_train[i])
plt.show()

CNN basic Model from Scratch

This is normalization step. The X-train and X-test data is normalized between(0-1) by dividing with 255.

X_train = (X_train.astype('float32')) / 255.
X_test = (X_test.astype('float32')) / 255.

CNN basic Model from Scratch

It provide about the total  number of images and labels  in X-train, Y-train , X-test and Y-test.

print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)

CNN basic Model from Scratch

This is the CNN basic model with 6 convolutional layers by keeping kernel size  3 x 3  and activation function is ReLU. The input image size isset 32 x 32 . The batch normalization and Dropout is also used to avoid overfitting.

#Define Model with BatchNormalization and Dropout
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(32, 32, 3)))
model.add(BatchNormalization())

model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.2))

model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())

model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.3))

model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())

model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.4))

model.add(Flatten())
model.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(10, activation='softmax'))

# compile model
opt = SGD(lr=0.001, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
print(model.summary())

The above output shows us our summary of the basic model of CNN. As we mentioned before, we use three convolutional layers for feature extraction. The flattening layer is used to flatten 128 of the 17 x 17 feature maps we output from the third convolutional layer. This will be fed into our dense layer to get the final prediction of whether the image should be ‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’, ‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’

. This is the entire content of the model training process, so let’s use the following snippet using the fitting function (…) to train our model.
The following terms are very important to train our model:
1) batch size represents the total number of images passed to the model in each iteration.
2) Updates the unit weight in the layer after each iteration.
3) The total number of iterations is always equal to the total number of training samples divided by batch size
4) An epoch is when the complete data set passes through the network once, that is, all iterations are completed on a batch basis of data..

CNN Basic

Add Callbacks, e.g. ModelCheckpoints, earlystopping, csvlogger.

from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping, CSVLogger
early_stop = EarlyStopping(monitor='val_loss', patience=10, verbose=1)
callbacks_list = [early_stop]

history = model.fit(X_train, y_train_cat, 
                    epochs=25, batch_size=64, 
                    validation_data=(X_test, y_test_cat), 
                    verbose=1, callbacks=callbacks_list)

CNN basic Model from Scratch

The accuracy of model can be evaluated by using  model.evaluate.

_, acc = model.evaluate(X_test, y_test_cat)
print("Accuracy = ", (acc * 100.0), "%")

CNN basic Model from Scratch

plot the training and validation accuracy and loss at each epoch. If validation loss is lower than training loss this could be becuase we are applying
regularization (Dropout) during training which won’t be applied during validation. Also, training loss is measured during each epoch while validation
is done after the epoch.

loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(1, len(loss) + 1)
plt.plot(epochs, loss, 'y', label='Training loss')
plt.plot(epochs, val_loss, 'r', label='Validation loss')
plt.title('Training and validation loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()


acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
plt.plot(epochs, acc, 'y', label='Training acc')
plt.plot(epochs, val_acc, 'r', label='Validation acc')
plt.title('Training and validation accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

CNN basic Model from Scratch

Once the whole model is  build and complete training . Then we test our model using testing data, and check how much it is accurate

import random
test_img_number = random.randint(0, len(X_test))
test_img = X_test[test_img_number]
test_img_input=np.expand_dims(test_img, 0)
ground_truth= np.argmax(y_test_cat[test_img_number], axis=None)
prediction = model.predict(test_img_input)
#print(prediction)

predicted_class = np.argmax(prediction, axis=None)

plt.figure(figsize=(2, 2))
plt.imshow(test_img)

classes = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
original_label=classes[ground_truth]
prediction_label=classes[predicted_class]

print("Original class is:", original_label)
print("Predicted class is:", prediction_label)

Further more you can used regularization  technique  as well as  Image augmentation  and many more  Click here

Read more:: Machine learning algorithm,  Explainable AI, Block chain 

Github: CNN-model-for-image-classification