python - Validation Loss and Validation Accuracy do not change during training

Question

I wrote a face classifier program with Tensorflow. In this project, first I just had 2 faces so I used binary_crossentropy as loss function. When I decided to add more faces I switched from binary_crossentropy to categorical_crossentropy.

My code:

import tensorflow as tf
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten
from tensorflow.keras.layers import Conv2D, MaxPooling2D


import pickle

pickle_in = open("/content/gdrive/My Drive/Deep Learning/Yüz Tan?ma/X.pickle","rb")
X = pickle.load(pickle_in)

pickle_in = open("/content/gdrive/My Drive/Deep Learning/Yüz Tan?ma/y.pickle","rb")
y = pickle.load(pickle_in)

X = X/255.0

model = Sequential()

model.add(Conv2D(128, (4, 4), input_shape=X.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(128, (4, 4)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dense(128))
model.add(Activation('relu'))

model.add(Dense(128))
model.add(Activation('relu'))


model.add(Flatten())  

model.add(Dropout(0.4))

model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

model.fit(X, y, batch_size=32, epochs=20,validation_split=0.3)

model.save("/content/gdrive/My Drive/Deep Learning/Yüz Tan?ma/model.h5")

And here is my training log:

Epoch 1/20
1728/1728 [==============================] - 30s 13ms/step - loss: 0.0000e+00 - accuracy: 0.4833 - val_loss: 0.0000e+00 - val_accuracy: 0.4826
Epoch 2/20
1728/1728 [==============================] - 22s 13ms/step - loss: 0.0000e+00 - accuracy: 0.4847 - val_loss: 0.0000e+00 - val_accuracy: 0.4826
Epoch 3/20
1728/1728 [==============================] - 22s 13ms/step - loss: 0.0000e+00 - accuracy: 0.4827 - val_loss: 0.0000e+00 - val_accuracy: 0.4826

As you can see my val_loss and val_accuracy don't change. What's wrong with my code and how can I fix that?

Answer 1

The answer would've been more precise if more information about the data were given.

For starters, you've used categorical cross-entropy as your loss function and Sigmoid as activation of the last layer, which is kind of contradictory (sigmoid means you're classifying among 2 classes and categorical cross-entropy is something you use when you've got more than 2 classes). Either change your loss to Binary Cross-entropy if you want to use sigmoid (which means you've 2 classes) or you should change your sigmoid to softmax function if you want to classify among more than 2 classes.

Also, you've to use flatten after the last CNN layer and before the first dense layer (it turns the feature map (a matrix) into a vector, which is the correct input shape for dense layers.

Lastly, after doing all those you can play around with your hyperparameters (learning rate, batch_size, etc.) to see if you can get some accuracy gain.