How to use the pre-trained VGG16 model for Transfer Learning and Fine-Tuning with the Keras API and TensorFlow. github.com/Hva... This tutorial has been updated to work with TensorFlow 2.1 and possibly later versions.
You deserve more viewers and now i understood, people believe other scrap very easily than your content one.Thank you for your wonderful explanation, it helped me a lot for my master thesis project
Sir you are great, i was searching for the same for the past few weeks but didn't find the solution, but finally found it here thank you so so much for such great videos
This is the best Video I have seen based on Tensorflow, Deep Learning. You actually helped us all out by preparing such descriptive documentation. I had read so many projects on GitHub and various other websites but is the best one. Keep up the great work. Awaiting more Tutorials from you.
I would love to see you do an update to this work especially in the area of object detection! Your channel is a hidden gem on YT but extremely informative. Thanks.
Good timing Magnus! I have been trying for three days to fine tune the Slim Inception_V3 model. I had no idea those weights were frozen. I will try the Keras Inception_V3 model following your tutorial. Thanks again for your great work!
Hello Magnus, so I was able to load the InceptionV3 model in the tutorial. It performed the prediction a bit faster than vgg. About .13 seconds vs. .7 seconds. During this process I noticed that there is a potential improvement in the way the layers are added to the pretrained model. As it is now in the tutorial if you save the model and reload it the pretrained model is made one layer so you cannot select which layers of the pretrained model to fine-tune after reloading. I recommend adding the layers to the pretrained model as follows: model = InceptionV3(include_top=False, weights='imagenet', input_tensor=Input(shape=(image_shape[0], image_shape[1], 3)), classes=1000) transfer_layer = model.get_layer('mixed7') x = Flatten()(transfer_layer.output) x = Dense(512, activation='relu')(x) x = Dropout(keep_prob)(x) x = Dense(256, activation='relu')(x) x = Dropout(keep_prob)(x) x = Dense(num_classes, activation='sigmoid')(x) fin_model = Model(inputs=model.input, outputs=x) Then after saving fin_model you can access all the layers in the pretrained model as well.
@Hvass Laboratories Thanks for sharing. Do you have a plan to do this in Tensorflow only (not Keras API)? It is really helpful if you also provide a tut how to configure, train, test in TF for multiple outputs (heads) VGG16 model, which combines 2 new classifiers (all fc layers can be reconfigured (number of neurons)). Fine-tuning classifier 1: Block1 to Block 5 pre-trained with imageNet, fc weights and biases are randomly initialized. After training classifier 1, we get a new VGG16_1, adding one more head (classifier) we will obtain a two heads model VGG16_2. We then train only FC layers of the 2nd classifier of VGG16_2 (Transfer learning).
Dear Magnus, Thanks for the tutorial! Could you please clarify for me what will happen if we initiate all layers as 'trainable' and do optimization directly? Your tutorial says: 'But once the new classifier has been trained we can try and gently fine-tune some of the deeper layers in the VGG16 model as well. We call this Fine-Tuning.' or from F.Cholet's tutorial: Fine Tune can be done in 3 steps: 1) instantiate the convolutional base of VGG16 and load its weights 2) add our previously defined fully-connected model on top, and load its weights 3) freeze the layers of the VGG16 model up to the last convolutional block' Both of you says that firstly classifier should be trained. That leads (as it given in your example) to the two steps training process, two model compiling etc..What happens if I will load VGG (with or without the top), allow all layers being trainable and run training? I'm asking because its become very time-consuming, first of all, I have to find right hyper-parameters to train classifier, then the right combination of learing rate + trainable layers for fine tuning.
Good question. As I recall, I first trained the entire network (VGG plus new classification layers) because I had forgotten to disable the training for the original network. I think it was possible, but I needed to have quite low learning-rates, maybe 1e-4 or 1e-5. With high learning-rates it didn't work, as I recall. I don't know if it will work for all data-sets and models, though. It may also happen that the training takes much longer because of the lower learning-rate, so it might actually be faster to first train the new classification layers, and then fine-tune the entire model. Please write back here what you find out so others may benefit from your experiments.
Why are you using generator_test to evaluate the model if you use it to validation_set to train the model? In theory... There are 3 independent sets.. Train / validation / test... No???
Hi! I have a question regarding the number of steps which you set to 100 in 21:12. You've mentioned in the video that you did this because Keras would run for eternity. I have found other scripts where the number of steps is calculated using the formula: number of steps = The total number of images (either in the validation or trainig set) divided by the batch size. This number would still be a fixed number. What is it that makes the difference between the two methods? Thank you for your efforts!
I can't remember exactly why I chose that number here. It's been a while since I made this tutorial. But as I recall it generates an infinite number of variations of the images in the training-set, so I probably just set the number of steps per epoch to some number that was reasonable.
I've made a classifier from scratch to classify Yoga poses i've got 72% accuracy so i wanted to improve my model using the pre-trained model VGG16 , i've got only 70% but the performence had improved ( convergence, less iterations .. ) does transfer learning always improve accuracy ?
I don't know. That's an unusual task and I imagine your data-set is quite small? Where did you even get the data-set? You could make a Notebook on GitHub Gist and ask a more detailed question on StackOverflow to get more feedback. Please write a link here to the question as I'm curious to see your Notebook.
I've collected the images from google images and i spent time cleaning it up , i end up with 7 classes each containing 370 images so it is fairly small. i will upload my notebook and send to you. Thank you
Hvass Laboratories I tried but I have very less data i.e, 22 faces per subject and total 19 subjects it throws me some error and I found out it was due to size of the dataset, I am trying facenet. Thanks for replying..
Your video help people know how to do it. Thank you so much for your contribution. I admire you. Hope that you can help us know how to do the localization and custom object detection together with image caption one of these days. These topic really interesting but it's not easy for us but sometime they are all secret for thoes who have just get familiar with deep learning like us.
How to use the trained model anyway? I tried "new_model.save('nm.h5');load_model('nm.h5')", but it won't load, complaining: ValueError: Unknown layer:name
hi sur...thanks a lot,you realy deserve better i had a little problem,i wish u reply i used your code i just changed the dataset(i used auc distracted driver dataset) and i used 50 epoches i got 75% accuracy of train(before fine tuning) .BUT 30% accuracy of test !!!! please any suggestions !
Thank you so much for an amazing/helpful tutorial!! I did your tutorial on google colab and then I tried using different datasets (Fashion_Mnist). I used: from keras.datasets import fashion_mnist (x_train, y_train), (x_test, y_test) = fashion_mnist.load_data() x_train, x_test: uint8 array of grayscale image data with shape (num_samples, 28, 28). y_train, y_test: uint8 array of labels (integers in range 0-9) with shape (num_samples,). However, unlike the datasets you used, where all the images are separated into different folders, it is just arrays . How could I use this for transfer learning? Thank you :)
For those who get the error when trying to load the kera API: ModuleNotFoundError: No module named 'tensorflow.python.keras' replace from tensorflow.python.keras with from keras. I've tested it with the rest of the code and it works fine.
I think I forgot to put version-numbers in this Notebook. It was made in TensorFlow 1.4 which should include Keras and it is imported like I wrote in the Notebook. I believe the way you import it requires Keras to be installed as a separate package e.g. using pip install.
In the original inception model, we were able to classify a single image after training. From what I understand, we set the model = inception.Inception() Then we defined for the model to classify in the def classify(image path) with pred = model.classify(image_path=image_path) Is there a similar way to call for the new model we've trained on the forks and knifes to form predictions on an image whether it contains forks, spoons, knifes or none of the three? Currently, it's only classifying everything in the test set. Thank you very much for your videos. They are such a valuable resource!
I'm glad you like my work! The Inception-code in my previous tutorials was a quick API that I put together myself. Using the Keras API you have to call new_model.predict(image). First you need to load the image as a numpy array, see the Notebook's predict() function for an example. You can also see Tutorial #03-C.
Wow, Keras makes it much easier to pull predictions from the model. I missed your #03-C file. I will definitely go through it in detail. So far, Ive gotten the new conv_model to make a prediction on the new image with: pred = conv_model.predict(img_array) However, I’m not sure how to edit this part to get the new prediction to make sense. Is decode_predicitions a function that is specifically for the VGG1 model? I’ve looked in the VGG16.py file and couldn’t find a reference to it. # Decode the output of the VGG16 model. pred_decoded = decode_predictions(pred)[0] The error message is: `decode_predictions` expects a batch of predictions (i.e. a 2D array of shape (samples, 1000)). Found array with shape: (1, 7, 7, 512) I’m guessing the new conv_model’s predictions are not the same format as VGG16?
So far this is what I was able to come up with: for pred[0,0], pred[0,1], pred[0,2] in pred: print("{0:>1%}".format(pred[0,0]), class_names[0]) print("{0:>1%}".format(pred[0,1]), class_names[1]) print("{0:>1%}".format(pred[0,2]), class_names[2]) I still cant find documentations for pred_decoded = decode_predictions(pred)[0] I assume it only works for the the output of the VGG16 model?
You may want to watch the tutorial again. conv_model is the part of the pre-trained VGG16 model that we are re-using inside new_model. So you need to call new_model.predict(img_array) to use the output of new_model. I think decode_predictions() is used for all pre-trained models in Keras that were trained on ImageNet. If you use PyCharm you can press ... hmmm... Ctrl+B or Ctr+Alt+B or Ctrl+Shift+B (I think, but not sure) and it takes you to the definition of the function. resources.jetbrains.com/storage/products/pycharm/docs/PyCharm_ReferenceCard.pdf But new_model is trained on your new data-set so it doesn't make any sense to use decode_predictions() on the output of new_model, because it is only for ImageNet classes.
Hvass Laboratories lol yes I see the mistake I made. Called the wrong layer. So decode_prediction is just to sort out the 1000 possible outputs from the full model and isn't necessary here for 3 outputs?
