No video :(

Image preparation for CNN Image Classifier with Keras

Подписаться 150 тыс.

Просмотров 99 тыс.

50% 1

In this episode, we go through all the necessary image preparation and processing steps to get set up to train our first Convolutional Neural Network (CNN).
🕒🦎 VIDEO SECTIONS 🦎🕒
00:00 Welcome to DEEPLIZARD - Go to deeplizard.com for learning resources
00:30 Help deeplizard add video timestamps - See example in the description
08:12 Collective Intelligence and the DEEPLIZARD HIVEMIND
💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥
👋 Hey, we're Chris and Mandy, the creators of deeplizard!
👉 Check out the website for more learning material:
🔗 deeplizard.com
💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES
🔗 deeplizard.com/resources
🧠 Support collective intelligence, join the deeplizard hivemind:
🔗 deeplizard.com/hivemind
🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses
🔗 neurohacker.com/shop?rfsn=648...
👀 CHECK OUT OUR VLOG:
🔗 / deeplizardvlog
❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li
🚀 Boost collective intelligence by sharing this video on social media!
👀 Follow deeplizard:
Our vlog: / deeplizardvlog
Facebook: / deeplizard
Instagram: / deeplizard
Twitter: / deeplizard
Patreon: / deeplizard
RU-vid: / deeplizard
🎓 Deep Learning with deeplizard:
Deep Learning Dictionary - deeplizard.com/course/ddcpailzrd
Deep Learning Fundamentals - deeplizard.com/course/dlcpailzrd
Learn TensorFlow - deeplizard.com/course/tfcpailzrd
Learn PyTorch - deeplizard.com/course/ptcpailzrd
Natural Language Processing - deeplizard.com/course/txtcpai...
Reinforcement Learning - deeplizard.com/course/rlcpailzrd
Generative Adversarial Networks - deeplizard.com/course/gacpailzrd
🎓 Other Courses:
DL Fundamentals Classic - deeplizard.com/learn/video/gZ...
Deep Learning Deployment - deeplizard.com/learn/video/SI...
Data Science - deeplizard.com/learn/video/d1...
Trading - deeplizard.com/learn/video/Zp...
🛒 Check out products deeplizard recommends on Amazon:
🔗 amazon.com/shop/deeplizard
🎵 deeplizard uses music by Kevin MacLeod
🔗 / @incompetech_kmac
❤️ Please use the knowledge gained from deeplizard content for good, not evil.

Опубликовано:

2 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 362

@deeplizard 6 лет назад

Keras Machine Learning / Deep Learning Tutorial playlist: ru-vid.com/group/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL Machine Learning / Deep Learning Tutorials for Programmers playlist: ru-vid.com/group/PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU

@viswatejvarma7792 4 года назад

Then why I am getting "Found 0 images belonging to 2 classes. Found 0 images belonging to 2 classes. Found 0 images belonging to 2 classes."?? please tell me I used the code what exactly you used

@joe8ones 4 года назад

@@viswatejvarma7792 mine is also doing the same ting

@joe8ones 4 года назад

Pls help,Found 0 images belonging to 2 classes. Found 0 images belonging to 2 classes i have tried everything possible, still getting this message i cant move on because of this

@68sharmaani 6 лет назад

This was the best video I've seen to explain this. Short, simple, sweet. Thank you so much!

@BPDanek99 6 лет назад

I've been looking for this exact solution for 4 days now. Thank you so much for being thorough in your explanations. Keep up the awesome work!!

@allisonlowe9594 2 года назад

you and me both, my friend

@betojasz 6 лет назад

Great!! Exactly what I'm looking for!! Thanx!!

@PrashantKumarPrasad 6 лет назад

This is what I am exectly looking for. Thanks a lot.

@coop0199 5 лет назад

Thank you for the informative video!

@deepaksingh9318 6 лет назад

Perfectly explained again..

@osmankultur1943 4 года назад

Gosh...Lady, you are AWESOME !!!

@sandejai 3 года назад

Thank you

@nicolitox 6 лет назад

Thank you very much for this!

@deeplizard 6 лет назад

You're welcome, nicolitox!

@adanegebretsadik8390 5 лет назад

thank you so much

@portgasdace8961 6 лет назад

you're doing a great job keep up :)

@asim-gandu-phenchod 3 года назад

Mandy from deeplizard. Thanks a lot. Really nice

@katia-renaepurnell3279 4 года назад

Thank you so much!! This is what I've been looking for for the past three days. Is there a video about how to turn the image labels shown here into string categories?

@user-jd3so8tj2x 6 лет назад

thanks!!!!

@rohan4748 5 лет назад

Amazing tutorial

@jieyideng4482 5 лет назад

Hi, thanks for sharing your knowledge and experiences in DL! I have one question about the datasets you generated in this video: if we preprocess the images first by turning it into BGR array first, how to write the generator for the fit_generator?

@noeltam75 6 лет назад

1) For image preparation, here you coded target size 224x224 I understood for VGG16 reason. I happened to have 14970 images (13classes) , but ALL of different sizes. Do I have to pre-resize all to 224x224 prior training? Or I can just leave them as it is (diff sizes). 2) Batches = 10, am i correct to say the number of images in each folder had to be divisible by 10 too? (i.e 200 is ok, but not 234).

@deeplizard 6 лет назад

Hey Noel, 1. The images can be of all different sizes, and the target_size parameter will resize them to 224x224. 2. The number of images within each folder is not required to be divisible by the batch size. If there were a remainder, like if you had 234 images when using a batch size of 10, then the last batch would just be smaller. It would be a batch of 4 in this case.

@lakshmitejaswi7832 5 лет назад

@@deeplizard at the test time if have only one folder then I am getting error

@luismisanmartin98 5 лет назад

@@lakshmitejaswi7832 leave the classes argument empty and see what happens. (Rather than "classes=['dog', 'cat']" write "classes=[]".)

@lawrencechoo6492 6 лет назад

Thanks deeplizard, btw you could make more video on unsupervised learning such encoder or decoder and generative adversarial network. I find that your teaching is complete and easy to understand.

@deeplizard 6 лет назад

No problem, Lawrence. I've done a video on unsupervised learning, and I touched on autoencoders there. Check it out if you haven't already: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-lEfrr0Yr684.html But yes, in general, I do plan to expand more in that area, and I have GANs on my list of potential topics to cover in future videos as well. Thanks for the suggestions!

@shubhamraorane3383 4 года назад

Jag tackar dig från mitt hjärta.

@zeegtab 4 года назад

Hi thank you for the video! Is there a different version of that plots function that lets you plot the images with their respective labels using IDLE (not Jupyter notebook)?

@dhirajpatel6093 4 года назад

this is what just i am looking for

@DrKhan-hd4cd 5 лет назад

I would like to clarify something, many of the tutorials online are not as organized and do not focus on generating custom datasets as you have done. The concept of one-hot encoding is important for multiclass data, this can cause much confusion if not explained. Excellent work!

@deeplizard 5 лет назад

Thanks, Dr. Khan! I agree it's a subtle but important concept to grasp. We have a couple of other videos that go hand-in-hand with the one-hot encoding topic if you're interested in checking those out as well! One-hot encoding explained - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-v_4KWmkwmsU.html Mapping Keras labels to image classes - ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pZoy_j3YsQg.html

@pervaiz.khan1857 5 лет назад

Great series of videos indeed. I have a question here, do we need to specify class names for Validation and Test sets? If not then how can we skip them ?

@deeplizard 5 лет назад

Hey Pervaiz - The validation set requires the data to have labels, but the test does not require this. If you don't know the labels for the test data, you need to have one more directory within the "test" folder. Call the nested directory "unknown" for example. Then, within this unknown folder, place all the unlabeled test images there. With this structure, you will need to modify the test_batches variable. The parameters called classes and class_mode within this variable will need to be changed. Specifically, the change will be to make classes = None and class_mode = None. This is due to not knowing the labels for the test data.

@TerragonDE 6 лет назад

It is 2018 and we are dealing with artificial intelligence and how do we organize and show pictures in folders? With the command line! :-D

@mortalrahu 5 лет назад

When we specify the classes and path to the ImageDataGenerator(), does it swoop into the subfolders and embed labels to them appropriately with the help of their folder names? Is my assumption correct? The generator assigns the class labels to the images based on the folder names, right?

@deeplizard 5 лет назад

Yes, correct!

@bobypardamean7355 4 года назад

Then how can we make confusion matrix from this?

@volverearound6445 4 года назад

thanks, that's what I am confused about

@lawrencechoo6492 6 лет назад

How do you reset the batches into another set of 10 for the images. That you had done in the video?

@deeplizard 6 лет назад

Hey Lawrence - I just re-ran cell #7 in the notebook. next(train_batches) will grab another batch from train_batches.

@rajvijay3276 6 лет назад

Awesome work mam. Could you please use your full screen for future videos?

@deeplizard 6 лет назад

Thanks, Raj! Yes, in later videos, I started zooming in on the code. Check this one out for example: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-zralyi2Ft20.html

@kullbackleibler9510 5 лет назад

What if you don't have the Training, Test and Validation Set divided by Folders? How do you split it, when using a generator?

@dr.saidboumaraf2126 5 лет назад

Hello deeplizard, I really appreciate theses series of videos, I have a question please, there is a specific image formats that keras can read (PNG, JPG, ...), I'm working with medical images and the format is ''DICOM' : a famous medical image format, so how to deal with this type of images and how to prepare them to the CNN? thank you.

@arthurchavescosta1262 4 года назад

Very nice video. In my case, i am using flow_from_dataframe, and the output batch only contains images with all pixels equals 255... Don't know what's wrong...

@ldthan 6 лет назад

There are any recommendations that how many percent and how many datasets should o use own datasets, including training, validation, and test?

@deeplizard 6 лет назад

Hey Than - A decent rule of thumb is to split your data into 80% for training and 20% for testing. Then, from your training set, split 20% of it out for the validation set. If needed, I cover these sets more in depth in this video in case you haven't seen it yet: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Zi-0rlM4RDs.html

@TheAnna1101 6 лет назад

great video. if I have a multilabel dataset, can I still use ImageDataGenerator().flow_from_directory(...)? thanks

@deeplizard 6 лет назад

Thanks, R Change! I'd imagine you could, but I've not tried this myself. If you used flow_from_directory() for a multilabel dataset, then the directories would still need to be labeled with the corresponding labels. So, if your model could identify whether an image contained a cat, or a dog, or _both_ a cat and a dog, then the labels of your directories would need to be "cat," "dog," and "cat-and-dog," for example.

@josemattus4729 4 года назад

Great video, very helpfull. Im having a problem, im having batches of 25 imgs each one, but when i plot them all are the same and from the same category, could anybody give me a solution pls?

@WKhan-jl3fv 6 лет назад

Hi, I love your videos. My question is, how we choose the right image size i.e. 224x224 in this case. I mean, how to choose best parameters and Image size etc? Can you share some link so I can follow?

@deeplizard 6 лет назад

Hey Wasif - Thank you! I'm glad you're liking the videos! We're working with 224x224 in this case because that is what the VGG16 network that we use later in the playlist expects to receive. (That is covered in the videos with the title containing "fine-tune" later in the playlist.) If using a pre-trained net, the target size of your images will be determined by what the net expects to receive, like we've done with the images for the VGG16 network that I mentioned above. If you're building a model from scratch, then I'd suggest looking at other networks that classify similar images to see what image sizes they use for a similar task, and then implementing that in your own model and see how it goes. For example, a net trained on images of cats and dogs may expect images with far different dimensions than a net trained on images of cancer samples. One key piece of information is the larger the dimensions, the more parameters there will be in the network. So, training will take longer, and you may need relatively more resources on your machine to process these large images. You also don't want your images too small either because then you will possibly lose detail/resolution.

@WKhan-jl3fv 6 лет назад

Thank you :)

@ebtihalm926 5 лет назад

Other question please, When using img , labels = next ( train_batches) is there a possibility to redundant the same image formate with other created batches? Thank you deeplizard, You can't image how much your videos are helpful. appreciated ALOT

@deeplizard 5 лет назад

You're very welcome, Ebtihal! Glad to hear that! I'm not quite sure that I understand the question. Can you please elaborate?

@ebtihalm926 5 лет назад

@@deeplizard thank you for your reply. I understand the the main purpose of using img , labels = next ( train_batches) code is to generate new data/images that are not available in our original dataset. My question is, is there a change to generate any image that already generated before.. if yes, this might affect the accuracy reading , is it?

@lahaale5840 5 лет назад

Hello, Thanks for sharing. How the labels is attach to the images? I did not see any code do that job?

@deeplizard 5 лет назад

Here, Keras interprets the labels for the images based on the directory structure. For example, all images contained within the dog directory are recognized by Keras as having a dog label. The dog label is then one-hot encoded.

@vrajpatel8256 4 года назад

Suppose I have an image sized 40*40 pixel and I call the class of image with a target size of 224*224 using ImageDataGenerator, does the size of generated tensor remain the same or is it converted to a size of 224*224 with zero paddings around the actual image?

@biologyassignment1955 4 года назад

Thank you Mandy for the great video! If we want to normalise the pixels (by dividing by 255), can we do it using flow_from_directory()?

@deeplizard 4 года назад

You can create your own function to normalize the data in this way and then set the preprocessing_function parameter to be your function in ImageDataGenerator(). For example, if your function is called normalize_data, then: ImageDataGenerator(preprocessing_function=normalize_data, ...).flow_from_directory(...) See the requirements for the function here. www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator

@madhurjodas9933 4 года назад

Is it possible to do the coding on spyder? and if possible can we copy the plotting code there as well?

@prudhviraju2629 6 лет назад

how can i work with gabor filters as preprocessing step in my case before feeding my image data to transfer learning model. How can i intiate log gabor, gabor, laplace gabor etc..various image processing filters to image and then process through cnn model we have from vgg, resnet,etc..basically weights=kernel=filter am i r8 in this case i only have mathematical formulas in case of these filters and i want to convert into matrix of its form before processing into transfer model any idea. Or suggestion thankful to u.

@deeplizard 6 лет назад

Hey PRUDHVI, Yes, the words "kernel" and "filter" mean the same thing here, and the weights are the numerical values within the filter. The two videos below may shed more light on this concept if you're wanting to gain further insight. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-YRhxdVk_sIs.html ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-cNBBNAxC8l4.html To use Gabor filters with Keras, I believe you will need to work with Keras initializers (keras.io/initializers/). These allow you to define the initial random weights of Keras layers. They have several available initizliers, but I'm not seeing that Gabor is one of them according to the link to the documentation above. You'll also see that they allow you to use custom initializers, which are initializers that you define yourself. You could use a Gabor filter as a custom initializer. I haven't done this myself, but it appears the author of this stackoverflow post did it and posted the corrected code as one of the answers: stackoverflow.com/questions/46177505/keras-kernel-initializer-with-gabor-filter Hope this helps!

@ayushrai6699 4 года назад

How to predict when the test set images are not present in the specified way as you mentioned in this video. Then how to use Image generator to predict and preprocess.

@deeplizard 4 года назад

Check out the corresponding blog for this episode. deeplizard.com/learn/video/LhEMXbjGV_4 Specifically the section titled "Scenario Of Not Having Test Labels," as well as the last paragraph in the section titled "Process The Data."

@ajitjadhav7540 5 лет назад

This is the only video tutorial that I've found to be great. I have a question regarding image size. You specify target_size=(224, 224), is this an image size of all images that I need to have in my PWD as training, valid and testing datasets??

@deeplizard 5 лет назад

Thanks, Ajit! No, the target_size parameter just specifies the dimensions to which all images found will be resized. So, your images do not have to be this size ahead of time.

@ajitjadhav7540 5 лет назад

thanks a lot

@Famas54321 5 лет назад

Hi, how did you encode your labels? I'm looking at what you've shown in your folder structure, but nowhere do i see there labelling of data. How do you get one-hot encoded labels when you print out the batch sizes?

@deeplizard 5 лет назад

Hey Famas - Check out the video below. There, I talk about how Keras maps the one-hot encoded labels to relevant classes. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pZoy_j3YsQg.html

@rajuthapa9005 6 лет назад

you have imported:: "import matplotlib.pyplot as plt" and also "from matplotlib import pyplot as plt" , are these used for different purpose?

@deeplizard 6 лет назад

Hey Raju - This redundancy was included by accident. There is no difference between what these two lines of code are importing or how the imported library is being used.

@natashakulkarni2171 4 года назад

Hey don't we need to normalise data before feeding into the network? What about converting images into numpy array?

@faiqahmadkhan6009 5 лет назад

Thank you so much,, is there any tutorial available about training our CNN model on sounds,ECG and other one dimensional signals rather than images.?

@deeplizard 5 лет назад

You're welcome, Faiq! We currently don't have any content regarding networks that accept sounds or one dimensional signals.

@faiqahmadkhan6009 5 лет назад

@@deeplizard it's okay, thanks you, your videos are very helpful, God bless you,

@KishoreKumar-vb8nn 6 лет назад

Should the images be of the specified size (224,224) beforehand or will the ImageDataGenerator function resize it to (224,224) ? Asking because I used VGG-16 model as per your following video and yet getting very bad accuracy(0.5) for the same classification problem(cats and dogs).

@deeplizard 6 лет назад

Hey Kishore - The images do not need to be of the specified size ahead of time, as ImageDataGenerator() will take care of sizing. If you're using the exact code from the videos and similar images, you should be obtaining very similar results to what you saw in the video. Have you made any code changes? Also, perhaps your images differ widely from the ones I used. Try using images from the link below. It's where I got the ones for the video. www.kaggle.com/c/dogs-vs-cats/data

@danielherweg5085 5 лет назад

Hi, fantastic series! Question - Would there be a way to include sample weights in the way you have constructed the batch generators here, or would it be better to go about it another way entirely?

@deeplizard 5 лет назад

Thanks, Daniel! I'm not sure that I understand your question about sample weights. Can you please elaborate?

@danielherweg5085 5 лет назад

Thx for responding! I would like to weight the importance of each sample and I am not sure how to make it work. Here we pass the train_batches generator, which yields 'images' and classes, to the fit_generator function. We can pass a generator with three elements to fit_generator if we would like to also incorporate sample weights. I am curious if we can modify the train_batches generator, as it is constructed here, to also yield this third element. I have kept a dict of the image file names and weights, but I am not sure how to pull the ones that correspond to the batch of images selected and add them to the generator. My sense is that this is not easily done with the functions used here and that I may have to build a generator from scratch. Does this make sense? Any thoughts?

@deeplizard 5 лет назад

Ok, I see. I'm not aware of how we can make that work using flow from directory function shown in this video. The Keras flow function, on the other hand, will allow you to pass a sample weights parameter. If you use that though, then the nature of how we generate and pass data to the model in this video will indeed change. keras.io/preprocessing/image/#flow

@developer8726 5 лет назад

For test_batch, suppose we don't know the classes what changes need to be done to ImageDataGenerator line as just removing classes says found 0 images from 0 classes.

@parthdodiya9428 5 лет назад

yes, same here, have you found any solution?

@mohamedberrimi4850 5 лет назад

Thank you so much for alllll your videos and your efforts !! I have one question : i have a dataset of 84K images. When i set the " batch_size=16," in image generator , what does this means exactly ? does this means that every epoch i take 16 images to train on ?

@deeplizard 5 лет назад

You're very welcome, Mohamed! Check out the video and blog below where I explain batch size and its relation to an epoch in full detail. Let me know if this clears up your question. deeplizard.com/learn/video/U4WB9p6ODjM

@mohamedberrimi4850 5 лет назад

@@deeplizard Thanks , i've already seen the blog (i'm following your series ) , the problem is in dataGeneator there are many parameters to use , batch size , steps_per_spoch , samples_per_epoch .. . for Exemple , for a given dataset of 80K images. ------------------------------------- train_generator = train_datagen.flow_from_directory( train_data_dir, target_size=(IMG_SIZE , IMG_SIZE), batch_size = 88, subset='training', class_mode='categorical') valid_X, valid_Y = next(train_datagen.flow_from_directory( train_data_dir, target_size=(IMG_SIZE , IMG_SIZE), subset='validation', class_mode='categorical')) --------------------------------- and in fit generator i'm putting steps_per_epoch=64. Does this means that all my data are passed or not ?

@deeplizard 5 лет назад

Generally, you want steps per epoch to equal the total number of training samples divided by the batch size. For example: steps per epoch = 80K samples / 88 = 909.09 ~ 910 (round up to nearest whole number) With your current code using steps per epoch = 64, during one epoch you will only pass 88x64 = 5632 samples to the model.

@LucasPit 5 лет назад

Hey, DpLizard, Love your vids, I'm currently having trouble with this "next" function. I always get: "ImportError: Could not import PIL.Image. The use of `array_to_img` requires PIL." Does it mean my Pillow is broken? Should I import an PIL library? Edit: I uninstalled pillow and reinstalled it through Anaconda promt, and after restarting the notebook it worked flawlessly

@deeplizard 5 лет назад

Thanks for sharing your solution, Lucas!

@actechforlife 5 лет назад

thanks, Lucas, I also had the same problem and did exact thing as yours, it worked.

@LucasPit 5 лет назад

Glad to hear it worked for someone else! =]

@Waleed-qv8eg 6 лет назад

Great job. Excuse me, How can you label them as one-hot encoding? I mean [0. 1.] for cats and [1. 0.] for dogs? Thank you!

@deeplizard 6 лет назад

Hey Net4Easy - The images here were one-hot-encoded based on the flow_from_directory() function. There is a parameter specified within that function, called "class_mode," which determines the type of labels for the data. The default value for this parameter is "categorical," which are one-hot encoded labels. We didn't explicitly set that here because it is the default value. I talk a little bit more about mapping the labels from images to their one-hot encoded vectors in this video: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pZoy_j3YsQg.html Does this help?

@Waleed-qv8eg 6 лет назад

So does this mean if we have another folder for another class the function should be like this: test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size =(224,224), classes =['dog', 'cat', 'lezard'], batch_size = 10) so that would generate this hot-one like this [1 0 0] [0 1 0][0 0 1]? Thnaks :)

@deeplizard 6 лет назад

Yes, exactly!

@mudasserahmad6076 10 месяцев назад

Hey i am dealing with mel spectrograms i have created 3 mel spectrogram with different windows length 93ms 46ms and 23ms. My input to cnn is 128,216,3 what does 3 show here

@SuiGio 6 лет назад

Hello, could you somehow specify the paths in a more dynamic way? Lets say you had multiple classes separated over subdirs. How could we build a generator where after the last file of a folder, start reading the first file of the next folder. Thanks for all these tutorials, they are awesome! Keep it up!

@deeplizard 6 лет назад

Thanks, George! I'm glad you're liking the videos. The only paths we're really needing to explicitly state are the paths to the training set, validation set, and test set. The paths to the individual directories of the classes of cat and dog images are not given. We do, however, state the actual classes (without the paths) in the train_batches, valid_batches, and test_batches when we specify the _classes_ parameter within each one of those. If we don't specify this classes parameter though, Keras will infer the classes automatically from the subdirectory structure/names. So, in this video, if we didn't specify ['cats', 'dogs'] for the classes parameters in each of the batch variables, then Keras would automatically obtain the classes based on how we set up the subdirectory structure within the train, test, and valid paths. Does this help clarify?

@SuiGio 6 лет назад

deeplizard I almost fully understood. I am working on different dataset, which has features and labels in the same dir, wondering if I could somehow read every file from every dir in my dataset, for creating the generator. I've tried glob library which indeed gathers the proper information, but i dont know how to apply it on my data generator. Thank you for replying so fast and being so informative!

@deeplizard 6 лет назад

Hey George, I see. Hm... I'm not sure how you might be able to implement that using Image_Data_Generator().flow_from_directory(). If you do happen to figure it out, I'd love to hear the solution if you're willing to share.

@algorithmo134 Месяц назад

Hi, where can I obtain the dogs and cats images?

@efferossi3186 5 лет назад

Hi! How can I prepare a dataset for multi-labeled images, for example the label could be if the animal is a cat or dog and also how many dog there are

@deeplizard 5 лет назад

HEy effe - To create a model that accepts multi-labeled inputs, you need to make use of the Keras Functional API. Note that in most of the videos of this Keras series, we use the Sequential API. The Functional API allows us to define more complex models, like multi-input or multi-output models. For more information on this topic, check out this link: keras.io/getting-started/functional-api-guide/#multi-input-and-multi-output-models

@decentralmultiverse6719 4 года назад

I have uploaded the dataset as mentioned in this video but getting "Found 0 images belonging to 2 classes." for all train, test and valid folder

@blogtheorem_ 4 года назад

give absolute path like this train_path=r'D:\WORK bs\cats-and-dogs\train'

@samcha8001 5 лет назад

Hello Deeplizard, Whenever I run "plots(imgs, titles=labels)" on Jupyter notebook, I have an error saying "Kernel Restarting The kernel appears to have died. It will restart automatically". I am using TensorFlow as a backend engine. Could you please help me out on this issue?

@Ketzunouka 4 года назад

thankyou for the explanation, btw this video is not in keras tutorial playlist, i'm a little confused with the tutorial order lol XD

@deeplizard 4 года назад

You're welcome :) The reason why this video is not in the playlist is because it episode has been updated and replaced in the course: deeplizard.com/learn/video/LhEMXbjGV_4 This course is still currently being updated, but you can stay up-to-date with the most recent videos and updates to the blogs by following the course from here: deeplizard.com/learn/playlist/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL

@emmayin6822 4 года назад

Are all you images the same size? Can I use images of various sizes? Thank you!

@deeplizard 4 года назад

You can use various image sizes. The target_size parameter that you specify in train_batches, valid_batches, and test_batches will resize images to that particular size before passing them to the network.

@viswatejvarma7792 4 года назад

@@deeplizard Then why I am getting "Found 0 images belonging to 2 classes. Found 0 images belonging to 2 classes. Found 0 images belonging to 2 classes."?? please tell me I used the code what exactly you used

@foodtube9338 4 года назад

Hi! Here you're using 10 test_images and the batch_size for test_batch is also 10.But if i have test_images= 400 then what should be the batch size.will it be 400?if not then how will i take the test_label input using next(test_batch). I mean how will i have the labels for all my test_images if i have 400 test_image. Can you please help me out? Thanks

@sandejai 3 года назад

Please arrange the playlist, so if I click next video button, it takes to your video, currently its taking to some other channel video

@deeplizard 3 года назад

The playlist is arranged in order, however, we do not have control over what RU-vid chooses to display as the next autoplay video. To view the courses in order, you should watch from the RU-vid playlist page or from the deeplizard website. Note, this video is from the first version of the Keras course. It has since been updated. Follow the updated Keras course linked below: Full course: deeplizard.com/learn/playlist/PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL RU-vid playlist: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-tDaGT4N4aCA.html

@chr132nort 6 лет назад

Hey, so at the moment I am working on making a CNN to detect Retinoblastoma(eye cancer). I already have a dataset and I have made an SVM with sklearn and I am having trouble changing it into a CNN. First, could you explain the validation dataset, because in my SVM the validation set is just a 0 or a 1 saying if its a normal eye or a cancerous one, so what kind of images would I put in the folder? and why does you valid dataset have less images than the test dataset?. Second, when I try train_batches = np.array(train_batches) my computer runs out of memory and I get a memory error, not sure what I could do about that though. Thanks!

@deeplizard 6 лет назад

The type of data in the validation set should be the same type of data that is in the training set, just different samples. For example, the training set used in this video is made up of images of cats and dogs, and the validation set is made up of the same type of data-- images of cats and dogs-- but does not contain any of the exact same images as the training set. The video below goes over the train, validation, and test sets in detail. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-Zi-0rlM4RDs.html For the issue with train_batches, I assume your training data is image data, right? If so, are you defining train_batches as ImageDataGenerator().flow_from_directory(), like we do in this video? If you are, then there is no need to convert it to a numpy array.

@nandeenandee2078 6 лет назад

Hey deeplizard.. I downloaded the images from the kaggle website and it contains just two directories with the images for the two classes. How do i select the images for the test and valid directories? Do i just copy and paste any images from the two directories?

@deeplizard 6 лет назад

Hey Nandee - Yes, I just randomly selected a small subset of the supplied cat and dog images and organized them into the directory structure shown in the video via copy/paste. In another video, I wrote a script to organize images into directories for another data set. This creates an automated way to do this task, rather than manually creating the directories and copy/pasting files. You can check that out and see if you're interested in using and modifying that script for this data set: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-FNqp4ZY0wDY.html

@nandeenandee2078 6 лет назад

Deeplizard, thank you very much for your support. It's highly appreciated.

@rajuthapa9005 6 лет назад

Here is that additional code: github.com/fastai/courses/blob/master/deeplearning1/nbs/utils.py def plots(ims, figsize=(12,6), rows=1, interp=False, titles=None): if type(ims[0]) is np.ndarray: ims = np.array(ims).astype(np.uint8) if (ims.shape[-1] != 3): ims = ims.transpose((0,2,3,1)) f = plt.figure(figsize=figsize) cols = len(ims)//rows if len(ims) % 2 == 0 else len(ims)//rows + 1 for i in range(len(ims)): sp = f.add_subplot(rows, cols, i+1) sp.axis('Off') if titles is not None: sp.set_title(titles[i], fontsize=16) plt.imshow(ims[i], interpolation=None if interp else 'none')

@jurgenk.4434 6 лет назад

Your adoption is that all images have the same size. How do you bypass pictures which all have different dimensions?thanks

@deeplizard 6 лет назад

Hey Jürgen - The images aren't required to all be of the same size. In the video where we set up the train_batches, test_batches, and valid_batches variable, we set the target_size parameter to be the desired dimensions for the images. In our case, the desired size was (224, 224). So, the images within the directories could all be of different sizes, and then would be scaled to the size that we specified in the target_size parameter. Does this clarify things?

@jurgenk.4434 6 лет назад

deeplizard Yes, thanks a lot

@shreyastasamal3549 6 лет назад

Hello there, At 3:23 in the test folder you mentioned there would be no separate folder containing cats and dogs label, so the code for test_batches would be test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size =(224,224), classes = None, batch_size = 10). Here the classes would be none initially and only after prediction model is set, we can label these images as cats and dogs right?

@deeplizard 6 лет назад

Hey shreyasta - Yes, you're mostly correct. The only addition is that you'll need at least one directory inside of your test_path directory for flow_from_directory() to work appropriately with the unlabeled data. So your test_path may look something like test-directory\unknown\. Additionally, just as you set classes = None, you will also need to specify class_mode = None within flow_from_directory() as well.

@shreyastasamal3549 6 лет назад

Hi Deeplizard, Thank you for replying. Yesterday I did add an additional folder called cats_or_dogs_predict , moreover, I used the following line test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size =(224,224), batch_size = 10) In this line I tried not specifyingthe classes and class_mode and after reading the keras documentation, I think in case values are not specified, the defaults value is taken as classes = None, and class_mode = None within flow_from_directory() . Does this impact the code in any way ?

@deeplizard 6 лет назад

Hey shreyasta - No problem! If you don't specify the classes parameter, then you're right, the default value is None. So this is fine to not specify here. If you don't specify class_mode, however, then the default is "categorical." So you'd actually need to specify class_mode = None. This is the link to where I found the default values: keras.io/preprocessing/image/

@shreyastasamal3549 6 лет назад

I will change the value of class_mode, thank u so much for your input and clarifying my confusion. As one does get lost with too much documentation. Have a good day :)

@deeplizard 6 лет назад

Absolutely! Good day to you as well!

@harshalbagul4406 6 лет назад

i'm using 4 classes and how should i set the confusion matrix prediction array cm = confusion_matrix(test_labels, predictions[:,0])

@deeplizard 6 лет назад

Hey Harshal - You can plot a confusion matrix with more than two classes. For example, the links below are to scikit-learn's documentation and code on the confusion matrix. They include an example of plotting a confusion matrix with three classes. Let me know if this helps. scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-selection-plot-confusion-matrix-py scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html

@harshalbagul4406 6 лет назад

Thank you!

@zeegtab 4 года назад

Would it be possible to use this (with apt modification in the code) for multiple classes e.g cats, dogs, rabbits, foxes etc?

@deeplizard 4 года назад

Yes

@adanegebretsadik8390 5 лет назад

first of all thank you for all, 1. i am doing a research on some disease detection using CNN so how can i pre-process the image dataset and feed in to the machine ? 2. do i need image processing or computer vision to get the images or to collect by my self?

@evidencemonday5948 5 лет назад

please, may i know the dataset you're using? Thanks

@adanegebretsadik8390 5 лет назад

i am using the HAM 10000 dataset for skin cancer disease but it has only training set. could you please help me to divide into test set and validation set? thanks

@mohammedshehada5373 5 лет назад

where did you label the images at first?

@deeplizard 5 лет назад

I downloaded the images from the link below where they were already labeled. www.kaggle.com/c/dogs-vs-cats/data

@evidencemonday5948 5 лет назад

i keep getting this errr: TypeError: show() got an unexpected keyword argument 'interpolation' is there something I'm missing?

@pavanilla4374 5 лет назад

if my image size is less than the target_size parameters will it resize them into my target_size TIA

@deeplizard 5 лет назад

Yes

@RedShipsofSpainAgain 6 лет назад

If anyone encounters the error "ImportError: Could not import PIL.Image. The use of `array_to_img` requires PIL." you basically need to uninstall PIL and install the Pillow package: stackoverflow.com/questions/41124353/importerror-could-not-import-the-python-imaging-library-pil-required-to-load Then run in your notebook: from IPython.display import display from PIL import Image

@deeplizard 6 лет назад

Good tip!

@prasannakumar7035 5 лет назад

good explanation but where i can grab the dataset?

@deeplizard 5 лет назад

Hey prasanna - The images I'm using are a random subset of this data set: www.kaggle.com/c/dogs-vs-cats/data

@elainewang7050 4 года назад

Hey Mandy, I don't know why when I run this code below: def plots(ims,figsize=(12,6), rows=1, interp=False, titles=None): if type(ims[0]) is np.ndarray: ims = np.array(ims).astype(np.uint8) if (ims.shape[-1] != 3): ims = ims.transpose((0,2,3,1)) f = plt.figure(figsize=figsize) cols = len(ims)//rows if len(ims) % 2 == 0 else len(ims)//rows + 1 for i in range(len(ims)): sp = f.add_subplot(rows, cols, i+1) sp.axis('Off') if titles is not None: sp.set_title(titles[i], fontsize=16) plt.imshow(ims[i],interpolation=None if interp else 'none') imgs,labels = next(train_batches) plots(imgs, titles = labels) It always says that there is some error in the second line: IndexError: index 0 is out of bounds for axis 0 with size 0 What's the problem with this?

@harishgowdabp 5 лет назад

what is expected salary for this job (6 months exp)

@hermonjay4744 6 лет назад

I got prime number in my validation and test set (71). Any suggestion what number should I put in batch_size as well as in steps_per_epoch? My training set is also lack of divisible number (566).

@deeplizard 6 лет назад

Hey Hermon - Good question. It's okay if your datasets are not evenly divisible. The last batch will contain the "left over" amount. For example, with your validation set of 71, if you chose 5 as your batch size, then the steps_per_epoch would equal 15. This is because 71/5=14.2, so we round up to 15. So you'd have 14 batches of 5, and then the 15th batch would contain the 1 left over sample. (14 * 5) + 1 = 70 + 1 = 71 Make sense?

@hermonjay4744 6 лет назад

deeplizard Yes thank you it makes sense. 👍

@amritaranamagar3332 6 лет назад

hello deeplizard...great tutorials. but i am getting this error 'integer division or modulo by zero' while running this code of line 'imgs, labels = next(train_batches)' Thanks.

@deeplizard 6 лет назад

Hey amrita - It sounds like your train_batches object doesn't actually contain any images. When you define train_batches, you should see output that states something like, "Found x images belonging to y classes." What does your output say? Make sure your directory structure is organized in the same manner as shown in the video and that you're pointing to a path that contains class directories with images inside of each class.

@kaitodaimon7929 5 лет назад

I keep encountering the following problem: Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

@BrettClimb 5 лет назад

Same here.

@BrettClimb 5 лет назад

I kinda got it working by putting this code inside plt.imshow: (imgs[i] * 255).astype(np.uint8)

@fit_with_a_techie 4 года назад

Thanks for the video. I have not downoaded the dataset and directly using it from keras.dataserts... How can I split my training, testing and validation data into batches, as I don't have any path of the directories?

@deeplizard 4 года назад

See the later episode where we download a data set and use a script to organize it into the respective directories. You will need to change the code for the relevant categories, but you can get the general idea from the episode below. Otherwise, you can manually separate the data. deeplizard.com/learn/video/FNqp4ZY0wDY

@datascience_with_yetty 6 лет назад

Is there a github link to this code? Great Tutorials!!!

@deeplizard 6 лет назад

Thanks, Sanni! The code files are available as a perk for the deeplizard hivemind at the following link: www.patreon.com/posts/code-for-keras-1-19266488 Check out the details regarding deeplizard perks and rewards at: deeplizard.com/hivemind

@kareemjeiroudi1964 5 лет назад

So both the source code as well as the images are available one Patreon??

@deeplizard 5 лет назад

Hey kareem - yes! code: www.patreon.com/posts/code-for-keras-1-19266488 images: www.patreon.com/posts/cat-and-dog-data-19555349

@franciscaifebuzor7851 5 лет назад

hello, I followed the exact instruction, but I am getting some that states "found 0 images belonging to 2 classes".. any idea how I can solve this?

@deeplizard 5 лет назад

Hey Francisca - This means Keras isn't seeing any image files inside of your cat and dog folders. Can you confirm that inside of each of the train, valid, and test folders that you have a cat and a dog folder, and inside each cat and dog folder, you have image files of cats and dogs inside?

@muzammilayaz4352 5 лет назад

mam should the image be in tat size(244,244) or it ll take only tat portion as region of interest?

@deeplizard 5 лет назад

No, the image can be any size, but then Keras will resize it to the target_size parameter that you set, which in our case is 224 x 224.

@muzammilayaz4352 5 лет назад

@deeplizard ma'am one more doubt.....I used a tool called labelImg to label the image...but it's output is in XML format nd has only 5 values in it....what shld I do...though if I convert it into numpy array y 5 values?

@nadaalay4528 6 лет назад

Should I classify the test data into two folders? Can I put all test data in one folder?

@deeplizard 6 лет назад

Hey Nada - Yes, you can put all the test data into one folder. This is usually the case if you don't know the labels for your test data. If you don't know the labels for the test data, you need to have one more directory within the "test" folder. Call the nested directory "unknown" for example. Then, within this unknown folder, place all the unlabeled test images there. With this structure, you will need to modify the test_batches variable. The parameters called classes and class_mode within this variable will need to be changed. Specifically, the change will be to make classes = None and class_mode = None. This is due to not knowing the labels for the test data.

@nadaalay4528 6 лет назад

deeplizard Thank you for replying 💓

@aamirmustafa7731 5 лет назад

Is the code available anywhere so that we can copy it simply. Thanks

@deeplizard 5 лет назад

Hey Aamir - Download access to code files and notebooks is available as a perk for the deeplizard hivemind. Check out the details regarding deeplizard perks and rewards at: deeplizard.com/hivemind If you choose to join, you will gain access to the notebook from this video at the link below: www.patreon.com/posts/code-for-keras-1-19266488

@HalkerVeil 5 лет назад

What if it's a bunch of pictures of Wolf Blitzer?

@matildanordahl3500 4 года назад

When I try to print the images I get black squares.... I don't get an error message just black squares (with the classification above as it should be). I have done this on my own sets of pictures and have 6 classes - have I missed anything of the "show pic in notebook"-part that I need to change?

@charank7718 3 года назад

I encountered the same issue @Matilda Nordhal. Did you find the solution?

@CarlosRincon 3 года назад

Quick question: in the code, at what point you made dogs equal to 1 and cats equal to 0? whats that done automatically? thanks

@deeplizard 3 года назад

Hey Carlos - It happens automatically. I elaborate on how this is done in the later episode below: deeplizard.com/learn/video/pZoy_j3YsQg

@CarlosRincon 3 года назад

@@deeplizard thanks mate!

@oussamaamara9931 5 лет назад

i got this error : Could not import PIL.Image. The use of `array_to_img` requires PIL. for the line imgs,labels=next(.... i need help lpz

@deeplizard 5 лет назад

The error is stating that you don't have the Python Imaging Library (PIL) installed. This is a core library for image manipulation in Python. To install it it, run pip install Pillow from the command line. Then, shutdown and restart your Jupyter notebook, and run the code again.

@DEEPAKSV99 4 года назад

@@deeplizard Thanks :-)

@nikhilramesh7105 4 года назад

Hi, I am getting an error while running this line : imgs, labels = next(train_batches) The error is TypeError: 'ImageDataGenerator' object is not an iterator Can somebody help me out??

@devadinesh1609 4 года назад

add this from keras.preprocessing.image import ImageDataGenerator

@MrDots99 4 года назад

What is the purpose of the valid folder?

@deeplizard 4 года назад

Hey Aidan - See the earlier video in the series on validation sets. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-dzoh8cfnvnI.html

@negarolfati36 4 года назад

Hi! I get syntax error from the function part. (for i in range(len(ims)): ^ Does anybody know what the problem is?

@deeplizard 4 года назад

Hey Negar - You shouldn't have a parenthesis at the start of your for loop. I believe that is what is generating the syntax error.

@seiftarek9122 5 лет назад

how to add 81 classes in the batch ? i cant copy all of their names in the {classes}

@deeplizard 5 лет назад

Hey seif - If you have your directory structure set up for your 81 classes like I have in the video, then you don't have to specify the classes parameter when you define the batches. Keras will automatically infer what the classes are based on the directory structure. You can make sure that Keras is inferring correctly by calling train_batches.classes after you define train_batches.

@namratabalachandra7616 6 лет назад

Hey I have two classes buildings and non buildings and everything is getting classified as building can you help me out !

@deeplizard 6 лет назад

Hey namrata - You may need to add more data if your data set is small. Additionally, have a look at this: stackoverflow.com/questions/34796596/machine-learning-how-to-identify-if-there-is-no-object-of-trained-classes-in-im

@namratabalachandra7616 6 лет назад

deeplizard thank you !

@ricardojunior9987 6 лет назад

For get the source code is need to pay?

@deeplizard 6 лет назад

Hey Ricardo - Code files and notebooks are available as a perk for the deeplizard hivemind. Check out the details regarding deeplizard perks and rewards at: deeplizard.com/hivemind If you choose to join, you will gain access to the notebook from this video at the link below: www.patreon.com/posts/code-for-keras-1-19266488 Just for clarity, note that all of the code is shown in the videos, so the code itself is freely available. The convenience of downloading the pre-written organized code files is what is available as a reward.

@ratanrohith1013 6 лет назад

Mam after encoding labels how can I decode them. Can you provide me code for that??

@deeplizard 6 лет назад

Hey Ratan - Check out this video, and let me know if it answers your question. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pZoy_j3YsQg.html

@ratanrohith1013 6 лет назад

Hmm unfortunately no madam. That's showing corresponding class names to labels. But I want the result as decode labels means for example the output class belongs to cats or dogs but not their encoded values.

@deeplizard 6 лет назад

I see. I'm not aware of a way to retrieve this directly. I believe you would need to first access the encoded vector of a given prediction/output, and then write the logic to map that encoded value to the class name using the class_indices parameter discussed in the video I linked to.

@ratanrohith1013 6 лет назад

OK mam I will try it. Thanks mam

@Viekash2001 4 года назад

WHERE CAN I GET THE DATASET FROM

@deeplizard 4 года назад

Check out the section called "Obtaining the data" in the corresponding blog for this video. There's a link to the data set. deeplizard.com/learn/video/LhEMXbjGV_4

@JimmyCheng 5 лет назад

Hey deeplizard, where can I download these images for testing purposes?

@deeplizard 5 лет назад

Hey Ziqiang - The images I'm using are a random subset of this data set: www.kaggle.com/c/dogs-vs-cats/data

@JimmyCheng 5 лет назад

@@deeplizard Thank you so much!

@JimmyCheng 5 лет назад

@@deeplizard hmmm somehow I am not able to download it, even though I have agreed to the terms. also the file is pretty big at 800+MB, is there a way to get the sets you're using

@JimmyCheng 5 лет назад

@@deeplizard ah problem solved, just signed up at patreon and downloaded the file!

@deeplizard 5 лет назад

I see that! Thank you so much! 💗