Тёмный

How to Train custom Object Detection Neural Network using TensorFlow 2.3 on Google Colab Free GPU 

When Maths Meet Coding
Подписаться 14 тыс.
Просмотров 67 тыс.
50% 1

#tensorflow #objectdetection #computervision
If you like the video, please subscribe to the channel by using the below link
tinyurl.com/1w5i9nnu
Link for my deeplearning udemy course coupon code added
www.udemy.com/course/linear-r...
Hi Everyone, In this tensorflow tutorial, I have explained how to train tensorflow object detection api with your own data. I have used tensorflow latest version here. For
model training, I am using google colab free GPU. It means it’s a full-on tutorial on how to train object detection api with your own data or custom object detection model on google colab.
I have taken the example of playing card detection in this video and the model is taken from tensorflow model zoo.
----Time links to each step in the video----
Chapters:
0:00 Introduction
3:24 Install TensorFlow object detection api using powershell,Set up Object Detection directory and python virtual environment at once
6:30 Gather and label pictures
13:51 Generate training and testing dataset
16:39 Create train tfrecord and test tfrecord files
20:23 Create label map and configure training
28:55 Setup google colab for object detection model training
33:20 Start model training on colab
33:56 Export inference graph
36:57 Try out your object detector for images.
39:46 Try out your object detector on a live webcam.
1. Github ripo link - github.com/jakkcoder/widows-o...
this contains:
a. PowerShell script to install and set up tensorflow object detection api.
b. generate_labelmap.py :- to create labelmap file for object detection.
c. generate_tfrecord.py :- to create tfrecord file for training and testing data.
d. object_detection_tutorial.ipynb:- run the inference file for live cam object detection.
e. xml_to_csv.py :- Generate the csv file for training and testing images
2. Download labelImg tool for this link :- tzutalin.github.io/labelImg/
3. run this command for generating csv file for training and testing images
python xml_to_csv.py
4. run this command for generting labelmap.pbtxt file
python generate_labelmap.py
5. Generate tfrecord file for training by this command
python generate_tfrecord.py --csv_input=images/train_labels.csv --image_dir=images/train --output_path=train.record
6. Generate tfrecord file for training by this command
python generate_tfrecord.py --csv_input=images/test_labels.csv --image_dir=images/test --output_path=test.record
7. Here are the argument to be updated on the config file for model training
num_classes: 5 [give number of classes here]
learning_rate_base: 0.8e-3
warmup_learning_rate: 0.0001
fine_tune_checkpoint: "efficientdet_d0_coco17_tpu-32/checkpoint/ckpt-0"
fine_tune_checkpoint_type: "detection"
label_map_path: "images/labelmap.pbtxt"
input_path: "train.record"
label_map_path: "images/labelmap.pbtxt"
input_path: "test.record"
8. train model command
python model_main_tf2.py --pipeline_config_path=training/ssd_efficientdet_d0_512x512_coco17_tpu-8.config --model_dir=training --alsologtostderr
9. export infrence graph command
python exporter_main_v2.py --trained_checkpoint_dir=training --pipeline_config_path=training/ssd_efficientdet_d0_512x512_coco17_tpu-8.config --output_directory inference_graph

Опубликовано:

 

11 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 264   
@MahmoudMahmoud-xo4xe
@MahmoudMahmoud-xo4xe 3 года назад
I can tell you had a lot of fun making this. Thanks for the tutorial and the easy setup file!.
@ashwinbabu4113
@ashwinbabu4113 3 года назад
i have been searching for good object detection video for so long. its really great, I loved it. thankyou
@susantasharma
@susantasharma 3 года назад
One of the complete video on this topic. Thanks for sharing.
@zanuarer4019
@zanuarer4019 3 года назад
It is helped me a lot, Thank you very much! I trained using faster-rcnn and tensorflow 2.
@thekunafamuncher1173
@thekunafamuncher1173 3 года назад
Thanks Jay! Fantastic tutorial!
@JemmyWu
@JemmyWu 3 года назад
Awesome! It's indeed the best guide for learning Colab TF2. Thank you so much!
@whenmathsmeetcoding1836
@whenmathsmeetcoding1836 3 года назад
Thanks
@petergoergen9991
@petergoergen9991 3 года назад
Das funktioniert super, vielen Dank!
@kenzilamberto1981
@kenzilamberto1981 3 года назад
Your videos are best at explaining, thank you brother
@saidmaroun7272
@saidmaroun7272 3 года назад
thank you very your explanations, the video was so great and well presented. keep up the great work
@giorgosiotis1557
@giorgosiotis1557 3 года назад
You are Great. Thank YOU VERY MUCH.
@amandal6943
@amandal6943 3 года назад
Thank you so much you are amazing!!!!
@vinayakdumbre2828
@vinayakdumbre2828 3 года назад
Thank you sir, actually it helped me.
@geraldmusandirire5446
@geraldmusandirire5446 3 года назад
Best video online !!!!! Thank you !
@rafifrahman
@rafifrahman 3 года назад
Great Tutorial! Thank you so much😍
@whenmathsmeetcoding1836
@whenmathsmeetcoding1836 3 года назад
Glad it was helpful!
@utku_yucel
@utku_yucel 3 года назад
Thanks for the video!
@m.imamfaruqbudiman2019
@m.imamfaruqbudiman2019 3 года назад
Thank's you very muchh AWESOME !!
@sakshamp4488
@sakshamp4488 2 года назад
amazing videos sir !!!
@objectdetection4636
@objectdetection4636 3 года назад
Much Respect! great tutorial
@gogakakabadze
@gogakakabadze 2 года назад
You are super bro, really super, thank you very much !
@wiyadayawai9083
@wiyadayawai9083 3 года назад
Thank you so much. :)
@Sandeepsharma-mc1er
@Sandeepsharma-mc1er 3 года назад
After trying many tutorials this tutorial helps me lot !! Nicely explained!!
@rahulitsrahull
@rahulitsrahull 3 года назад
Hello Jay.... this is a great tutorial. If you could upload a video on how to convert the saved pb file into TFlite model, it would be much helpful. Thanks
@leamon9024
@leamon9024 3 года назад
Thanks for all the hard work and the effort, Jay. One question: In tensorflow object detection API, I know I can start training my model from scratch by running command like python ./object_detection/model_main_tf2.py --pipeline_config_path=/content/drive/MyDrive/Colab_SSD_Facemask/ssd_resnet50_v1_fpn_640x640_coco17_tpu-8_colab.config --model_dir=/content/drive/MyDrive/Colab_SSD_Facemask/training_process --alsologtostderr But how do I re-train my model from the last trained weights so that I don't have to start over everytime if there's an interrupt during the training? Say I have ckpt-1.index, and ckpt-2.index, which are generated during last training, already.
@nicolasachury9217
@nicolasachury9217 3 года назад
First of all, thanks for the amazing tutorial!!! I would like to ask you the following: I trained my model using Google Colab and I interrupted the processes after 30 min. Now the results are not what I was expecting, so I want to train the model much longer. How can I resume the training process using the checkpoints? any insight from you would be highly appreciated!
@chinmaysathe23
@chinmaysathe23 3 года назад
Thanx for the upload ❤️
@RakeshKumar-bm2dr
@RakeshKumar-bm2dr 3 года назад
hello please help me. I am facing a error tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 Traceback (most recent call last): File "model_main_tf2.py", line 32, in from object_detection import model_lib_v2 ModuleNotFoundError: No module named 'object_detection' please help me i am new in python
@NusratJahan-jw7vk
@NusratJahan-jw7vk 3 года назад
thanks a looootttttttttttttttttttttttttttttttttttttttttttttttt......you are the best
@whenmathsmeetcoding1836
@whenmathsmeetcoding1836 3 года назад
Welcome 😊
@farazahmad9079
@farazahmad9079 2 года назад
Thanks
@whataday3910
@whataday3910 3 года назад
Can you do a video about good training parameters for Yolo? Maybe you can explain the Yolo Cfg and tell us what parameter is especially important for learning and good detection
@gaaloulmohamed5535
@gaaloulmohamed5535 3 года назад
BigWork & BigRespect for the powershell installation Thanks
@francescovender9886
@francescovender9886 2 года назад
Hi, can i know what is the problem with this? When installing the Visual C++, it keeps telling me that I have missing packages or damaged packages for setup. I clicked either option for downloading the packages but to no avail. Can you help me on this?
@saravananbalaguru1065
@saravananbalaguru1065 3 года назад
Hi, Nice details on object detection. Is there a way to count the objects detected in image and/or in a video/Camera using the Custom training model. Thanks Saravanan
@robinrai341
@robinrai341 2 года назад
thanks for amazing tutorial sir i trained the image up to 9000 steps with minimum loses but not detecing the image it just show the test images
@annannhat3761
@annannhat3761 3 года назад
thank you for the tutorial, my loss=nan after a few minutes of training, what could be the cause of this problem?
@anilkoluacik
@anilkoluacik 3 года назад
setup file is either missing or damaged while installing vs c++ build tools. Pls help me to handle this issue.
@gautamkinfo
@gautamkinfo 3 года назад
the power sell is does not working at my system. it's open power sell window and then crash...
@ImranAli-qs1xn
@ImranAli-qs1xn 3 года назад
@Jay Bhatt will the power shell create a new environment ...and what does happen when python already installed on computer ???
@othmanali4232
@othmanali4232 3 года назад
thank you for the helpful tutorial, how can i re-train without having to compress and upload again?
@sheetal3955
@sheetal3955 3 года назад
Hi Jay! while doing Step 9. Try out your object detector for images. I am facing this error-- in the line -- "detection_model=tf.saved_model.load('C:/Users/dell/Documents/Tensorflow/models/research/object_detection/inference_graph-20201219T121406Z-001/inference_graph/saved_model')" the error is-- "FailedPreconditionError: Error while reading resource variable WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalBoxHead/BoxPredictor/kernel_159624 from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/WeightSharedConvolutionalBoxPredictor/WeightSharedConvolutionalBoxHead/BoxPredictor/kernel_159624) [Op:ReadVariableOp]"
@mr.millenniamachine1826
@mr.millenniamachine1826 3 года назад
Thank you. How to improve detection time to about 0.1 sec per frame? Please help.
@mihirkulkarni373
@mihirkulkarni373 3 года назад
hello sir ,its a very nice video , in my case labelimg is not opening pyrcc5 and pyqt5 both are present what should i do now
@shankarmohanakrishnan391
@shankarmohanakrishnan391 3 года назад
Can you tell me what is number of min training steps needed for this detection? I tried with 1000 steps but its not detecting the objects. Any idea ?
@annie157
@annie157 2 года назад
Hi, can you please guide me on how can I find the confusion matrix, if I am using ssd_mobnet model of object detection API?
@diegoperalta5023
@diegoperalta5023 3 года назад
Hello, sir. At the moment of making the detection of only 1 object in my image, there appear overlapping boxes. How can I fix this error? Great, perfect tutorial!! Greetings from México!
@joseantonioperalta8737
@joseantonioperalta8737 3 года назад
Why the training is not generating the meta files in the training folder? Only data and index
@jimmypark1791
@jimmypark1791 2 года назад
Also why do we need visual studio to run python in google colab?
@jiteshaneja3892
@jiteshaneja3892 3 года назад
Hi, Can we run Object detection more than one time on the same data-set, example:, for the first time my object detection algorithm will detect the only person in detection, and in the second step it will detect either the person in the last detection is wearing a mask or cap? if Yes then can u please help with how??
@johnurena2863
@johnurena2863 2 года назад
i get an error when trying to install visual studio ( a setup package is either missing or damaged)
@erickahumada4806
@erickahumada4806 3 года назад
Hi,. If I want to count the detected objects separately, how can I do it? I am detecting two objects an apple and a pineapple, and I want it to count each object separately.
@madhumalaghosh7885
@madhumalaghosh7885 2 года назад
ModuleNotFoundError: No module named 'tf_slim' , while running object_detection_tutorial.ipynb ... can you please guide me on this?
@iftikharzaidi1722
@iftikharzaidi1722 3 года назад
after running the powershell script. my protos folders is showing empty. Why?
@pavankumarpotta4565
@pavankumarpotta4565 3 года назад
I am setup the environment The visual studio package is not getting installed it is throwing me an error
@hanishlakhani9544
@hanishlakhani9544 3 года назад
Thanks for a wonderful tutorial. I was able train successfully on my PC but training same program i.e. "!python model_main_tf2.py -pipeline_config_path=training/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.config --model_dir=training -alsologtostderr" over colab threw a warning "tensorflow:Collective ops is not configured at program startup. Some performance features may not be enabled" and program stopped without training." Program stopped :( without training. Plz help.
@seharsana8867
@seharsana8867 Год назад
Can you please guide me...i am facing error IMPORTERROR:cannot import name 'model_lib_v2' from object_detection
@TheMrtaytac
@TheMrtaytac 2 года назад
when loading label map i take this error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 113: invalid start byte
@jhoanmartinezsilva2609
@jhoanmartinezsilva2609 3 года назад
How I label data from video? It seems you get data from your laptop cam, how you did it?
@akshayrajm5139
@akshayrajm5139 3 года назад
Getting this error during "detection_model = tf.saved_model.load('object_detection/inference_graph/saved_model')" any idea on how to fix this, am stuck with this. TensorBundle at object_detection/inference_graph/saved_model\variables\variables shard 0 (331442 bytes): Checksum does not match: stored 3958850664 vs. calculated on the restored bytes 759939483
@marianabuhattoum
@marianabuhattoum 2 года назад
33:15 I got "Invalid argument: input must be 4-dimensional[1,1,600,800,3] " how can I change the dimensions ?
@learnislam9577
@learnislam9577 2 года назад
hello sir. i don't have a GPU system should I run this on my laptop?
@tirthankarroy2906
@tirthankarroy2906 2 года назад
I tried running the script but it doesn't seem to work,it keeps closing on its own.Is there any specific reason for that?
@malikgaming5715
@malikgaming5715 3 года назад
self._read_buf = _pywrap_file_io.BufferedInputStream( esearch\object_detection\images\labelmap.pbtxt : The filename, directory name, or volume label syntax is incorrect.ole_Detection_FYP\models ; no protocol option facing this error please help
@sujanpaul99
@sujanpaul99 3 года назад
How can i convert this inference graph to tflite? Please help me. I am stucked in my college project
@duseokchoi2312
@duseokchoi2312 Год назад
anyone know whats idffernece between saved model under ssd..-tpu and inference graph/saved_model
@maazanis3998
@maazanis3998 Год назад
can you tell me how we can make tfjs file of this model so I can use this in react native
@blackpearlaxom6261
@blackpearlaxom6261 3 года назад
Please help... Got this error after running "!python model_main_tf2.py --pipeline_config_path=training/ssd_efficientdet_d0_512x512_coco17_tpu-8.config --model_dir=training --alsologtostderr" tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for efficientdet_d0_coco17_tpu-32/checkpoint/ckpt-0
@ayatkhaled5086
@ayatkhaled5086 2 года назад
hello please i nedd some help i got this error while iam run the detection code and i really want to get a solution tensorflow:Importing a function (__inference_bifpn_layer_call_and_return_conditional_losses_58639) with ops with custom gradients. Will likely fail if a gradient is requested. i am using raspberrypi 4
@kokebdese4787
@kokebdese4787 3 года назад
How can I follow the training process on tensor board?
@Revaldoi
@Revaldoi 2 года назад
can I run and create dataset model on macos?
@taqikhan5418
@taqikhan5418 2 года назад
Can i use it for real time face recognition?
@hafizasad8410
@hafizasad8410 3 года назад
I keep getting this issue every time I'm trying to train. File "C:\Python37\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 80, in _preread_check compat.path_to_str(self.__name), 1024 * 512) esearch\object_detection\images\labelmap.pbtx : The filename, directory name, or volume label syntax is incorrect._detection\models-master ; no protocol option
@nehemiahmaddela5011
@nehemiahmaddela5011 3 года назад
sir, i got many exceptions while installing through powershell, can you pls help
@dunnasuryanarayana5918
@dunnasuryanarayana5918 3 года назад
Could any one explain how to freeze the weights of feature extractor while fine tuning?
@zanuarer4019
@zanuarer4019 3 года назад
Sir, how to see the accuracy of the model?
@kaveeshamadhubhashana3094
@kaveeshamadhubhashana3094 2 года назад
. Please help me to fix this error .errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for efficientdet_d0_coco17_tpu-32\checkpoint\ckpt-0
@ghaithkattan8597
@ghaithkattan8597 2 года назад
hello thank you for that video... wanna ask how can i print category name (class name)
@muhammedsafeer8415
@muhammedsafeer8415 3 года назад
sir, how to convert this trained model into tensorflow js
@mucahityuksel4196
@mucahityuksel4196 3 года назад
AttributeError: module 'tensorflow' has no attribute 'contrib' help!
@burhanuddinlimdiwala6754
@burhanuddinlimdiwala6754 3 года назад
Hi sir after inference graph how to create frozen model
@akindperson22
@akindperson22 3 года назад
PowerShell script is not not working on my laptop. When I open the script, it is showing a blank PowerShell Script. Anyone please help?
@spicytuna08
@spicytuna08 2 года назад
Did anyone get "cuDNN failed to iniialize error when trying to train?
@user-lq8be1fx6v
@user-lq8be1fx6v 2 года назад
tensorflow.python.framework.errors_impl.AlreadyExistsError: Another metric with the same name already exists. got this error while running on Colab, can you do something?
@sattysattu
@sattysattu 3 года назад
Good tutorial. Is it also possible to get x, y, w, h of the bounding boxes from this library?
@zoroXgamings
@zoroXgamings 3 года назад
you can get them by printing the detection_boxes
@lakshit6429
@lakshit6429 3 года назад
Hey that’s well explained but I have one question, If I want to store the captured cards into csv in sequence of their detection how can I do that???
@zoroXgamings
@zoroXgamings 3 года назад
open a excel file inside the detection function and print the detection values , you can check how to write data in excel file using python
@berndriedel563
@berndriedel563 2 года назад
That´s great! I managed to follow all steps. The only issue I have is that my GPU is not really used for training. Only 1-3% of my 3080 are used. Any hint?
@waqarahmed2866
@waqarahmed2866 2 года назад
Did you find a fix for this issue?
@RakeshKumar-bm2dr
@RakeshKumar-bm2dr 3 года назад
I am getting this error please help me 2021-05-22 09:02:45.459641: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0 Traceback (most recent call last): File "model_main_tf2.py", line 32, in from object_detection import model_lib_v2 ModuleNotFoundError: No module named 'object_detection'
@lajinsaji8863
@lajinsaji8863 3 года назад
I'm getting an error while training that says: 2021-03-16 08:38:07.184481: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 392 of 2048 2021-03-16 08:38:17.643263: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 696 of 2048 2021-03-16 08:38:27.345481: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:177] Filling up shuffle buffer (this may take a while): 726 of 2048 ^C Then the training gets automatically stopped. Can anybody pls tell me what the cause could be?
@user-id3qp7bq9o
@user-id3qp7bq9o 3 года назад
Well explained!! Great job. Adding a python file for measuring model accuracy, would be great addition to the GitHub Repo.
@deeppatel7779
@deeppatel7779 3 года назад
Well we r getting only loss information how to get Accuracy information
@stevencarrell83
@stevencarrell83 3 года назад
@@deeppatel7779 mAP mean average precision
@noureldienosama1545
@noureldienosama1545 3 года назад
Do I need to have git installed before running the jupyter notebook, because I am getting an error, says: 'git' is not recognized as an internal or external command, operable program or batch file.
@whenmathsmeetcoding1836
@whenmathsmeetcoding1836 3 года назад
yes sorry I missed out it in the video you should have git installed
@GSanthosh8995
@GSanthosh8995 3 года назад
How did you generate Powershell
@shubhamrajendraingole3439
@shubhamrajendraingole3439 Год назад
can u update the power shell please
@zmonx_gg
@zmonx_gg 2 года назад
I have the question. In last step of the train model on colab. I want to know how to measure or find the accuracy of that model?
@nourheneaouidi8383
@nourheneaouidi8383 2 года назад
did you found out how ?
@85krajek
@85krajek 3 года назад
hi @When Maths meets coding , When i run your script, i get this msg and script turn itself off.. any idea? Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
@deepangshudas8674
@deepangshudas8674 3 года назад
put the python path above the windows app path
@TOPTOP-wp3wm
@TOPTOP-wp3wm 3 года назад
It does not work for me in real time ERRORS: 'numpy.ndarray' object has no attribute 'seek' and 'numpy.ndarray' object has no attribute 'read' .Help me please
@rumiisufi1187
@rumiisufi1187 4 месяца назад
The file that you have provided has error :(
@spicytuna08
@spicytuna08 2 года назад
there is so much dependencies to this project. it is like trying to finding a needle in a farm when something breaks. i hope this is not too much to ask. do you have something that we can load a jupyter notebook from your github account to colab and make the same thing. this method is building from windows and importing to colab. can we build straight from colab? is there a reason why this won't work? sorry if i asked you a stupid question. it looks like colab was not designed to work for TF Object detection get go. So this seems like a hack to make Colab to work for TF Object Detection. Why COlab did not make it TF Object Detection friendly in the first place?
@rinolaharto9281
@rinolaharto9281 2 года назад
Thank you, Jay..this is awesome...I have subscribed and like your channels..please I have a problem with this.... tensorflow.python.framework.errors_impl.NotFoundError: images\labelmap.pbtxt; No such file or directory please help me..
@crackITnow
@crackITnow 3 года назад
Enable to download efficiendent_d0_coco17_tpu-32 Can anyone help plzz 🙏 ASAP...
@jimmypark1791
@jimmypark1791 2 года назад
Thank you for nice power shell script. while installing i got an error that the following file is damaged or can not be found. how can i solve this problem ? C:\temp_provision\packages\BuildTools_MSBuild_amd64\\BuildTools_MSBuild.msi
@deepakkumaryadav2381
@deepakkumaryadav2381 3 года назад
sir please make a video "A Fire Detection Using ANN AND FPA Algorithm" on machine learning using python
@deepakkumartahsildaryadav7505
@deepakkumartahsildaryadav7505 3 года назад
Yes sir please this topic cover
@giorgosiotis1557
@giorgosiotis1557 3 года назад
I tried to run Install-TensorFlow-Python3.7.ps1 script on windows 10 but this script says errors. It never reached to complete the installation. what may be the reason ??
@pavankumarpotta4565
@pavankumarpotta4565 3 года назад
me too the setup is not getting installed atleast
@tjtj1122
@tjtj1122 3 месяца назад
Python genrade error please kindly help me
Далее
Цены на iPhone и Жигули в ЕГИПТЕ!
50:12
Train a custom object detection model using your data
12:10
100+ Linux Things you Need to Know
12:23
Просмотров 712 тыс.
YOLOv5 training with custom data
20:29
Просмотров 412 тыс.
TensorFlow in 100 Seconds
2:39
Просмотров 912 тыс.