Тёмный

How to Train DETR Object Detection Transformer on Custom Dataset 

Roboflow
Подписаться 37 тыс.
Просмотров 25 тыс.
50% 1

In this tutorial, we'll show you how to train Object Detection Transformers using DETR as an example. We'll guide you through every step of the process, starting with setting up your Python environment and demonstrating DETR model inference on example images.
Next, we'll show you how to download custom datasets from Roboflow Universe and build custom PyTorch COCO Detection datasets. We'll also cover how to visualize COCO dataset entries and build custom PyTorch data loaders.
At the core of this tutorial, we'll demonstrate how to build a custom PyTorch Lightning DETR module for training your own object detection model. We'll then guide you through training DETR on your custom dataset, followed by custom DETR model inference and evaluation.
Chapters:
0:00 Introduction
1:06 Setting up the Python environment
3:19 DETR model inference on example images
8:40 Download custom dataset from Roboflow Universe
10:31 Building custom PyTorch COCO Detection datasets
12:25 Visualising COCO datasets entry
13:15 Building custom PyTorch Data Loaders
15:59 Building custom PyTorch Lightning DETR Module
18:50 Training DETR on custom dataset
20:06 Custom DETR model inference
22:07 Evaluating custom DETR model
22:40 Outro
Resources:
🌏 Roboflow: roboflow.com
🌌 Roboflow Universe: universe.roboflow.com
🗒️ How to Train DETR with 🤗 Transformers on a Custom Dataset notebook: colab.research.google.com/git...
🤗 Transformers DETR documentation: huggingface.co/docs/transform...
🎬 Exploring The COCO Dataset RU-vid video: • Exploring The COCO Dat...
⌛ PyTorch Data Loader documentation: pytorch.org/docs/stable/data....
⚡PyTorch Lightning Module documentation: pytorch-lightning.readthedocs...
Stay updated with the projects I'm working on at github.com/roboflow and github.com/SkalskiP! ⭐

Наука

Опубликовано:

 

30 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 131   
@soumenhalder2811
@soumenhalder2811 4 месяца назад
Best ever Deep Learning video I have found. please make such video
@abdshomad
@abdshomad Год назад
When i see notification appeared from Roboflow, I always press like button before watching the show 👍😁
@abdshomad
@abdshomad Год назад
Always quality content.
@Roboflow
@Roboflow Год назад
@@abdshomad thanks a lot! I’m so happy that someone is clicking on those notifications 🔔
@sihameghaprabashwara6025
@sihameghaprabashwara6025 Месяц назад
​@@RoboflowIs there any way to detect in a video ?
@leandrodiascarneiro5663
@leandrodiascarneiro5663 Год назад
Excellent video. Thanks!!
@Megan-gl7pi
@Megan-gl7pi 3 месяца назад
Great tutorial. I'm going to train Detr on a custom medical dataset with x-ray images to detect bolus, stent, catheter, etc. Hopefully this will turn out to be fruitful. There hasn't been much research using Detr on medical datasets yet.
@body1024
@body1024 Год назад
amazing tutorials keep it coming
@Roboflow
@Roboflow Год назад
Thanks a lot! Stay tuned. I already have idea for next one ☝️
@MiaLovEm
@MiaLovEm Год назад
Thank you for such a great tutorial! I have learnt a lot. And I also watched other videos you made. They really helped me understanding the models and training process better. About the false detection you mentioned, I came acorss similar problems on my dataset. I'm thinking maybe add null images to the dataset (about 10%) could help? I am just about to try!
@AmarnathReddySuarapuReddy
@AmarnathReddySuarapuReddy 4 месяца назад
Great lesson
@ComedyArchives
@ComedyArchives 4 месяца назад
great video, thanks!
@Roboflow
@Roboflow 4 месяца назад
Pleasure!
@raymond-andrade
@raymond-andrade Год назад
Great Video. I resonate with your intro, on how it's hard not to get imposter syndrome in this fast changing AI/ML landscape, when you don't know how to implement the latest "thing".
@Roboflow
@Roboflow Год назад
Thanks a lot. At the same time there is nothing better than just to face your latest “thing” and learn!
@tomaszbazelczuk4987
@tomaszbazelczuk4987 Год назад
Piotr thanks a lot👍
@Roboflow
@Roboflow Год назад
Our viewership in Poland 🇵🇱 grew a lot over the last few months.
@DrShroom.
@DrShroom. Год назад
Cool!
@DigiDriftZone
@DigiDriftZone Год назад
What is the impact of small batch sizes on inference accuracy? - I only have 8gb of vram on my 3060ti and for the larger models I’m only getting batch sizes of 1 or 2.
@helloansuman
@helloansuman Год назад
Nice video. Can you please tell how to run the training on multi-gpu? Which lines to be added or modified?
@anaisabelrua189
@anaisabelrua189 Год назад
Loved it!! have you tried using DETR with Detectron2? BTW, it wasn´t long for me. Very precise learned a lot!
@Roboflow
@Roboflow Год назад
Thanks a lot for that positive feedback. I actually didn’t trained DETR with Detectron2. I came a bit late to the party and went straight with Transformers.
@ALOKAGIMHARI
@ALOKAGIMHARI 2 месяца назад
Thank you for your tutorial.❤ Let me know how to use this trained model for test an image thats not in test dataset.
@user-dk2rx7wu2k
@user-dk2rx7wu2k Год назад
How do I use my saved pre-trained model in a new laptop with my own weights?
@aaminataskeen3448
@aaminataskeen3448 8 месяцев назад
Hi, very informative video! I’ve been working with DETR for mitosis object detection and your code has been of great help. But when im training the model, the loss is decreasing very very slowly. Like even after 50 epochs my loss has gone down from 3.7 to 3.1 . So I was thinking maybe it’s coz of the learning rate. Is there any way I could change the learning rate coz when I tried doing that during DETR class instantiation, it threw an error while error. So as of now I can’t change the learning rate and my loss is almost stabilized at 3 and just fluctuating between the above mentioned values. Appreciate your reply :))
@michaelscheinfeild9768
@michaelscheinfeild9768 Год назад
HI THANKS FOR VIDEO. Where i can read article of model
@user-js1tx4wu4p
@user-js1tx4wu4p 9 месяцев назад
annotations are not found during the evaluation process any way out for this model
@Roboflow
@Roboflow 9 месяцев назад
Would you be a bit more specific?
@chrisboutsikas5973
@chrisboutsikas5973 22 дня назад
Is there reason in particular why you install a spesific verison of supervision (i.e. supervision==0.3.0) ?
@arfashaha4074
@arfashaha4074 Год назад
Thank you so much for the great video! Your quality content has helped me a lot in my thesis ☺. Just wanna ask, can we apply our trained model to predict a video/webcam directly?
@Roboflow
@Roboflow Год назад
Not entirely “directly” but you can use tools in Supervision to process video
@arfashaha4074
@arfashaha4074 Год назад
@@Roboflow Ah I see, is the concept the same as your video about detect and count object in polygon zone?
@Roboflow
@Roboflow Год назад
@@arfashaha4074 yes exactly! Generate frames from video. Process frame. Annotate. Save to result file.
@arfashaha4074
@arfashaha4074 Год назад
@@Roboflow Perfect, thanks for the help! You've been wonderful
@Roboflow
@Roboflow Год назад
@@arfashaha4074 thanks a lot for kind words!
@lorenzoleongutierrez7927
@lorenzoleongutierrez7927 Год назад
great !
@Roboflow
@Roboflow Год назад
Thanks a lot! 🙏
@lorenzoleongutierrez7927
@lorenzoleongutierrez7927 Год назад
@@Roboflow, really thanks to all of you for the teaching and truly democratization of this knowledge changing the world. As an user following almost 3 years ago the evolution of roboflow, it seems to me incredible how well is directed the whole ecosystem service and clear information for so many people (Including me, as an agronomist interested in locating weeds and crops!). a Big Bravo from Chile and have a great weekend!!
@Roboflow
@Roboflow Год назад
@@lorenzoleongutierrez7927 This is so kind! Thanks a lot 🙏I'll pass your message to the rest of the team.
@SalmanFarsi-qt6fr
@SalmanFarsi-qt6fr Год назад
I am getting the 'Key Error'='annotations' And Type of Error: Detections.___Getitem__ not supported for index of type .... Please Do guide me with the following error how can i come up with it. Will really appreciate it.
@canciones381
@canciones381 4 месяца назад
It is possible mix this training custom data set with transformers with the PolygoneZone tool? Great Video!
@bharanis1004
@bharanis1004 3 месяца назад
hey any idea on how to use pretrained detr
@seroshmannan
@seroshmannan Год назад
how can we tweak with network block
@felipegerard1613
@felipegerard1613 4 месяца назад
Hi, first thanks for this great work, I am having a problem if I try to create this notebook on my development server, when I use "inputs=image_processor(images=image,return_tensor='pt').to(DEVICE)" (device(type='cuda', index=0)) I get in transformers/feature_extraction_utils.py:229 "TypeError: is_floating_point(): argument 'input' (position 1) must be Tensor, not list" any help will be very appreciated
@chandank5266
@chandank5266 2 месяца назад
Hey Roboflow bro! I found this vidio really helpfull and I have a question striking me: where to use the checkpoint, I want to load the model using checkpoint that got saved while logging inside lightning_logs and not by the config file saved in custom-model directory; is there a way?
@10straws59
@10straws59 Год назад
Thank you so much for this tutorial! I am getting this error: KeyError: 'annotations'. I think this is because "predictions" is an empty dict, so is "results". However "orig_target_sizes" is not empty. I have tried with 20 epochs, is it because the model is not learning well enough to be able to actually make predictions? Furthermore, I tried to annotate images in the train and val datasets, and I am not seeing any detected boxes around the objects (when comparing to the ground truth) Would rly appreciate some help regarding this issue!
@Roboflow
@Roboflow Год назад
Could you create new issue here: github.com/roboflow/notebooks/issues and describe all those issues.
@SalmanFarsi-qt6fr
@SalmanFarsi-qt6fr Год назад
Bruh i am also getting this type of error continuously please if you have come up with it guide me too. Will really appreciate ❤ Thanks
@nhauwu
@nhauwu Год назад
I tried to use it with my own model, but it said "NameError: name 'image_processor' is not defined" in the first code cell of "Create COCO data loaders
@Roboflow
@Roboflow Год назад
I’d need more details. Could you create Agnew thread here: github.com/roboflow/notebooks/discussions/categories/q-a and give me a bit more details?
@afriquemodel2375
@afriquemodel2375 6 месяцев назад
NameError: name 'image_processor' is not defined
@umarkhalid5463
@umarkhalid5463 2 месяца назад
Can this be applied to realtime surveillance cameras to detect shoplifting?
@muhdnajhan886
@muhdnajhan886 7 месяцев назад
what code for random run any image
@biswajitroy-zp6lk
@biswajitroy-zp6lk Год назад
TypeError: Detections.__getitem__ not supported for index of type . Hi this is a error i am getting when i am about to detect bounding box on the test dataset .i have one class
@Roboflow
@Roboflow Год назад
Hi 👋! Is that happening with the current version of the notebook without any changes?
@rezaazhar1292
@rezaazhar1292 Год назад
hi i wanna ask. can i deploy this model to mobile android or drone like dji tello?
@Roboflow
@Roboflow Год назад
You can convert the model to ONNX and in this format it is deployable on Android Phones. Maybe there are other ways, but I’m pretty sure this one should work.
@tongtongchen-qo6qj
@tongtongchen-qo6qj 26 дней назад
AttributeError: 'ToTensor' object has no attribute 'pad' that How to solve this problem ?
@user-mt4li6jt9m
@user-mt4li6jt9m 10 месяцев назад
AttributeError: type object 'Detections' has no attribute 'from_coco_annotation' I didn't find any raise issue on github or any stackOverflow so python``` sv.Detections.from_coco_annotation ``` and i get this error
@Roboflow
@Roboflow 10 месяцев назад
What version of supervision you use? Tutorial was dedicated for 0.3.0. from_coco_annotation got drop from supervision API.
@Thedrive17
@Thedrive17 Год назад
Thanks for the great video! Just wanna ask can we detect image from image that we upload in colab?
@Roboflow
@Roboflow Год назад
Thanks a lot. I’m not sure I understand. Could you rephrase the question?
@accongnetwork1208
@accongnetwork1208 Год назад
@@Roboflow Hi, may I ask you a question? How can we show the IoU after the detection?
@Roboflow
@Roboflow Год назад
@@accongnetwork1208 and I responded a month ago :) IoU of what? Detections and ground truth?
@Thedrive17
@Thedrive17 Год назад
⁠​⁠@@Roboflow Yes, the IoU of detections and ground truth, I dont know how to show the coordinat of detections, and Is there any code to show the IoU of Ground Truth and Detections automatically? Thank youu so much
@afriquemodel2375
@afriquemodel2375 6 месяцев назад
how to configure to work in local environnement
@MuhammadAdnan-tq3fx
@MuhammadAdnan-tq3fx 9 месяцев назад
Thank you so much for this video. i followed your video and received this results Accumulating evaluation results... DONE (t=0.16s). IoU metric: bbox Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.546 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.766 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.629 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.328 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.558 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.578 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.605 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.605 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.617 please guide me how i can increase my value upto 95%. i use same parameter that you have used in your video.i will be very thankful to you
@Roboflow
@Roboflow 9 месяцев назад
Did you used the same dataset we did?
@MuhammadAdnan-tq3fx
@MuhammadAdnan-tq3fx 9 месяцев назад
I used the "SeaShip" dataset, downloaded from the Roboflow website, which consisted of 7,900 training images and approximately 1,661 validation images. For both the train_dataloader and val_dataloader , I set the batch size to 4. training and model parameters was: model = Detr(lr=1e-4, lr_backbone=1e-5, weight_decay=1e-4) trainer = Trainer(devices=1, accelerator="gpu", max_epochs=50, gradient_clip_val=0.1, accumulate_grad_batches=8, log_every_n_steps=5) on these i received the results that i have uploaded in my first comment. I again starts a second training iteration with significant adjustments to hyperparameters. In this instance, I enlarged the training dataset to 9,843 images while simultaneously reducing the validation dataset to a mere 300 images., also increase the epochs up to 100 and for train_dataloader batch_size=4 and for valid_dataloader batch_size=2, also reduce the learning rate from lr=1e-4 to lr=1e-5. during first time total training time was 23 hours but this time my training is still running from last 18 hours it will take one more day i think.after that i will share the result again. and one more thing i have NVIDIA rtx 6000 (24GB)
@MuhammadAdnan-tq3fx
@MuhammadAdnan-tq3fx 9 месяцев назад
@@Roboflow Second time accuracy comes 81% how can i increase the accuracy.
@nimrakhan5785
@nimrakhan5785 11 месяцев назад
which object detection model is best for accuracy, Yolov8, dert, transofrmer or Faster RCNN?
@Roboflow
@Roboflow 10 месяцев назад
You only care about top accuracy? Speed does not matter? Or top accuracy in real time detection?
@iMJKhan
@iMJKhan 10 месяцев назад
@@Roboflow Thanks for the great content. How about each use case? 1. Accuracy, 2. Speed, 3. Accuracy in RT-detection, and 4. Speed in RT-detection
@sangamithramurugesan8178
@sangamithramurugesan8178 10 месяцев назад
Hello, the acuial purpose of DETR is to ELIMINATE the use of NMS. But your code uses NMS. why?. obtained from the original paper:"Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components like a non-maximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task"- PLEASE EXPLAIN
@Roboflow
@Roboflow 10 месяцев назад
The answer is pretty simple. I run the model and faced a lot of problems with double detections. Results were pretty much unusable without NMS and that’s why I added it.
@JUANG0MEZ
@JUANG0MEZ Год назад
hello good afternoon, is it possible to detect objects and that the bounding box has a rotation?
@Roboflow
@Roboflow Год назад
What do you mean by retention?
@raymond-andrade
@raymond-andrade Год назад
Generally if you want more accuracy than a horizontal bounding box, I feel most people opt to use Segmentation instead of Bounding Boxes, since Bounding Box is meant to just have XY center position with Width Height (to minimize model output needed to identify the object)
@Roboflow
@Roboflow Год назад
@@raymond-andrade I’m still not sure if that was the original question, but if so than I agree.
@mahachaabene6750
@mahachaabene6750 Год назад
great content! I have tried using it on my own dataset but the labels were not correct with the coco format, although i tried yolov5 and it did good . any explanation for that ?
@Roboflow
@Roboflow Год назад
Boxes were detected but with incorrect labels?
@mahachaabene6750
@mahachaabene6750 Год назад
@@Roboflow yes that"s exactly what happened
@mahachaabene6750
@mahachaabene6750 Год назад
my dataset contains numbers stamped on metal in a curved way, the model was detecting the boxes but with the wrong labels in this case, but it worked fine when i tried yolov5
@Roboflow
@Roboflow Год назад
@@mahachaabene6750 any pattern? Some numbers are always generated as other number? Maybe there is some mapping problem?
@mahachaabene6750
@mahachaabene6750 Год назад
@@Roboflow it seems like each number is detected as its successive number, like 1 is detected as 2, 3 detected as 4.....
@user-qh8zh3fx8n
@user-qh8zh3fx8n Год назад
Hi, thanks for your great video!! I am trying to train detr on lab server. I tried to set up the environment, but when I typed pip install -q ./transformers and pip install -q pytorch-lightning in terminal. I got the following errors ERROR: Invalid requirement: './transformers' Hint: It looks like a path. File './transformers' does not exist. Can you help me with this? Thanks a lot !!
@Roboflow
@Roboflow Год назад
I’m a bit confused. Why ./ before transformers? I believe it should not be there :
@user-qh8zh3fx8n
@user-qh8zh3fx8n Год назад
@@Roboflow oh, that's my fault ! Thank you!
@Roboflow
@Roboflow Год назад
@@user-qh8zh3fx8n no problem! 😉
@TheArkLade
@TheArkLade Год назад
Does anyone know why " __init__() got an unexpected keyword argument 'gpus'" error shows up when training with pytorch_lightning? It used to work...hummmm. Its this part: "trainer = Trainer(gpus=1, max_epochs=MAX_EPOCHS, gradient_clip_val=0.1, accumulate_grad_batches=8, log_every_n_steps=5)"
@Roboflow
@Roboflow Год назад
Did the error accrue when you run our notebook? Or you have done some custom changes?
@dreamin4444
@dreamin4444 Год назад
@@Roboflow Tak, na waszym notebooku. Mam ten sam problem :/
@Roboflow
@Roboflow Год назад
@@dreamin4444 I'm working on fix right now
@Roboflow
@Roboflow Год назад
I just fixed the notebook. There was a change in pytorch-lightning. And that broke our notebook. It should work fine now.
@dreamin4444
@dreamin4444 Год назад
@@Roboflow Thank u so much!
@sihameghaprabashwara6025
@sihameghaprabashwara6025 Месяц назад
Is there any way to detect in a video ?
@miladfarzalizadeh3058
@miladfarzalizadeh3058 Год назад
how inference time for each frame of video?
@Roboflow
@Roboflow Год назад
You should be able to get around 20-25 fps with ResNet50 backbone.
@arami-me4wy
@arami-me4wy Год назад
Great video👍, I have used my own dataset and got annotations using roboflow. I got this error can you please resolve this --> 243 raise TypeError( 244 f"Detections.__getitem__ not supported for index of type {type(index)}." 245 ) TypeError: Detections.__getitem__ not supported for index of type .
@Roboflow
@Roboflow Год назад
What is the version of supervision that you have installed?
@arami-me4wy
@arami-me4wy Год назад
@@Roboflow torch: 1.13 ; cuda: cu116 ; supervision: 0.3.0 ; transformers: 4.26.1
@Roboflow
@Roboflow Год назад
@@arami-me4wy wow, this is unexpected. Could you please create a bug report here: github.com/roboflow/supervision/issues and attach link to your notebook. I'll try to debug it live.
@arami-me4wy
@arami-me4wy Год назад
@@Roboflow ok thank you
@SalmanFarsi-qt6fr
@SalmanFarsi-qt6fr Год назад
Bruh did You resolved this issue. If yeah then guide me too i am also getting this for too long. Will really appreciate Thanks❤😊
@kilikia8939
@kilikia8939 8 месяцев назад
Why you dont show, how look annotation?
@user-dk2rx7wu2k
@user-dk2rx7wu2k Год назад
How do I count the various objects in the photo?
@Roboflow
@Roboflow Год назад
You can do it quite easily, for example like that: count = len(sv.Detections.from_transformers(transformers_results=results))
@user-dk2rx7wu2k
@user-dk2rx7wu2k Год назад
@@Roboflow And how do I determine the number of objects for each class in the photo?
@Roboflow
@Roboflow Год назад
@@user-dk2rx7wu2k we do not have that functionality yet, but I'll add it to our roadmap. In the meantime you can do something like that: { int(class_id): len(detections[detections.class_id == class_id]) for class_id in np.unique(detections.class_id) }
@user-dk2rx7wu2k
@user-dk2rx7wu2k Год назад
@@Roboflow Thank you very much!
@afriquemodel2375
@afriquemodel2375 6 месяцев назад
it is possible to do the same thing in local environnement??
@Roboflow
@Roboflow 6 месяцев назад
Yes. But you really should have GPU train it.
@afriquemodel2375
@afriquemodel2375 6 месяцев назад
@@Roboflow i have Gtx nvdia 1050 4Go
@YahyaSamet-vi3pu
@YahyaSamet-vi3pu 8 месяцев назад
how to test this model in real time
@varshanathkm6947
@varshanathkm6947 Год назад
Is there any way to detect in a vedio
@Roboflow
@Roboflow Год назад
Yes! you can use process_video function from supervision package: roboflow.github.io/supervision/video/#process_video to automatically process every frame from video.
@olanrewajuatanda533
@olanrewajuatanda533 Год назад
@@Roboflow the link leads to nowhere. Please, can you check again? I'd appreciate if you can do that. Also, how can one see the mAP, F1 and Recall from the training and validation?
@pranks4167
@pranks4167 10 месяцев назад
Hello Peter, I waana you to become my teacher , how can we do it ?
@systembreaker4651
@systembreaker4651 Год назад
Why you didn't make with pycharm its hard to do like in the video 😢
@Roboflow
@Roboflow Год назад
You men in colab?
@systembreaker4651
@systembreaker4651 Год назад
Yes on it
@systembreaker4651
@systembreaker4651 Год назад
@@Roboflow i need to make it on my laptop
@afriquemodel2375
@afriquemodel2375 6 месяцев назад
--> 366 raise MisconfigurationException("No supported gpu backend found!") MisconfigurationException: No supported gpu backend found!
@rupalibalabantray5178
@rupalibalabantray5178 Год назад
Hello sir I am a Research Scholar I want a dataset of thermal images of a Distribution transformer can you help me out.
@FirstNameLastName-fv4eu
@FirstNameLastName-fv4eu 5 месяцев назад
hahahaha !! 12324343 KM from reality
Далее
🛑 до конца!
00:12
Просмотров 26 тыс.
Вопрос Ребром - Субо
49:41
Просмотров 1,4 млн
Tryolabs | Automated soccer ball possession using AI
0:59
АЙФОН Г0ВН0
0:54
Просмотров 1,5 млн
Новодельный ноутбук Pocket386
1:16:17
📱магазин техники в 2014 vs 2024
0:41