330 - Fine tuning Detectron2 for instance segmentation using custom data

Подписаться 104 тыс.

Просмотров 22 тыс.

50% 1

This video tutorial explains the process of fine tuning Detectron2 for instance segmentation using custom data. It walks you through the entire process, from annotating your data, to training a model, to segmenting images, to measuring object morphological parameters, to exporting individual masks (results) as images for further processing.
Code generated in the video can be downloaded from here: github.com/bnsreenu/python_fo...
All other code:
github.com/bnsreenu/python_fo...
Detectron2 repo: github.com/facebookresearch/d...
Annotations were done using Makesense: www.makesense.ai/
Dataset from: leapmanlab.github.io/dense-cell/
Direct link to the dataset: www.dropbox.com/s/68yclbraqq1...
Data courtesy of:
Guay, M.D., Emam, Z.A.S., Anderson, A.B. et al.
Dense cellular segmentation for EM using 2D-3D neural network ensembles. Sci Rep 11, 2561 (2021).
Data annotated for 4 classes:
1: Cell
2: Mitochondria
3: Alpha granule
4: Canalicular vessel

Наука

Опубликовано:

22 авг 2023

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 88

@SkalskiP 9 месяцев назад

Hi 👋 I'm the creator of MakeSense! Thanks a lot for using the tool in your tutorial!

@DigitalSreeni 8 месяцев назад

Nice to meet you. Thanks for creating a very useful tool :)

@utkarshdeshmukh3268 5 месяцев назад

@SkalskiP. You have made a fantastic tool. Great job! I found a minor bug in it. I believe the coco-json structure has evolved. Under "annotations" it is expected to have a category_id, which your tool does not populate.

@temiwale88 10 месяцев назад

I love you Doc. I needed this. Thank you, DigitalSreeni!

@zubairkhalid3209 6 месяцев назад

Appreciate your tutorials a lot! Keep the good work going.

@astitva5002 10 месяцев назад

your resources have been incredibly helpful. thank you so much

@sozno4222 10 месяцев назад

I honestly believe you have the best tutorial videos on youtube for vision ml. It's very thorough providing all the necessary information for somebody to work through it from scratch. Keep it up. Love these

@satwikpandey9977 11 месяцев назад

Happy teacher's day bhaiya. You've made me fall in love with machine learning. Your videos are to the point and nicely explained. Thanks❤

@ajay0909 11 месяцев назад

The hardest part is annotating the images, I have done a lot of annotations before. You have done a good job by explaining annotations in detail which is very rare. Can you please share the type of applications that are in demand so that we can try out new things with the knowledge you give us here. Thanks for all the extra effort you put in to explain each line of code.

@patis.IA-AI 5 месяцев назад

Merci , super tuto , bien clair , bien explique en un mot "amazing"

@user-qs5rb2tj1p 9 месяцев назад

Thank you very much for this helpful tutorial. God bless!

@soumen_das 6 месяцев назад

Excellent Tutorial!

@QubitsQuantum 29 дней назад

KING YOU ARE A KING!!!!!!!!THANK UUUUUUUUUUUUUUUUUUUUUUU U ARE THE BEST!!!!!!! I THINK U SHOULD GET AN AWARD. THANK YOU!!!

@msaoc22 11 месяцев назад

thank you again for this great tutorial. You make amazing job for us. Detectron works a lot better for me then Yolo8 or Faster RCNN. I will play a bit more with parameters in future, but even with small dataset I was able to get reasonable result =)

@DigitalSreeni 11 месяцев назад

Similar experience as mine. I too like Detectron2 results better and I find it to be more reliable compared to other instance segmentation frameworks available.

@mahilkr Месяц назад

Awesome explanation

@buildingconstructioninnova6620 8 месяцев назад

Great job. We will reference you. Keep impacting

@DigitalSreeni 8 месяцев назад

Awesome, thank you!

@acmontesh102 10 месяцев назад

Thank you for such a great explanation. It would be very nice to complement the tutorial with the workflow to build on top of detectron2, i.e., to use the pooled feature map that the object detection and the masking model use to add a new module within the architecture.

@moumitamoitra1829 4 месяца назад

please make a tutorial on DETR instance segmentation. Your videos are really helpful.

@computing_T 7 месяцев назад

23:17 I love coffee as well hahaha . Thanks sir for the content

@nguyenthehoang9148 10 месяцев назад

A point you made at the very end of this video is very important. U-NET is not capable of segmenting overlapping objects. Would you think of making a video comparing between different models for segmentation or other computer vision tasks? Thank you!

@inquisitiverakib5844 10 месяцев назад

It will be very helpful if you look into MMsegmentation framework. It's quite similar to this. But there is very few videos about this...😊

@alexiscarlier2773 10 месяцев назад

Thanks again. Do you know if it is possible to train mask r cnn using only bounding boxes that still performs the segmentation after training?

@askitloud3168 11 месяцев назад

Thanks for this sir. I understand each and very thing very clearly. I have few questions. 1) - Why size of the model_final.pth is 334.2 MB and how we reduce size of model without loosing model accuracy and detection rate. 2) - For getting best accuracy and detection rate from detectron2 what should the minimum annotated labels per class and how many max classes we involve in single model. 3) - How, we check our model performance after training, is there any metrices and anything so that we can get rough understanding about model accuracy.

@yahaisha 11 месяцев назад

thanks sir .🎉🎉

@DigitalSreeni 11 месяцев назад

Most welcome

@alexis.martin-comte 10 месяцев назад

Hey, i watched your tutorial and it was very clear but i would love to use custom data format (like tiffs with more than 3 bands) and i find it very difficult to do

@ErikCarrieri 8 месяцев назад

Do you know how to adapt this process in order to work on images with multiple bands? For instance, for aerial images of homogeneus areas sometimes having just RGB images could not provide enough information and the accuracy of the segmentation could result too poor. Hence, to improve accuracy more bands (such as a Digital Terrain Model, or a slope band) can be used, since they provide more information for the segmentation of the object. Do you think something like this is feasible?

@user-qs5rb2tj1p 6 месяцев назад

Thank you very much for this heloful tutorial. I have a question regarding the unites of area of each class- is it pixel?

@rariwa 7 месяцев назад

Thanks for the tutorial, do you know a reference to train instance segmentation using label mask only (same image size with original image) ? I have one class but I would like to differentiate different object.

@minhloinguyen8639 7 месяцев назад

Thank you for worderful lecture, but I get in trouble with dataset detached in description. I don't know how to convert it to COCO format to apply detectron2, I think data include image and label so I don't need to use makesense to label it manually. Give me some tips for using data provided.

@seanolivieri4829 2 месяца назад

for d in random.sample(train_dataset_dicts, 2): img = cv2.imread(d["file_name"]) visualizer = Visualizer(img[:, :, ::-1], metadata=train_metadata, scale=0.5) vis = visualizer.draw_dataset_dict(d) plt.imshow(vis.get_image()[:, :, ::-1]) plt.show() I dont see the right labels, they are labeled as 1-12 in my dataset instead of the category names. Probably its a problem with my json but does anyone know a solution?

@zeynabbayat5596 6 месяцев назад

I appreciate your free tutorial. The problem is that I want to use the weights and the output of TensorFlow, but the result in the output folder is not like what I had in my YOLO training. I don't know how to find the weight folder and TensorFlow output images.

@prayag_pawar 6 месяцев назад

Hey , as i work on aerial imagery, i thinking of using detectron2 resnet101 model and create my own pretrained model as they used coco datatset for creating pretrained in .pkl format than fine tune the model according my needs how can i do that thank you helping 👍

@yahaisha 11 месяцев назад

sir..can you show how to display result performance using tensorboard for this example .for total loss

@jackyu6466 7 месяцев назад

if you start the training process ( trainer.train() ) inside the same block as default trainer, tensorboard should work

@junk90s92 4 месяца назад

Could you provide the annotations and json files you used? I was not able to reproduce the results -- the accuracy on the test images was quite poor.

@junk90s92 4 месяца назад

Hey DigitalSreeni. Love the video! Where can I access the .json files and .png images? I see tiff files in the dropbox but not the others If unaccessible, do I need to create them myself?

@dimitheodoro 11 месяцев назад

Thanks for sharing knowledge! Again i cannot find the code link!!!!

@DigitalSreeni 11 месяцев назад

You will find the code link now. There is a 4 to 5 hr gap between the video publishing time and me waking up and pushing the code to github. You caught me while I was still asleep :)

@sushmanthreddymereddy3074 11 месяцев назад

@DigitalSreeni I have been thinking to fine-tune dataset on cell-lineage analysis for C .elegans??

@DigitalSreeni 11 месяцев назад

Do you have the training data? Often times it comes down to data and knowledge to annotate.

@pointrally1713 6 месяцев назад

Why do no results appear in the image after training with images with only 1 class? Of course I changed it to NUM.CLASSES = 1.

@nguyenthehoang9148 9 месяцев назад

I looked at some notebooks in the Sartorius - Cell Instance Segmentation competition on Kaggle but people don't report quantifying metrics. It's kind of confusing. Do people don't really care about the metrics like AP in segmentation tasks?

@user-on1gn8wr6z 10 месяцев назад

thanks a lot sir, can you show us how to run it in python under windows environment?

@alocatnaf 11 месяцев назад

thanks for your great tutorials! are there any benefits using detectron2 with maskrcnn instead of, e.g, matterports maskrcnn?

@DigitalSreeni 11 месяцев назад

Detectron2 makes it easy to access Mask R-CNN compared to matterport or other forms. Underneath, they are all using the same approach. So, no benefit when it comes to results but super beneficial when making the technology accessible.

@alocatnaf 11 месяцев назад

@@DigitalSreeni ok i see. very helpful, thanks alot!

@billlee2641 2 месяца назад

at 26:02, does initiate weight means exactly same as transfer learning ?

@chewchengkai2769 4 месяца назад

Hi Sir, I want to ask about the evaluation part, how to find performance metrics other than mAP, such as confusion matrix, and IOU? Find so many code online no show example with using detectron2 to find these.

@praveenavala2297 Месяц назад

same doubt bro, did u get it?

@raihanvalentino2175 11 месяцев назад

Very nice content. But I think your link in description is not working well. Please fix it. Thank you!

@DigitalSreeni 11 месяцев назад

If you are referring to the dataset link then try this link: www.dropbox.com/s/68yclbraqq1diza/platelet_data_1219.zip

@shreyasinha7926 11 месяцев назад

Thanks for sharing! I really needed exactly this. I cannot find the dataset. The link is not working

@DigitalSreeni 11 месяцев назад

Can you try this direct link for the dataset, I hope it works for you (unless region-blocked for some reason): www.dropbox.com/s/68yclbraqq1diza/platelet_data_1219.zip

@lukasb7013 11 месяцев назад

Detectron2 is a Two-Stage-Detector right? Is it therefore better at detecting and segmenting filigree defects? And does the library mentioned offer the possibility for hyperparameter tuning as i sparsely see any content about it with regards to the topic of instance segmentation

@DigitalSreeni 11 месяцев назад

I do not know what Filigree is so I cannot share my opinion based on any experience. But if you are trying to detect specific objects in an image then this approach will work for you. Regarding hyperparameter tuning, you can do it via adjusting the parameters in the config file.

@lukasb7013 11 месяцев назад

@@DigitalSreeni Thanks a lot. Filigree = very small details :)

@DigitalSreeni 11 месяцев назад

Thanks for educating me :) Learning something new every day :)

@lukasb7013 11 месяцев назад

@@DigitalSreeni :D Back to my question, 2 stage detectors should be better for such details right?

@sudiptaproggaislam7321 8 месяцев назад

please make a video of detectron2 for panoptic segmentation...

@saichandanakoganti3410 8 месяцев назад

Sir, I need to try instance segmentation with Mask RCNN using my custom dataset could you help me with this.

@DigitalSreeni 8 месяцев назад

That is exactly what this video is all about.

@saichandanakoganti3410 6 месяцев назад

Sir, i have a separate annotation file for every single image in the form of .csv can i convert all those files into single .json file?

@boppadapusaiteja8074 9 месяцев назад

You didn't discussed about finetuning why the tile that you have kept

@jayakrishna7995 11 месяцев назад

for annotation tool lets use roboflow for auto segmentation process...🤚🤚🤚👍

@DigitalSreeni 11 месяцев назад

You can try Roboflow on these images. Hopefully you will have better luck than I.

@-_MUBESSIRULUMMAH 2 месяца назад

@@DigitalSreeniI tried using roboflow to annotate and apply your code for detection, but I experienced an error.

@user-fg9dj7dt8n 11 месяцев назад

very goood explanation,how can i contact u?

@vsbhadouria6768 11 месяцев назад

Can you make a video on how to perform virtual staining using deep learning?

@DigitalSreeni 11 месяцев назад

Hopefully in future. Here is a link from my bookmarks. I saved it to test for future video but haven't found time yet. github.com/softmatterlab/live-apoptotic-virtual-staining

@vsbhadouria6768 11 месяцев назад

@@DigitalSreeni Thanks a lot for your response. I will go through the github link and see if I can crack it.

@user-lv7wo4nl5n 6 месяцев назад

Is MakeSense safe to use? Any data security issues?

@DigitalSreeni 6 месяцев назад

Not sure what you mean by safe. Please check their data policy. In general, if you are using any product or service for 'free', there is a good chance that your data will be used to improve their future offering. This can range from learning about how their product gets used to using customers' data to train their own DL models. Nothing is free in life!!!

@user-lv7wo4nl5n 6 месяцев назад

Is it safe to use MakeSense?

@DigitalSreeni 6 месяцев назад

@vtjustin333 5 месяцев назад

Dear Sreeni, I love all of your videos. I have a specific use case. Is there any chance to connect with you.? I have tried in Linkedin, but unfortunately couldn't connect with you yet. :(

@kridsumangsri964 11 месяцев назад

Sir please guide how to deploy custom model on web app thx you ^^

@DigitalSreeni 11 месяцев назад

ru-vid.com/group/PLZsOBAyNTZwYSk07iwEbWFaDxeRw1NXt6&si=d8qH4MK-ACg6pNya

@kridsumangsri964 11 месяцев назад

Thank you❤

@Brickkzz 10 месяцев назад

What a pain to install it on Windows...

@clabrop 9 месяцев назад

were you able to do it?

@user-dd6gz1wk3r 6 месяцев назад

trainer.train() time error results like RuntimeError Traceback (most recent call last) in () ----> 1 trainer.train() #Start the training process 13 frames /content/detectron2/detectron2/layers/wrappers.py in nonzero_tuple(x) 151 return x.nonzero().unbind(1) 152 else: --> 153 return x.nonzero(as_tuple=True) 154 155 RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. Reason for the error??