Label Data with Segment Anything Model (SAM) in Roboflow

Подписаться 37 тыс.

Просмотров 19 тыс.

50% 1

We are excited to release support for zero-shot segmentation labeling in Roboflow Annotate using Meta AI’s Segment Anything Model (SAM).
Using the Smart Polygon feature, you’re accessing a cloud-hosted Segment Anything model enabling you to apply polygon annotations faster, easier, and more accurately than ever before, right inside the Roboflow UI. No setup. No servers. No integrations. Create annotations with one click.
Chapters:
00:00 - Introduction
00:35 - Concrete Cracks Annotation with Smart Polygon
03:06 - Converting Bounding Box Blueberry Object Detection Dataset into Polygons
06:43 - Annotating Low-Resolution Blood Cells Images with Smart Polygon
08:47 - Conclusion
Resources:
🌏 Roboflow: roboflow.com
🌌 Roboflow Universe: universe.roboflow.com
📚 Launch: Label Data with Segment Anything in Roboflow blog: blog.roboflow.com/label-data-...
🔥 Automated Data Labeling with SAM: • Accelerate Image Annot...
On July 29th, 2024, Meta AI released Segment Anything 2 (SAM 2), a new image and video segmentation foundation model. According to Meta, SAM 2 is 6x more accurate than the original SAM model at image segmentation tasks. Learn more: blog.roboflow.com/what-is-seg...
Don't forget to like, comment, and subscribe for more content on AI, computer vision, and the latest breakthroughs in technology! 🚀
#MetaAI #SegmentAnythingModel #SAM #ImageSegmentation #ComputerVision

Наука

Опубликовано:

5 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 52

@changtimwu Год назад

Wow you guys are so fast. Let’s see how long after other labeling tools would add this feature especially those pricey ones.

@lorenzoleongutierrez7927 Год назад

thanks for sharing !

@user-qq5bq9nr9e Год назад

Is it possible to set threshold values for area coverage of similar objects in an image and segment them as same class but with different colored masks depending on their size?

@Blooper1980 Год назад

Good job on implimenting this so quickly. Other applications would have taken weeks.

@Roboflow Год назад

💪🏼

@865we 11 месяцев назад

@@Roboflow excited sir 🙏

@shohrehkia8432 8 месяцев назад

Thanks for the time you invested to let us know about this cool Model. but one question, I have material pieces in size of 1 to 4 mm, mix of plastics and copper, wood, glass and etc. is it possible with SAM label them also or its just masking? I need to label them and create different classes for them.

@isabellalee6213 Год назад

I'm on my a free trail and keen to have a go, and my labeling toolbar only has [Drag tool] and [Comment] options, which is different from what you have shown in the video. Is that becase the [Smart polygon] can only being accessed to paid version?

@michaelringer5644 Год назад

Is a 2s lag after selecting an object with sam in the roboflow editor still normal?

@SweetShotGaming Год назад

This is awesome! Will there be an implementation in the future to have an auto labeling feature? For example in this case, not even needing to click on the cracks, but the auto labeler knowing to automatically segment the cracks in the image.

@Roboflow Год назад

Full zero shot annotation? We are experimenting with it… we will crate Google Colab and video tutorial on Tuesday. As for feature in the editor I guess it all depends if it will work good enough. Stay tuned.

@AmartyaMandal-ml9uq Год назад

Thanks for integration! I couldn't use the semantic masks generation for my dataset and use SAM. Is it not under the free trial?

@Roboflow Год назад

Hi, it is Peter from the video! I know that SAM was accessible for free in RF for some time. I'm honestly not sure if that is still the case. Please let me ask around and come back to you.

@user-sr3ci2lr9w Год назад

Is it possible to use SAM to actually detect and count specific objects? (for example, fruits on a tree)

@Roboflow Год назад

If you just want to do detection and count I'd go with your regular object detector like YOLOv8 for example.

@fahadmajeed5630 Год назад

Hi, can we do medical images or x-ray images segmentation using smart polygon..?? Because if we talk about histopathology images than we have to segment very very tiny areas in one single image. Can we do that kind of segmentation using roboflow..??

@Roboflow Год назад

Hi 👋🏻! It’s Peter from the video. I didn’t show any X-ray example because I didn’t wanted that video to be super long. But sure! I think that automated X-ray image segmentation is possible. Feel free to take a try and annotate some images. You can create account for free and that feature is also free for now.

@EmphaticTrain Год назад

Any help is appreciated. I downloaded my custom dataset, 1's where I highlighted my object of interest, and zeros everywhere else, but my labeled images are completely black. No regions of images that are white? Is there a tutorial on how to successfully download a custom dataset?

@EmphaticTrain Год назад

ah I figured it out. For anyone who gets stuck. when you download your custom data set all your objects of interest will be labeled as one (that's if you're just doing one class); therefore, in order to view your labels you need to open up the labeled image using matplotlib. To actually see it you do this: import matplotlib.pyplot as plt import cv2 path_to_label_img = " " image = cv2.imread(path_to_label_img) plt.imshow(image[ : , : , 0])

@Roboflow Год назад

Glad you figured it out… I was super busy with latest video. And haven’t noticed that question.

@dinanmutamaddin400 Год назад

can SAM do segmentation mask annotation to bounding box or can roboflow add this feature ( like convert all bbox to polygon ) ? please share the collab code or other reference because I have dataset annotated with segmentation and I want convert to bounding box for research with some model , thank you.

@Roboflow Год назад

If you save the segmentation dataset in COCO format you will get boxes for free and can use it both in object detection and image segmentation scenarios.

@dinanmutamaddin400 Год назад

@@Roboflow thank you, but why not add features convert ALL smart polygon to bonding box in annotating mode ? 😅 because maybe we can look it for review the annotations again hehe...

@itayhilel2168 Год назад

I don't understand why you don't have a button for "convert all bbox"

@Roboflow Год назад

It was not there when I recorded but it is there now 💜 That was my feedback to. You can convert all with one click.

@yannsieyadjeu Год назад

@@Roboflow Hey fam. Where is the button "convert all"?

@Roboflow Год назад

@@yannsieyadjeu good question. Take a look at this tweet: twitter.com/braddwyer/status/1646953843790434314?s=46&t=PmKZyPs_J7tyW5sS8kHeLg top right corner of editor.

@drasafasforis 5 месяцев назад

@@Roboflow Do we have the ability to change all images in a dataset from bboxes to SAM without going through each image manually?

@winighfinw-ig3mt Год назад

Nice 👍 But why don't you add Grounding DINO directly as button so you can annotate images with boxes using text prompt

@Roboflow Год назад

I’m dropping Google Colab with that next week. And who knows? If that goes well maybe we will add full auto annotation to Roboflow editor 🔥

@body1024 Год назад

Amazing new upgrade please i am having problem to upload my trained yolov8x instance segmentation mode to roboflow using api it works for object detection models but not for instance segmentation

@Roboflow Год назад

Thanks! Are you using our notebook?

@snehitvaddi Год назад

Hello Team! I am working something where I need to find size of the object in centimeters/mm. I remember Peter made a similar implementation one of the video but I missed the link of that. I have been scratching my head since past 30 mins. Could you please share the link of that video..🙏🏽🙏🏽

@Roboflow Год назад

Sure! Here it is: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-_Zr1pOi7Chw.html starts at 8:04. We will most likely have a dedicated vid for that use case, but that won’t happen in near future so this is all we have.

@snehitvaddi Год назад

@@Roboflow Suuuperb! Thank you.🤝

@Roboflow Год назад

@@snehitvaddi my pleasure ;)

@chinnagadilinga5742 Год назад

Hi Sir I'm Beginner in I saw your Computer vision video's its fully combined and merged can you please update one by one video order that time we can understand easily thank you.

@Roboflow Год назад

Hi, it is Peter from the video? Do you mean videos related to zero-shot annotations?

@yannsieyadjeu Год назад

Can I use SAM for object detection with YOLOv8?

@Roboflow Год назад

SAM is a segmentation model. What you can do is use YOLOv8 for object detection and than convert those boxes into segmentations with SAM.

@yulei7199 Год назад

@@Roboflow If the performance of SAM on COCO detection is validated (for example, use masks and CLIP model to get the class label in COCO categories), perhaps YOLOv8 is not needed at all?

@Roboflow Год назад

@@yulei7199 depends on what you want to do. In some cases YOLO is not needed. In some cases when speed is primary objective it is better to run YOLO segmentation without SAM.

@patrickwasp Год назад

It would be awesome to convert all bounding boxes to polygons without having to right click each object

@Roboflow Год назад

Hi 👋🏻 It is Peter from the video. That functionality is the right now :) That was one of my recommendations to dev team after making that vid too… and now it is there. You can run conversion with one click :)

@aminmemar416 Год назад

Can i upload shapely polygon wkt as annotation to roboflow?

@Roboflow Год назад

Unfortunately not :/

@aminmemar416 Год назад

@@Roboflow oh that’s 😞. Coco did support interior either what should i do?

@Roboflow Год назад

@@aminmemar416 if you can convert it to COCO it would unlock roboflow upload for you

@aminmemar416 Год назад

@@Roboflow it is not supports holes either. I want to segment walls and exclude pictures and windows

@Roboflow Год назад

@@aminmemar416 SAM does not support holes unfortunately:/

@user-qq5bq9nr9e Год назад

Is it possible to set threshold values for area coverage of similar objects in an image and segment them as same class but with different colored masks depending on their size?

@Roboflow Год назад

It is but not through UI. You would need to create Python script to do that and interact with Roboflow through the API