Тёмный

Explanation How to Integrate Intel ROS Realsense D435 with Gazebo and Rviz - Jetbot AI Kit 

Rangel Alvarado
Подписаться 336
Просмотров 19 тыс.
50% 1

Опубликовано:

 

28 окт 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 50   
@bxtbold
@bxtbold 3 года назад
Hey Rangel I wonder if you added gazebo.launch manually. I'm asking it because i couldn't find that launch file from the realsense2_description package. Thank you in advance :)
@issaiass
@issaiass 3 года назад
It is on the intel_realsense repository, anyway, If you follow my repo documentation you will get it
@margaridapereira4654
@margaridapereira4654 Месяц назад
Hello Rangel, thank you so much for the video. I'm currently facing an issue where the camera appears in RViz but doesn't seem to spawn in Gazebo. Looking at the Worlds -> Models in my Gazebo UI I only see ground_plane... Any advice on how to resolve this? I am using those exact files with ROS Noetic and Gazebo 11.
@issaiass
@issaiass Месяц назад
You have to first check if the urdf parses. If parses with xacro then try to spawn manually Finally try to spawn with the launch file Sometimes is problem of the path, sometimes is problems of the physics if you modified the file
@margaridapereira4654
@margaridapereira4654 Месяц назад
@@issaiass Thank you for your quick response! I solved the problem, I was working in a ROS workspace with a lot of packages and I had misconfigured paths. I created a clean workspace and everything worked fine :) I attached this camera in a pioneer3at with p2os_urdf pkg with the help of this video of yours: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-DWLXYEmcUsA.html
@issaiass
@issaiass Месяц назад
@@margaridapereira4654 good to know that worked for you
@ahmetsaglam8852
@ahmetsaglam8852 3 года назад
Hi Rangel, good work! Quick question: Were you able to add some noise to the simulated depth plugin?
@issaiass
@issaiass 3 года назад
No, because i think the default plugin does not have it. Anyway, there are three ways. One is directly to the plugin, add a gaussian noise function and mix it with the image and The other is to do that in the sdf file as explained in the gazebo tutorial gazebosim.org/tutorials?tut=sensor_noise. The last one is to use the image topic, then use opencv to add a gaussian noise to the image and republish to a new topic; in that way you will not mess with the plugin. Probably the last one is the easiest.
@leeosaya
@leeosaya 3 года назад
Thanks Rangel, this is excilent and a very clear explanation. Just a quick question though, how would you recommend going about using some of the Intel Realsense library's in cooperation with this i.e get_distance or deproject_pixel_to_point?
@issaiass
@issaiass 3 года назад
If you want a project example related to perception and depth cameras common and interesting tasks could be classification (analyze and filter point clouds to see which object is), visual serving (knowing the depth, you can approach and grab the object), reconstruction 3d (make a 3d map of an unknown environment)
@leeosaya
@leeosaya 3 года назад
@@issaiass thanks for your response Rangel, I am currently trying to port my visual servoing code over to simulation. It is reliant on using the Intel python libraries, so I'm just wondering if they are compatible with the simulated camera?
@issaiass
@issaiass 3 года назад
@@leeosaya I think it should be if the libraries are just for post process the PCL using the topics
@hayashi3674
@hayashi3674 2 года назад
This was a great video thank you! Do you have any idea how to add this model onto another model such as a robot?
@issaiass
@issaiass 2 года назад
You have to check the axis and put in that frame, you could check the repo and have an undertanding of that, is very easy
@Fyletikos
@Fyletikos 2 года назад
@@issaiass Can you please explain further? I am trying to implement this sensor on my drone but i can't find the documentation on how to do it, or any tutorial...
@issaiass
@issaiass 2 года назад
@@Fyletikos look at one of my last videos, i explained how to attach an arm, is the same process
@1000MZ1000
@1000MZ1000 2 года назад
Hi Rangel great video! Thanks to your instructions I was able to add this camera to my robot really easy! I have one question regarding the plugin: Is there a reason that the point_cloud rgb values are all gray? The d435 should have rgb depth imagery support so I am not sure if I missed a setting or if this is a limitation from the plugin.
@issaiass
@issaiass 2 года назад
That's all configurable via rviz, do not worry about that
@1000MZ1000
@1000MZ1000 2 года назад
@@issaiass Thanks for the response. Sadly the topic publishes a gray pointcloud instead of rgbd data. Do you have any idea for how change that?
@issaiass
@issaiass 2 года назад
@@1000MZ1000 have no rviz in front of me but what i remember is to change in the point cloud to rgb and intensities
@1000MZ1000
@1000MZ1000 2 года назад
@@issaiass I am not able to find a setting in rviz that displays the object colors. My settings are PointCloud2, depth/color/points topic, color transform rgb8. Could you please verify on rviz as soon as you have the chance? I don't want to start rewriting the plugin before I am sure that I could have not avoided it. Thanks!
@issaiass
@issaiass 2 года назад
@@1000MZ1000 ok, but you can look at ros velodyne bag file that is in the ros wimi, they are in colors the point cloud
@draggador
@draggador 5 месяцев назад
Can you please tell me which versions of ros and gazebo are being used? I need an intel realsense camera plugin for ros humble hawksbill & gazebo ignition fortress.
@issaiass
@issaiass 5 месяцев назад
In this video, ROS Noetic and Gazebo 9-10 i think
@draggador
@draggador 4 месяца назад
@@issaiass Thank you for letting me know!
@yikanggui8231
@yikanggui8231 3 года назад
Thank you! I have a question about plugin in Gazebo. Is that normal that the model of camera in Gazebo is all grey? It seems that the camera rendered correctly in Rviz but in Gazebo, it was grey and didn't have any color.
@issaiass
@issaiass 3 года назад
They do not have a mesh file just an RGB value for gazebo, check the urdf materials of the realsense and you will get more clear about that. If you want to link the mesh file then you will have to put it on the uri
@naruto7350
@naruto7350 3 года назад
Thank you for the video! I have a question. Why is the pointcloud2 on Rviz appearing as a rainbow-ish colour and why is the camera of the simulated realsense grey/black in colour (in Rviz too)? I changed the visual of the ball in front of the camera to bright orange colour but I get the same rainbow in the pointcloud2 and dark silhouette in camera in Rviz. I need to be able to detect the orange using the RGB values to proceed to detect the orange object. Any idea on why this is happening? Thank you!
@naruto7350
@naruto7350 3 года назад
Note: I changed the visual in Gazebo model editor*
@naruto7350
@naruto7350 3 года назад
Okay so i fixed the issue of the camera showing in grayscale. When i change the topic to camera/color/image_raw (instead of camera/depth/image_raw), i can see the orange. However when i try to observe the pointcloud in RGB8, it is in grayscale all of a sudden. When i set it to AxisColour, it becomes that rainbow colour i mentioned previously. I do not want this grayscale point cloud, I want the RGB as well go show the orange colour in the depth point cloud. How can I achieve this? Thanks!
@naruto7350
@naruto7350 3 года назад
I realise the reason i am getting grayscale. The depth is not aligned to the colour. In the actual realsense camera, i can set align_depth to true in rs_camera.launch (inside the realsense2_camera package). How do I do it here in gazebo? Thanks!
@issaiass
@issaiass 3 года назад
@@naruto7350 rviz is just a tool for visualizing robot interactions, if you get the PCL you will get points as features of objects, if you get the raw image you get pixels at, in example, rgb then you could detect the color by range of HSV
@muhammadshaheerbalouch6244
@muhammadshaheerbalouch6244 6 месяцев назад
Hi Rangel, I've cloned both your repos, unfortunately when I run it, the camera doesn't capture anything according to Rviz. Under camera it says "No Image Received", and "No CameraInfo received on [/camera/depth/camera_info]. Topic may not exist." Any reason why this might be happening?
@issaiass
@issaiass 6 месяцев назад
Monitor the ros topic to see whats happening
@issaiass
@issaiass 6 месяцев назад
See the logs on initialization to see which problems are there
@muhammadshaheerbalouch6244
@muhammadshaheerbalouch6244 6 месяцев назад
@@issaiass Thanks for the reply, I checked rostopic list, and can see all the topics are there. I checked rostopic info for the specific topics but there are no publishers, only subscribers, which may be the issue. How do I figure out specifically what's wrong and how to fix it? Is it the fact that I have a different ROS version?
@muhammadshaheerbalouch6244
@muhammadshaheerbalouch6244 6 месяцев назад
I'm quite new so I don't know what to look for in the logs.
@issaiass
@issaiass 6 месяцев назад
@@muhammadshaheerbalouch6244 any problem or something not loading, but i recommend you to start then with the basics, i have a course based on very basic stuff in ros in the list here
@RaviKumar-ub2ng
@RaviKumar-ub2ng 3 года назад
Hi Rangel! I am currently working on a thesis with ur5 robot and TIAGo from pal-robotics, and how can intel realsense camera publish the position for an object that for an example, ur5 can subscribe and perform a pick and place task?
@issaiass
@issaiass 3 года назад
I did not do that in simulation because a physics error I have but, when the octomap is generated you could use the moveit grasping example, they explain how to generate different grasping patterns and select the optimal one. Depending of the object it will be different how how to grasp it, but again, moveit examples of graaping are good enough.
@RaviKumar-ub2ng
@RaviKumar-ub2ng 3 года назад
@@issaiass Ah I see, Thanks! However when I am implementing this in my ur5 gazebo file and made changes in the xacro also, only the camera spawn. Ever encounter that problem before? I am wondering that i might have go make a new launchfile were i include them both?
@issaiass
@issaiass 3 года назад
@@RaviKumar-ub2ng use vscode first to see the urdf, also ros has some urdf tools to check the sintaxis, actually I think could be if rviz error to change the reference frame or If not then a link or some controller not loaded. I have a video that could give you some hints if you check on my video tab, something like "problems ..." Do not remember the title
@francescavergara8717
@francescavergara8717 3 года назад
does it also work for realsense t265 camera?
@issaiass
@issaiass 3 года назад
Any kind of plugin not only realsense
@UplenchwarRavindra
@UplenchwarRavindra 5 месяцев назад
it does not have noetic version
@issaiass
@issaiass 5 месяцев назад
I did in noetic
Далее
Rate our flexibility 1-10🔥👯‍♀️😈💖
00:12
MAGIC TIME ​⁠@Whoispelagheya
00:28
Просмотров 11 млн
Open-source SLAM with Intel RealSense depth cameras
27:40
Depth Camera - Computerphile
12:34
Просмотров 250 тыс.
Really Useful Robot #3 - Head & Intel Realsense Camera
18:24
Intel RealSense D435 - NVIDIA Jetson TX Dev Kit
15:38
Connect Intel Realsense with Python and OpenCV
13:21
Просмотров 17 тыс.
Rate our flexibility 1-10🔥👯‍♀️😈💖
00:12