should line: rollCamera, pitchCamera, yawCamera = rotationMatrixToEulerAngles(RFlip*R_tc) be the same as line: rollMarker, pitchmarker, yawMarker = rotationMatrixToEulerAngles(RFlip*R_tc)?
This is by far the BEST tutorial about this specific topic... I would love to watch similar lessons and more advanced ones about OpenCV and Aruco. Again, this is amazing!
that was great video, im just having issues with my z position it seems to be showing 100 mm less than what is actually is (i measured using a ruler). did anyone else have this issue? if so what is the fix for it? thanks
Great job as usual, it is the first time we see a fully detailled video on the ARUCO tracking , cant wait you see your implementation with ArduPilot's Precision Landing
Thanks Patrick, but it's all out there. I didn't invent anything here, simply making aware people of the freely available tools that people greater than me have been developing all around the World. The real thank should go to OpenCv and Ardupilot. I am going also to release something similar with ROS, as the one you have developed.
There is a disconnection with dronekit when the drone takes off. The companion computer is no longer able to read the accurate vehicle mode. I ran the read telem script, this script throws a key error for min thr. When I compiled dronekit from your repo, and ran the script, it returned with syntax errors on connect at Init function of dronekit. You've done an amazing job, It'd be amazing if you could let us know the correct python version that runs the code🙃
Hi sir, i've studied your work in a very long time and i think i figured out something: The function R_ct=np.matrix(cv2.Rodrigues(rvec[0]) is used to transform the output rotation vector - rvec[0] (which equivalents to the altitude of Aruco respect to the camera's frame) into a rotation matrix. So Why did you plus R_ct with T which gives you the altitude of camera respect to the marker to find MARKER's Altitude ? Thank you for all of your good work!
Hey Tiziano! Awesome as always. Since you asked us to leave a comment for related work (concept) I want to share mine. I was previously working on this project of navigating a quad in indoor and outdoor environment (without GPS) using aruco. I was working on something similar for my final year project for college degree. Because of some unfortunate events I wasnt able to complete it. I had to change a few deliverables and as of now my project is under consideration for gold medal (will be announced in a few weeks (fingers crossed)). Hope you find my original concept interesting and related to your work: The plan was to use LiDAR, camera and arucos and a servo. The quad will use LiDAR to perform assisted take-off (inside) and will detect the downward aruco code and change orientation accordingly. Once done, it will rotate camera and LiDAR (using servo) and detect the aruco on the wall in front. This aruco will tell where should the drone move next (left or right), using lidar assistance so we dont run into the wall. After the safe distance is reached this aruco will tell where to rotate. This keeps happening until the quad is out in outdoor, where a aruco downward aruco will tell that its out, this gps location is saved as home location. Once the quad is out in open environment it will use the mission waypoints to perform autonomous mission. After mission, the drone will come back to home location and perform the same manuever to find its way back and land in the same place (indoor) from where it took-off.
My plan was to complete this project anyway, but since I got employed within a week of end of final exams; I had to focus on other things. But in case I will be very happy to help you and seek help from you trying to make this concept a reality.
I bet you were. I think this project might require more than a couple of videos ;-). In time we'll get there, but we'd need to step into ROS, where there are already several modules for lidar and slam
Yeah it's gonna be a series of tutorials :) Oh and ROS can be used, but for a simple point distance detection you dont really need ROS. I used Benewake TFmini LiDAR, his one just transmits distance parameters over serial port (used USB to TTL converted as I was using pixhawk with serial port) and this could be implemented in directly in our python code (like the one you used to make in other tutorials)
@@prandtlmayer I think a shout-out would be enough :). Anyway, I'll have to compile and edit a few things before it can be featured in your channel (Its a mess right now). It would take a while, but not more than a month. I'll contact you whenever I'm done. I'll try to edit it in such a way that it can be easily understood by your viewers and can fit in a 10-15min video tutorial :)
There is nothing universal, and you need to be careful every time to the reference frame. In general in Computer Vision, camera frame is x right, y down and z being the line of sight. The Aruco algorithm uses its own frame. I explain all the transformations in the video, but they could be different from your particular example
Thank you very much for your video. Can I ask how to draw the axis of camera in the center and set it as reference position? I watched your code and couldn't find any syntax related to it. Please help me with it.
Hi Tiziano, I'm using the code on a raspberry pi 3B and a 5Mp raspberry camera module. The problem I am having is that the camera frame rate is extremely slow when I try to video capture. Is there any way to solve this issue?
Amazing result! How do you get such great framerates? I am stuck at ~8fps, using the precompiled openCV lib I installed via pip. Does compiling it yourself make such a difference?
Hi Tiziano, awesome video, im struggling with the video feed from my Raspberry Pi Camera Module v2 NoIR. For some reason the video displays with a green tint. I have tried using the opencv library set method to convert the raw footage to rgb but it doesn't produce a clean and clear video. could it possibly be a setting I need to change? if I execute libcamera-hello the video being shown is perfect so im confused as to why the opencv library gives me video with this green tint.
The NoIR usually produces a false color in daylight, as it is more suited for capturing night or low light scenes. Maybe that hello world sends settings that compensate for the green hue. What do you see with raspivid ?
thank you Tiziano, do you perhaps have a similar code using visual studio with opencv. I am new to coding so having troubles on making the basics work. Appreciate your help
Hi, whenever I run the code aruco_pose_estimation.py I get this error --> Assertion failed (scn == 3 || scn == 4) in cvtColor. Please help me, I have spent hours and hours but I get stucked.. Also can video.capture() run in rpi ? or do I have to manage the code with PiCamera() ?? Thanks..
That is smart. But it's not fair ;-). Jevois can get a marker at 60fps, if I am not mistaken, and outputs directly the pose... The only thing I am concern with is how far can you spot it, as the resolution is really low
I am pretty new to all this coding with Raspberry Pi. I don't know which place you are writing all the stuff from the beginning of the video. Could you please explain a bit? :)
Great Tiziano , thank for your tutorial . I'd like to use dronekit and opencv in my next UAV, from your experience rasp PI0w could be used with opencv or better to start with PI3 ?
Ok , probably for this scenario is better pi3, pi0 is interesting for the small size.Maybe i will try to use it only with dronekit to manage some sensors for obstacle avoidance . My goal for the next UAV is to to add some gesture function with opencv , your tutorial inspire the fantasy thanks
You can 2 in two ways: one is directly from the Raspberry Pi, using a GPIO and a packat like servoblaster, the other is through a servo control pin of the Pixhawk and Mavlink commands. Both ways work
hello Tiziano.. i have some question about opencv and dronekit module.. when i use code for takeoff and landing, everything is succeed.. but when i combine openCV module to takeoff and landing code, i don't know why the openCV module is not identified.. but when i only use the openCV module to open the camera, the OpenCV module is identified.. can you help me about it?? Thank you..
The frame? But that is the easy part. Try creating another output image and create a few lines there. Make sure that the position you are setting is actually feasible
Hi Tiziano, this is a great video, we want more like this. I never found such comprehensive video. I am doing a project where I am using apriltag instead of Aruco. With this tag my objective is to identify the marker by camera and move the bot nearest to the marker. Lets say if the marker is sticking on the ceiling it should stand just below the marker on the ground. How should I proceed. So far we have a setup that detect an apriltag marker and its pose. Also do you have any forum where you help people like us?
I have a custom object that am detecting using my pi camera with and drawing a bounding box on the object. I then obtain the x and y corcinates of the object. Can I transform these coordinates to north and east? The image size is 360 by 240
Hey Tiziano, I was hoping to find a solution to how to control the drone without GPS fix? When controlling via a companion computer, the drone seem to ignore all my commands due to no GPS fix, I have being looking for a solution to this for month.... would really appreciate if you could provide a solution.
while running the code when it was reaching the condition: if ids is not None and ids[0] == id_to_find: it's showing 'none' so that's why it's not detecting the aruco marker
Tiziano Fiorenzani thanks sir for your response. Actually I got a problem. I wrote all the same code as you shown. When I run the code there is some error. And it can’t detect the marker position.
Hi Tiziano, can I ask what is the meaning of "rvec, tvec = ret[0] [0, 0, :], ret[1] [0, 0, :]" in above code? I think ret[0] means the revc in ret, but what the following index [0, 0, :] stands for? Could you help me explain this?
Sometimes code changes, so you should adapt this code to the current status. Back then that was the syntax to get the rvec (the rotation vector) and tvec (the translation vector) as the last column of the output tensor.
Ciao Tiziano! following your tutorial, the code worked perfectly. But after a restart I received this error message. Could you please help me? OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor, file /home/pi/opencv-3.4.0/modules/imgproc/src/color.cpp, line 11111 Traceback (most recent call last): File "aruco_pose_estimation.py", line 114, in gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) #-- remember, OpenCV stores color images in Blue, Green, Red cv2.error: /home/pi/opencv-3.4.0/modules/imgproc/src/color.cpp:11111: error: (-215) scn == 3 || scn == 4 in function cvtColor
Ciao Tiziano. I have a problem, I wrote the code and run it, It displays the camera, but doesn't displays the frames. Then i run the aruco_pose_estimation.py from guthub and the same happens. I'm using Python 3.7 and Opencv 4.4.0. Any solution?.
@@prandtlmayer I realized that i was using a wrong library in the ArUco generator, so the program would never detect my Aruco marker. Anyway, Thank you very much Tiziano.
Hi, i have a question regarding the camera matrix....camera matrix=calibration matrix (intrinsic) *Rotation and translation matrix (extrinsic) which gives a 3*4 dimension of camera matrix. In that case aruco.estimatePoseSingleMarkers fails resulting an error "OpenCV Error: Assertion failed (src.size == dst.size && src.channels() == dst.channels()) in cvConvertScale, file /tmp/binarydeb/ros-kinetic-opencv3-3.3.1/modules/core/src/convert.cpp, line 5903". So is it always we consider camera matrix as calibration matrix (3*3)???
@@prandtlmayer if I change the camera facing forward can I find the location of the marker after the drone follows the marker? what did I change from the program? thank you any help is very helpful
Hi Tiziano, when trying to run your script I get the following error: Traceback (most recent call last): File "", line 2218, in _find_and_load_unlocked AttributeError: 'module' object has no attribute '__path__' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "aruco_pose_estimation.py", line 39, in import cv2.aruco as aruco ImportError: No module named 'cv2.aruco'; 'cv2' is not a package What did I do wrong? could you help me?
Hi sir. I got a problem when I run on raspberry using vscode "TypeError: 'cameraMatrix' is an invalid keyword argument for this function" do you know why this happens? thank you
@@prandtlmayer I am facing the same problem. I checked the documentation and the function detectMarker takes only 3 arguments; image, dictionary, and parameters. Any idea how this can be solved? THANK YOU!
@@muhammadamirulhidayattulla6973 removing these 2 arguments removed the error. I had to remove them from other places as well. Sadly I was not able to get it to work. I'm not sure why but I stopped working on this project
I hope someone can help me ,It didn't work for me T T, I printed the markers 60 and 45 , the size of the image is 640x480, and I used intrinsic parameters (calculated with a tool written in matlab, and i tested these calibration parameters and worked fine), the code is practically the same of the video(i reduced the code to print ids but always print "none")
It seems that the dictionary ARUCO_MIP_36h12 it is already not supported, i've tried with DICT_APRILTAG_36h11 and i generated the code 12 and 13 using one of the tools, but they didn't work, but when i try with the dictionary DICT_ARUCO_ORIGINAL using the marker 72 and 50 workd fine.
Hey tiziano, would really like to collaborate with you on some project, quite similar with the above vid. May pay you also on the consultation. But it involved fixed-wing vehicle. hope i can reach u properly and officially
