Тёмный

Rapid, Dynamic Obstacle Avoidance with an Event-based Camera 

UZH Robotics and Perception Group
Подписаться 16 тыс.
Просмотров 190 тыс.
50% 1

In this work, we study the effects that perception latency has on the maximum speed a robot can reach to safely navigate through an unknown cluttered environment. We provide a general analysis that can serve as a baseline for future quantitative reasoning for design trade-offs in autonomous robot navigation. We consider the case where the robot is modeled as a linear second-order system with bounded input and navigates through static obstacles. Also, we focus on a scenario where the robot wants to reach a target destination in as little time as possible, and therefore cannot change its longitudinal velocity to avoid obstacles. We show how the maximum latency that the robot can tolerate to guarantee safety is related to the desired speed, the range of its sensing pipeline, and the actuation limitations of the platform (i.e., the maximum acceleration it can produce). As a particular case study, we compare monocular and stereo frame-based cameras against novel, low-latency sensors, such as event cameras, in the case of quadrotor flight. To validate our analysis, we conduct experiments on a quadrotor platform equipped with an event camera to detect and avoid obstacles thrown towards the robot. To the best of our knowledge, this is the first theoretical work in which perception and actuation limitations are jointly considered to study the performance of a robotic platform in high-speed navigation.
Reference:
D. Falanga, S. Kim, D. Scaramuzza, "How Fast is Too Fast? The Role of Perception Latency in High-Speed Sense and Avoid", IEEE Robotics and Automation Letters (RA-L), 2019.
PDF: rpg.ifi.uzh.ch/...
Our research pages:
Vision-based quadrotor navigation: rpg.ifi.uzh.ch/...
Aggressive quadrotor flight: rpg.ifi.uzh.ch/...
Event-based vision: rpg.ifi.uzh.ch/...
Affiliations: D. Falanga, S. Kim and D. Scaramuzza are with the Robotics and Perception Group, Dept. of Informatics, University of Zurich, and Dept. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland rpg.ifi.uzh.ch/

Опубликовано:

 

29 сен 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 87   
@ubuu7
@ubuu7 5 лет назад
/tangent where did they get these grad students, the abercrombie model line? Is this the standard look at this school?
@sergiocastillo9857
@sergiocastillo9857 4 года назад
those are not common students, but well paid PHD students.
@nuradrikah3207
@nuradrikah3207 5 лет назад
Exactly..how do you decide which is the direction of the maneuever?
@w3bm0nk
@w3bm0nk 5 лет назад
What happens if you just try to slap the shit out of it?
@Lin-cx4zn
@Lin-cx4zn 5 лет назад
Awesome! By the way, how many quadrotors broken in the research? Hahahah~
@ailabRPG
@ailabRPG 5 лет назад
We had a few crashes and we repaired the broken parts, usually just the frame.
@openlink9958
@openlink9958 5 лет назад
@@ailabRPG Great progress
@channelforonne
@channelforonne 5 лет назад
@@ailabRPG shouldve used the light/nylon pool balls to avoid damage ;p
@segurall1
@segurall1 4 года назад
@@ailabRPG Can event based cameras be used to detect dynamic obstacle if the robot is in movement? eg: I'm moving forward and a object comes in from the right of my vehicle into its line of sight. could it detect that moving obstacle and differentiate it from static background?
@shmookins
@shmookins 5 лет назад
In the future, will it dodge bullets? O_o'
@Atlas_Redux
@Atlas_Redux 5 лет назад
No, since it's not strong enough. No drone has the acceleration possible to move away from a bullet, and there will be no drone who can. It's simply physically impossible.
@sodlock4666
@sodlock4666 5 лет назад
I don't think it can dodge bullets after its fired. Maybe it can dodge it by predicting the shooting track before fired
@warn2571
@warn2571 5 лет назад
When it's ready it won't have to.
@thanhvinhnguyento7069
@thanhvinhnguyento7069 4 года назад
@@warn2571 skynet confirmed
@ricasiogaming7873
@ricasiogaming7873 4 года назад
Shadow Heart I did the math. If a 9 mm was fired at a distance of 6 feet the robot would have less than 5 milliseconds to react, and would have to move about 6 inches at 80 mph.
@Ramtaha-tata-mango-yango
@Ramtaha-tata-mango-yango Год назад
what are those sensors on the corner of the wall ?
@leeamraa
@leeamraa 2 года назад
not very good experiments! In all, the ball wont touch the drone if the drone did nothing!!!
@terrilldimayuga2069
@terrilldimayuga2069 5 лет назад
But can it dodge a wrench?...
@ogdmv4751
@ogdmv4751 5 лет назад
If you can Dodge a 🔧 you can Dodge a ball
@SinuousGrace
@SinuousGrace 5 лет назад
Slaughterbots.
@mnageh-bo1mm
@mnageh-bo1mm 5 лет назад
What about throwing the ball at full power like a real man
@johnpellmann441
@johnpellmann441 5 лет назад
Seriously. I doubt the "harder" throws would have hit it to begin with...good progress, just get someone that can throw for the demonstration.
@arpxdagoodone
@arpxdagoodone 5 лет назад
:DDDD
@tienatho2149
@tienatho2149 5 лет назад
The only way to doge a ball with full power is predicting the throwing track before the ball actually "fly"
@mnageh-bo1mm
@mnageh-bo1mm 5 лет назад
@@tienatho2149 lol and how this is even possible?
@tienatho2149
@tienatho2149 5 лет назад
@@mnageh-bo1mm I know this is not drone but as you can see, the goalkeeper actually turn before Messi kick the ball ( but Messi trick the bot anyway lol )
@Bielocke
@Bielocke 5 лет назад
We are completely fucked as human species.
@annoikki
@annoikki 5 лет назад
nice guns bro, the experiment was cool too!
@darkodimitrijevic8496
@darkodimitrijevic8496 4 года назад
And this should proof that you made it? Well the point was to show die audience how you have achieve it. What camera, wiring, explaining, not just throwing ball to the drone!!!!!! Dislike
@paulkirkland9860
@paulkirkland9860 5 лет назад
Research can be fun
@aysommer
@aysommer 5 лет назад
Quadcopter isn't smart, man crookedly throws
@kenkim8295
@kenkim8295 5 лет назад
Can it avoid white ball with little features ? I saw the black tapes or strip on the ball.
@ailabRPG
@ailabRPG 5 лет назад
Yes. The tape was there was something else. The ball is clearly visible as you can see in the event camera stream.
@kenkim8295
@kenkim8295 5 лет назад
ailabRPG do you mean event camera perform as well with white ball with no tape?
@rooster443
@rooster443 5 лет назад
I had a dream...
@quanpham7698
@quanpham7698 4 года назад
I spoiled my iphone when researching in this field :'(
@EasyCircuits
@EasyCircuits 5 лет назад
Awsome, well done. Just it take a bit long to respond
@ABHIII272
@ABHIII272 5 лет назад
Dji is watching this
@RodionRomanov
@RodionRomanov 5 лет назад
Let alone DJI, Skynet is watching.
@L_inventeur_Rapide
@L_inventeur_Rapide 4 года назад
Yes bravo it very good
@RodionRomanov
@RodionRomanov 5 лет назад
Is it just me, or the experimentors really suck at throwing?
@WildChildDos
@WildChildDos 5 лет назад
A pain in the ass for airports.
@qingyizhou4696
@qingyizhou4696 4 года назад
Great work~ What obstacle detector are you using? I'm not sure how to get position based on events.
@UAS_AEROSISTEMAS
@UAS_AEROSISTEMAS 2 года назад
Enhorabuena
@arpxdagoodone
@arpxdagoodone 5 лет назад
Very important science.
@henrik1220
@henrik1220 5 лет назад
Skynet likes this
@회색-m2z
@회색-m2z 5 лет назад
other direction??
@OliverBatchelor
@OliverBatchelor 3 года назад
Cool, but I can see he's aiming low or off to the side each time...
@xiaojake479
@xiaojake479 5 лет назад
But I have questions about the video (time: 1’52”), how can the vehicle hover indoor? what sensors and algorithm you use to solve the positioning problem? by event-camera?
@Spearmeat638
@Spearmeat638 5 лет назад
Maybe, they use Motion Capture System (there is a maker on the drone). Or they may use visual inertial using event camera
@ailabRPG
@ailabRPG 5 лет назад
The event camera is only used for dodging. A standard downward looking camera plus IMU is used for state estimation. See the reference paper below for details.
@1997硫离子的收藏夹
@1997硫离子的收藏夹 5 лет назад
interesting!
@fouzanjaved5441
@fouzanjaved5441 3 года назад
sir, can you give me the circuit diagram and the codes to the obstacle-avoiding drone and also give me the library for the obstacle-avoiding drone.
@loebas667
@loebas667 3 года назад
does it calculate for gravity? or will it assume objects travel in a straight line
@andresalteclado
@andresalteclado 9 месяцев назад
It calculates a parabolic trajectory, otherway it wouldn't be able to do its job. Intrigin indeed, which algorithm uses that sensor may be a deeper question.
@DenSer178
@DenSer178 5 лет назад
Да он в него даже не целиться, дрон не успеет увернуться если кидать точно в него .
@НикитаДеревнин-з7б
1,53 - это точный , но не сильный бросок
@DenSer178
@DenSer178 5 лет назад
@@НикитаДеревнин-з7б Если бы мяч с такой же скоростью летел по прямой точно в дрона, а не по дуге .Я про это . Успел бы тогда дрон увернуться вот в чем вопрос.
@florentinalexandruiftimie8214
@florentinalexandruiftimie8214 5 лет назад
It's a bit hard even for a human to dodge a ball from that short distance. Or I think that because I'm not that agile. Anyway it's processing really well the images
@panagiotissavvidis8271
@panagiotissavvidis8271 4 года назад
Whats those projectors on the ceiling corners? Some kind of an assist? Brilliant work
@khchan3501
@khchan3501 5 лет назад
Out of curiosity, what about if it's kicked at a much higher speed? Or smaller objects like tennis or even golf ball? Or the object has extreme high or low temperatures?
@areteqin8772
@areteqin8772 2 года назад
That is impressive! What autopilot framework are u using? PX4?
@hernandesrio
@hernandesrio 5 лет назад
Very easy, we are developing a autonomous car here in fiap ;)
@tariqshaw6791
@tariqshaw6791 3 года назад
How did it go?
@cwcentralable
@cwcentralable 5 лет назад
Do basically drone defense systems that use kinetic strikes are now obsolete (well if this is implemented on full 360 vision). Obvious that camera latency utmost critical, which likely isn't achievable in real envirs.
@eyal4
@eyal4 3 года назад
nice work! what stereo camera did you use on the quadcopter?
@randolfaquino9998
@randolfaquino9998 2 года назад
Can i inquire if a wider field of view camera improves degrees of obstacle avoidance? if so existing sensors can be fitted with wider lenses then?
@srinivasanarumugam4922
@srinivasanarumugam4922 4 года назад
Can you please post the hardware details?
@FabianCook
@FabianCook 5 лет назад
Curious as to what the thinks mounted at the top of the walls are, are they cameras being used to track environment state? I see there is a little ball on the drone, something to do with that?
@AlexeiBoiko
@AlexeiBoiko 5 лет назад
В мае 2019 года швейцарские исследователи из Цюрихского университета продемонстрировали миру оснащенный событийной камерой дрон, который уворачивался от мяча, летящего на скорости порядка 7.2 м/с. Событийная камера отличается тем, что ее кадры не возникают с присущей привычным камерам периодичностью, а появляются асинхронно, что и обеспечивает стремительную реакцию системы. Беспилотники становятся по-настоящему беспилотными: с одной стороны, подход приближает нас к формированию воздушного логистического комплекса - с другой, заставляет задуматься о том, что в ближайшие годы аналогичные аппараты появятся и у злоумышленников. Мысли о том, что аппарат, целеустремленно рвущийся к турбине заходящего на посадку пассажирского лайнера, сможет уклоняться от разнородных противодронных вооружений, вызывает понятные опасения. robotrends.ru/pub/1921/bespilotniki-shag-k-avtonomii-bla---uklonenie-ot-dvizhushihsya-obektov
@muhammedyilmaz2907
@muhammedyilmaz2907 4 года назад
if it can do that it can also increase this avoid stance
@pilotcommunity3156
@pilotcommunity3156 3 года назад
Beautiful !!
@xiaojake479
@xiaojake479 5 лет назад
Awesome! Congratulation!
@deanstanleynapawutut2842
@deanstanleynapawutut2842 2 года назад
How can you maintain and level the altitude of the drone?did you a sensor under to control the speed of the motor,altimeter?what? I really don't know thanks!!!!!
@riteshhangal6300
@riteshhangal6300 2 года назад
pls repeat the experiment with multiple balls from different directions in series, and throwing ball at same time from all direction
@yalmadiable
@yalmadiable 4 года назад
Question: The test is great but that precision only achieved indoor and using using many cameras to enable the drone to be very accurate in response but how would that be feasible in real world with only having drone sensors to work with? Isn’t that a waste of time
@alterngmail5963
@alterngmail5963 3 года назад
I'm a phd student working on cooperative control, recently could suggest some improvements to obstacle avoidance techniques, loved your experiments guys, good luck
Далее
Acoustic cameras can SEE sound
11:52
Просмотров 2,5 млн
🦊🎀
00:16
Просмотров 311 тыс.
More Bizarre Attempts at Perpetual Motion Machines
14:40
Insightness Collision Avoidance for Drones
4:25
Просмотров 56 тыс.
Experimental Drone Positioning System
17:36
Просмотров 209 тыс.
How Luxury Brands Get You to Buy Into their Hype
37:05
Просмотров 114 тыс.
What Everyone Gets Wrong About Planes
18:49
Просмотров 7 млн
AI Learns to Run Faster than Usain Bolt | World Record
10:22