@1:42 I noticed that the robot moves down, stops and then uses the fingers to collect the object ( a serial operation ). Can it be improved, so that the fingers too closes in as the robot moves down, making it perform better
That's a very interesting question. I guess it would be very inaccurate in that case, unless it learned with the mirror in the environment. Them it might learn to recognize itself and figure out that there is a mirror. Theoretically it could also figure out the position of the mirror and use that to it's advantage. Another interesting question is what it would do with small glass walls that block the direct path to the target.
what about the releasing velocity and position, they are the final one of the end effector or not? I mean that when the robot meets the desired position and velocity, it continues his trajectory to reach zero velocity and then turn back to the initial position to grasp another object?
85% accuracy is good, but a pick and place robot that puts 15% of items in the wrong place doesn't sound too useful in a warehouse!? Either you could have a second slower robot that cleans up the misses while grumbling about TossingBot's untidiness, or maybe TossingBot could estimate its chance of success with each object and fall back to moving into position to lower risky objects into the container.
@1:21 you say the box is moved around by the robot as well to show case it's ability to throw to different target locations....but it looks to me that the box end is moving to meet the toss. ?
Tossing objects to a specific place is difficult as you need to adapt your tossing policy to lots of uncertainties like shape, weight, etc.. But it seems that you leveraged some visual servo techniques to adjust the position of the storage box. If so, then tossing will be easy because you just need to throw the object away and the box will figure out where to catch it.
The catching arm moves the box to where TossingBot _intends_ to throw each item, to show that TossingBot isn't limited to throwing to one location. As I understand it the catcher is not hooked up at all to the vision and neural network to adjust for bad throws. Though in an actual warehouse that could be useful for the 15% fail rate of TossingBot.
"Given an interest point on a 3D surface..." at t=0:57 is a bit hand-wavy. This research is excellent (and long awaited by researchers like me who only dabble in 3D vision), so I will be reading the paper, but can you elaborate? How are these points chosen? Also, you say you train a Siamese style network. For what purpose? Is one channel depth such as HHA/DHA and the other color?
