Yeah just a little bit to hold the rivet in place, and a thumb against the aluminum to prevent the tool from marring the finish when it bucks from the pin snapping.
What is the thing or surface behind the skin that you're riveting it to? When you drill the old rivet out, does is just free fall behind the skin until it hits a floor surface of some kind?
The old rivet falls into the deep beyond between the interior and exterior skin. That void is filled with a fuzzy insulation made from recycled plastic bottles so it doesn't probably fall too far. As for what I'm riveting to, the airstream panels overlap but sometimes it's hard to see in the video - so we're binding two panels together and inside the skin the rivet flares out somewhat like a mushroom to pinch the two panels between the head and the flare.