FLOCKING, How to get a velvet finish on any surface, demonstration 

it is quite thin, i would not recommend it.

i think it would come off easily especially with sweaty hands and constant rubbing

Probably depends more on the type of glue, as the reply mentioned the sweat and water from rain is going to be a huge factor in ruining it, so at the very least the adhesive used has to be water resistant.