You really should be changing gloves and washing your hands way more frequently. Not changing your gloves in between tasks is just the same as not washing your hands, you're touching everything with those same dirty gloves. It's incredibly unsanitary.
