You don't mention the very small spring on the actual key holder. For some reason, mine fell out and had to get on the floor to find that spring. Did I mention it is very small.
No don't use a screwdriver. Use your key that's in the key fob there's a slot right above where the key goes you just put the key in there turn the key and it pops it open. It also shows how to do it in the owner's manual. If you do it that way with a screwdriver there's a possibility of messing up the seams on the key fob.