How To Stop Fireplace Creosote Smell (HowToLou.com) 

Even after you have your chimney swept, you can still get that smokey smell in your house. That's because the creosote particles have soaked into the porous flue material. It's not only common, but to be expected. The problem of the smell is negative pressure in the house causing the air to be sucked into the home from the chimney opening. This means your house will smell like the inside of your chimney. SOLUTION - you can do one of two things. First - would be to figure out the source of the negative pressure or alleviating it simply by giving the house another and better source from which to suck air - like opening a window or a door. Second - would be to block the flu like this guy did. BUT his solution is WAY over the top. A much easier solution is just to shove something up by the damper. A section of fluffy pink insulation would do nicely. Or you could use a panel of rigid insulation cut to size. Either way, just cut it a little larger than the opening so it snugs in and holds itself in place. Come Autumn, just reach in an pull it out and save it for next year.

Can't you just close the damper?