I definitely had to remove the A pillar panel. I think I was able to yank the old one out without remove the B pillar. The C pillar panel definitely was popped loose at the top but I didn't entirely remove it. I may be forced to remove the B and C pillar panels to reinstall the headliner, though.
