There was not so much "back and forth" with Louisbourg. The New Englanders took it in 1745 much to the surprise of the British and a British expedition took it again in 1758. Halifax was established in 1749 just after Britain handed île- Royale back to the French. Britain had a garrison at Fort Anne, Annapolis Royal since 1710. There were indeed many freed slaves (Black Loyalist) but there were also slaves brought in by the White Loyalist. The earlier wave of New England Planters in the 1760 had also brought slaves.