What I do is build 2 seige towers and a battering ram. The enemy will concentrate their fire on the seige towers. Always make sure the gates are destroyed, especially the inner gate before commanding all of your men to attack
If you don't build 2 siege towers and you order them to attack even if you've only built 1 siege tower then their just going to all climb up the ladders and get shot and die. And that even if you'll broken through the gates