I have been killing the partrols. They seem to come in 3's. I am not worried about a Raid because my village has over 50+ Golems. I have only built 3 Golems, the rest just keep popping up. They're all on top of a hill, some go through the village. The Raids end quickly with so much Golem power. I feel safe when I leave my village, to travel.
The randomness of new biomes has really made it harder to find different ones. Been playing 2-3 hours daily for 2 months+, and have been 1k-2k in each direction, haven’t found a pure desert. I did find mushroom though. Haven’t wanted to use a seed finder map though.