I was thinking somewhere higher up, the area near the farm and Nami's orchard. It's on a hill, but not as far off as the mountains.

I'm not really talking about a fortress. Just somewhere people can go if things go sour.