Wealthy year-round residents have complained of loud, late-night parties, overflowing trash cans and inadequate parking.