Concerns have been raised about plans for hundreds of homes on fields and what used to be part of the county’s police HQ.