Town centre offices described as “underutilised” are to be converted into flats after plans were approved despite an objection.