Work is finally underway to transform a neglected town centre site once overrun with pigeons and branded an “unhygienic health hazard”.