A major Labor figure has broken ranks from the Albanese government, as concerns grow about inflation.