A devoted dad and grandad who was his partner's “whole world” has died after falling from a height while working as a scaffolder.