Mystery for me, it isn't Goat Willow, the bark is wrong. Your little tree looks more like Whitebeam. If it is, look after it, they look fantastic with their silver leaves. Whitebeam and Beech are my favourite trees.
The only thing holding me back from Whitebeam is the bark and growth pattern, it doesn't look right.
I found a Piittosporum up on the South Downs today, I spied it because it was wrong for the environment, probably why you saw yours. Got in close and found bunches of dead flowers at the base. The one I found mis probably a pet burial where the owners just don't understand the natural flora and fauna.