Great photos, we can really see the enigma now. Southey's idea of the tree changing/shedding bark did sound like a good one, but the fact there are no strips of bark on the ground, poses a problem, unless, as you suggest, someone took them away.............I don't know what to think now.
The hole looks more like an exit hole of a wood eating grub.
What is all that brown/reddish debris around the tree ?...it looks like shredded bark.