My take on it is as long as the sleeping bag is capable of trapping/using all your heat, the less clothes the better, as all they are doing is inhibiting the heat to transfer from you to the bag. If the bag is lightweight, or incapable of making efficient use of your heat, then the more clothes the better. Personally I sleep in my base layers and specific sleep socks, but that's as much for hygiene and trying not to dirty my bag. In the winter I put a down jacket on also to bolster my warmth.
I sleep cold tho' and can't ever remember waking up (other than in proper summer weather) and thinking I was ''too hot''