Coolmax or Merino base layer underneath. Norgies are cheap, good in dry, cold conditions and static activities, but they do soak up moisture.
I've started favouring wool jumpers (thin Merino, medium or thick wool blend dependent upon temperature) instead of fleece, although fleece would dry...