Properly constructed Canadian houses have double doors. What we call a "storm door," on the outside.
That covers the more massive, insulated core door within. All glass is at least doubled as well.
Downstairs, I've replaced all the weatherproof gaskets around the back door.
Insulated hollow walls of course and a thick layer of insulation just above the ceilings.
However, do too good a job, cutting off all air exchange and you get a "sick house".
I do agree though, plug all the really obvious air flow "leaks" as best you can.
If your home is draughty/drafty, don't let it stay that way, it costs you big bucks in lost energy.
I have a single, external 6" air feed to be heated by the central furnace.
That gets pushed throughout the house. Next, that air is pulled through return plenums
and used in combustion and flows up out the chimney. The make-up is the cold air intake.
"Temperate countries?" The swing here is enormous.
Shade air temperatures from -37C this January past to +47C,end of July.
The smart thing was to plant grape vines (2001) on the west side, 24" out from the wall.
They cover my entire west side now. 2013 was 30kg fruit per vine.
The August shade effect is 7-10C in the kitchen and master bedroom.
Electricity (all hydro) is relatively cheap here (sliding consumption scale) so I can afford to run
ceramic cube heaters to take the chill off a room I'm using. Great for rotten, shirtty dark days in January.