For drinking water then there is not much better than the 15lts Drinksafe gravity system:
http://www.drinksafe-systems.co.uk/products.php
Insulation is trapping a barrier of air so anything that does it to reasonable thickness and weight would do. Fire risk and arson being the real issues. Straw bail houses are warm and last as long as the wet doesn't get to the straw. Sphagnum moss collected and dried on mass is another one. The classic wood cabin used pretty thick trees with the wood itself being the main insulator. The trick is a good wood burning stove and burn plenty of fuel which will pump out plenty of heat; gets through a lot of wood though. Any stove needs a good flue pipe and correctly insulated from the wall as they get very hot.
Having tabbed over most of Wales in the military there is a lot of dead sheep and beasties in the becks and streams, loads of bracken spray and other farming chemicals too. Melt water has huge levels of bacteria which requires at least boiling. I would filter the water. Rain water collected is fine if fresh. The bigger the water tank the better as we all get through loads of water.
Rodents will gnaw through plastic in no time if they think there is food there. Steel bins are good just keep the bottoms off the dirt as the bottoms rust away pretty fast.
Loads of info out there for off the grid living.