Only if you can dry it out, or keep the wind off you.
Ventile as an outer works because the water swells the cotton and the weave is so tight that it becomes showerproof and more importantly, windproof. It can take days to dry out though, and the fabric gets really stiff as it becomes soaked. It's still quiet and comfortable though.
Richard (rik uk) said he used to wear jeans all the time on the hills, but if it rained he put on waterproofs pdq.
I've worked in them just like that, but they get damp with sweat and they stay damp and they take ages to dry out in our damp climate. If I stopped moving then I became bitterly cold pdq
Time and place; cotton is comfortable, can be both very warm as an insulating layer or cool when woven light and airy. Old fashioned string vests worked because they had loads of ventilation as well as acting as insulation when covered up with other layers.
Cotton really gets used because it's cheap, and it's cheaper yet when mixed with polyester.