If it's not breathable, it will be like sleeping in a plastic bag.
Assuming you don't suffocate in your bivvy bag (if it's not
breathable, how can you breathe?), your exhaled breath will form as
condensation on the inside and will have nowhere to go.
Your clothing will be dripping wet come the morning. So, the less
breathable your bivvy bag, the more condensation it will generate on
the inside.
Bivvy bags are made from a slightly different type of Gore-Tex than
clothing. It is more breathable due to the fact that sleeping doesn't
generate much body heat and you need to breathe when you sleep!
As such, it is more prone to contamination than clothing, especially
as it will come into contact with the ground.
Gore-tex is basically a membrane with thousands of tiny holes. Your
sweat is evaporated water (like steam) which can pass through the
holes, but the holes are too big for water droplets. If all the holes become
blocked, your bivvy bag will become the equivalent of a large plastic bag.