For me, it's probably the absence of human sounds I miss the most, being outside often isn't very quiet at all (lots and lots of corvids locally for example), but those sounds are just part of being outdoors. Like the sound of the sea breaking, or the wind through trees (or badgers grubbing about in the middle of the night doing badger stuff, no more than a few meters from your bivi)...
These sounds are not intrusive tho', unlike traffic rumble, idiot cellular phones, and just that general density of noise you get around lots of people.
Actually, now as I think about it, I don't like the homogenous and controlled environment of being in an urban area either, all tarmac, street lighting (!), inside everywhere strives to feel the same, the same temperature all year, nasty artificial and overbearing scents, horrid flat lighting, disconnection from the weather systems, grime, stickyness ... Oh, I could go on about this for hours (but I won't, as I'm sure you probably all feel the same way).