Lots of great walking thereabouts. Cwmcarn forest drive, The Raven Walk, the Ebbw trail, The Sirhowy trail. The Monmouthshire & Brecon canal if you fancy something leisurely with historical interest. Not much in the way of lakes that way, aside from pen-y-fan pond, but two good rivers (Ebbw & Sirhowy). For caves if you head a little bit north to the heds of the valleys there are masses of caves, one of the longest cave systems in the UK. I'm not sure if there are any more locally to Cross Keys but I will consult the map for you.
PM me for more details if you like. I might even be able to meet you.
Dave53 lives very close by too, so he may be worth dropping a PM.