I have a Weihrauch hw57 - a rather heavy but relatively compact .22 full power under lever air rifle. I think its magic! A nice quality scope on it (although it comes with iron sights that are fun for plinking), and it has no trouble with birds, rabbits, squirrels etc.
At times when in a suitable location- i.e. a friendly campsite, then with a few friends we do a bit of plinking (with wood chips variously ranged about the wood out to about 30/40m). I also occasionally head out to a nice little field surrounded by thick woodland where i know there are a few rabbits. Get there early morning, and there is no one about at all, or slightly more risky, late on a summers evening. When staying in rural holiday cottages/converted barns etc. with the family, it gets chucked in the boot of the car in case the opportunity presents itself (a wood pigeon taken through a partially wound down window in a car park this summer). I also take it on Scout Camps to provide a bit of variety to the scouts diet, and teach an important lesson too.
To be fair, it doesn't get a massive amount of regular use, but the rifle i have is of sufficient quality that it will last me for many years, and when i do use it, it gives me a lot of pleasure.
All i would say is that if you get a rifle, practice with it a lot first so you are sure of your abilities, and don't attempt to take game with it where you aren't largely confident you will get a clean kill. To do otherwise is rather cruel.
If you are sensibly minded, then you will join an organisation like BASC
http://www.basc.org.uk/ and approach local land owners for permission to shoot vermin like rabbits/pigeons etc on their land. In an ideal world they should say yes (as you will be doing them a service). In reality many will say no (no doubt worried about liabilities, the idea of someone they don't know running around their farm with a blaster etc). But keep bashing away at it, and you will get a few locations. Its polite to offer them some of your catch, and come to some arrangement with them to notify them each time you propose to visit. Don't dress like you have just come back from a trip whacking Viet Cong in the Mekong Delta when you go and knock on doors asking for permission!
Although there is an argument for both calibres, as a generalisation, it is often accepted that .22 is the one you want for most small game- and would probably give you most flexibility at least. .177 for birds mainly and target shooting (due to a flatter trajectory). If you want something that will last you, will be accurate and reliable- go for a make like BSA, Weihrauch etc, and stay away from the cheaper rifles of Chinese origin. Otherwise you will quickly reach their limitations, and possibly just give up on the whole idea rather earlier than you would otherwise. The legal limit without holding a firearms certificate in the UK is 12 ft lbs. You will likely want to get a rifle on or close to this limit if you intend to shoot animals with it.
As is mentioned above, a good padded bag is a must- it will give you more confidence to chuck the rifle into the boot of a car on the off chance you might get to use it on whatever trip you are off on.
Hope this helps give you an idea, and maybe kindle a bit of interest.