Remember that its important to distinguish between how fit you actually are, how fit you'd like to be (for whatever activities you do) and how fit you could be.
Most people who are unfit think of getting fit by doing several hours a day at the gym, or lots of running, or cycling or similar to build the ultimate body.
But then you have to ask what is it for? You could spend several hours a week to get fit, but then you'd have a nice body, and spend your life in a gym.
the balance is getting to a fitness level that keeps you healthy and makes your life easier whenever you are doing things that require a higher level of fitness.
Now I'm not the fittest person in the world by any means
but when I do try to keep fit its by doing things like flying kites, juggling, cycling to see places, walking, woodcarving and other fun hobbies that provide some form of exercise.
I think everyone would love to have that Mr Universe toned athletic body, but I think that keeping your fitness level significantly higher than your fitness 'needs' is just wasting good time in a gym when you coul dbe out having fun!