Most men who are going to accumulate fat will do so with a preponderance of it showing on the waistline. Consider your self lucky if it is showing up in your handles instead of the gut... it is healthier to have it in the handles.

Although you will see only a very few people that advocate it (Schwarzenegger for one), most nutrition/exercise experts downplay spot reducing. I haven't had or seen much luck in spot reducing. Working your obliques (which lay below the handles) may actually widen the waist. Just stick to your diet, and get adequate exercise to metabloize fat. Calculate out your Base Metabolic rate (without football!) and eat accordingly. Fat will come off everywhere including the handles. Another thing you may want to try is a cutting gel(epidril) product on the handles. I haven't had success with them myself, but I have talked to others who swear that epidril works if incorporated into a good fat reduction diet.