Katieannmary, This could be related to stopping breast feeding.  You might have a clogged or infected milk duct if the place is actually on the underside of your breast.  Or if it is not on your breast, but on your midriff below your breast, it might indeed be a boil.  I would suggest you call your doctor and describe the situation.  There is a good chance the doctor or whoever is on call will tell you to put hot compresses on the sore and call back if it doesn't get better, especially since the office is probably closed today.  If you have a fever, insist on being seen as soon as possible because a fever would indicate a more generalized infection.  Do not squeeze the area in any way.  This is unlikely to be cancer related, but if the problem persists even after antibiotics, ask for a referral to a breast specialist.  I hope you feel better soon.