Medication for White Coat Hypertension?

I have very high BP readings at doctor's visits, yet my blood pressure is normal to normal low at home.  This has been going on for quite some time.  My cholesterol is great, my triglycerides are great, so is my thyroid, and my cardiogram was good.  I am thin, don't drink, don't smoke, exercise an hour a day and have a good diet and no hypertension family history.  My doctor had prescribed for me a beta-blocker, saying it will help with my anxiety, but I am afraid to take it.  What if I take a beta blocker with normal blood pressure and heart rate?  Will I become faint, weak, dizzy, etc?  Will it slow my heart too much. My doctor tells me it will have no affect if my pressure is normal but unless I have more information, I can't bring myself to take it.