I think the answer is yes, but you don't need to have an anxiety-related condition to feel this way. It's certainly quite normal to prefer the company of people you are familiar with. If they feel comfortable with you they will no doubt have picked up the fact that sometimes you can be bothered more than others.

Solitude is a good thing for many people but I'm interested in the fact that you seem to prefer your own company. Perhaps this is just the type of person you are but I wonder if your question suggests a level of concern that its becoming just a little too easy to spend time by yourself? If you feel your level of anxiety is such that you find it stressful to be in company perhaps your treatment needs reviewing? If you feel that it's all too much effort and your interests tend to be home based and away from others you could be depressed. Unfortunately anxiety and depression are often associated.