Do you know which test you were given? There may be other reasons you may experience low vitamin D levels so it is wise to discuss the issue with your physician if you are concerned.
In the realm of IBD it is pretty common to note patients who have vitamin D deficiencies. Up to 68% of Crohn's patients have vitamin D deficiency according to the Crohn's and Colitis Foundation of America. This could be due to various causes.
One cause could be the lack of dietary sources due to restrictive diets implemented to lessen IBD symptoms. Another cause for low levels of vitamin D could be issues with absorption. This might be your problem since you report normal intake and sun exposure. Active inflammation in the small intestine or the use of corticosteroids can decrease the body's ability to absorb vitamin D (along with other vitamins and minerals). It may also be less easily absorbed than water soluble vitamins due to fat malabsorption.
It is good to hear that your calcium and bone density are normal. That does not mean that you don't need to worry about the vitamin D deficiency though. Over time, if not treated, it could lead to issues with your bones.
Your doctor was right to warn you that the body needs the vitamin D to utilize calcium properly. Has your doctor discussed the use of a supplement? Supplementation needs to be monitored by your physician to prevent causing hypercalcemia or toxicity. (Though this is usually seen only in patients given extremely high doses).
The National Institute of Health has a list of good food sources of vitamin D here.
There is a lot of research going into the role of vitamin D and autoimmune or inflammatory bowel disease (Here) and Health Central also posted an article recently about it's role in colorectal cancer. Obviously vitamin D is an important substance and getting a treatment plan to resolve your deficiency is something you need to discuss with your physician.