I, too, have recently just started a new job after I left behind a comfortable position with a large company I love and people I generally liked.
This new position has me terrified. I took on a job as an restaurant manager, and even though I have roughly ten years prior experience of working in restaurants, any time I start something new, it's a somewhat terrifying experience.
It's always nice to read that other people are in the same situation and/or have felt the same way. There is always comfort in knowing you aren't the only person with these emotions.
People always say change isn't a bad thing, and for the most part, I agree. I'm only 24 however, and any change at this point in my life is a very scary time, especially when going at it alone.
Try and look at it from a long term perspective. I know when people say "two years from now", etc...but think about it. Time seems to be speeding up as we get older, and a few years will both fortunatly and unfortunatly be here sooner then we expect.
My advice is to first understand you are not the only one who feels like this. It's common, and although that might not neccesarily make it easier, understand that the way you feel is perfectly normal. Myself, and most other people who have replied on this thread have been in the same scenario.
Next, try and not focus on your last job. You left for a reason. Whether you were underpaid, under appreciated, or even terminated (assuming the latter isn't the case because you mentioned calling them), don't focus on it. The anxiety will pass, and assuming you wrote your initial thread two years ago, I'm hoping for my sake as well as yours that is has lol.
Let your new co-workers know you mean buisness, and that this isn't a a short term portion of your life. Most people fear change, and they see new employees as a threat. Don't let that bother you, as it's not your concern. You were hired by THEIR boss(es). That means they believe in you, and over time, your ability to contribute to their company. It wont happen overnight, and your boss(es) are very aware of that. You aren't their to make friends. Of course it's always a plus if you like the people you work with, and can eventually build a friendship. But that is down the road.
You do what you need to do, and focus on yourself. You have enough tasks, as does anyone starting a new job. The rest will work itself out. It always does! I promise.