How can I overcome anxiety about going for sex therapy?

Make a call. It never hurts to make a phone call and see if you can have a short consult with the doctor before you schedule. Hopefully you'll be able to get a feeling as to whether you feel at ease with the doctor and can relate. Remember, if you are consulting a sex therapist they have plenty of experience talking about these things and they are not easily embarrassed or judgmental. We've seen/heard it all!
Stay active. Exercise is a great way to relieve stress. Your body releases endorphins during exercise, which can help you feel calm. If you exercise 30 - 60 minutes a day, your stress levels can improve. When stressed, take 5 slow, deep breaths with your eyes closed, then roll your shoulders forward 5 times, then back 5 times. This will slow your heart rate and release tension in your neck and shoulders.
It's NOT painful!! Good lord! did you expect a root canal? A good sex therapist will help you relax and talk about the history of your problem. The basic biology is so simple... But cultural issues and psychological "glitches" leading to guilt and shame, so often generate the problem! you are going to them to get help and it's unlikely that you are their first patient... So, relax! you deserve a satisfying sex life.
Logic. Just remember you will be talking to a professional, that is what we do, and we are never embarrassed about answering these questions.