Should I be concerned that I was referred to a perinatologist?

High risk pregnancy. it is very likely that you were referred to a Perinatologist because of some high risk pregnancy issue. Be reassured that you are being referred to someone who specializes in high risk pregnancies. If you are high risk, you need to be managed by someone who has both experience and comfort in dealing with complicated pregnancies. .
Yes and No. If a general OB or family physician or midwife refers a patient to a maternal-fetal medicine specialist, there is an opportunity for a woman to learn more about the condition that caused concern and possibly to prevent or treat it. There is also a chance that the condition might be ruled out, ie, no longer a problem. One should never miss an appointment since referral was made to help baby or mom.
Depends why referred. There are many reasons why patients are referred to a perinatologist - referrals could be routine or for a serious complication. Obgyns are more comfortable with high risk pregnancy and do not refer as often. The most important thing is to understand why you are being referred. If this was not explained to you then please ask for clarification on the reason for the referral.