How am I supposed to pick the right psychiatrist?

Do your homework. Ask your friends that you trust if they have suggestions. Is your family able to help you with that? Your primary care physician is a good resource generally. The national alliance for the mentally ill is also an excellent resource. The local medical society or psychiatric society can help. Yellow book advertisements are not a reliable source.Trusted family and friends are most often best sources.
Psychiatrists. You can generally check their credentials via their web sites & with state medical boards. Board certification is a positive. Consider what you're wanting help with, and find a psychiatrist interested in that or who's taken additional training in it. You want to feel comfortable confiding in the doctor, and feel that s/he hears you. Willingness to answer your questions is important also.