Can you tell me if you switch from glasses (prism lens) to contacts (different prescription) will that cause unusual discomfort?

It can. Contact lenses do not correct for ocular misalignment like prism glasses can. If you need prism to avoid discomfort, then contact lenses are not a good choice for you.