Should I worry about jaw pain that comes on when I'm climbing stairs?

Yes. Jaw pain is frequently experienced when a blockage in one of the arteries of the heart prevents adequate blood (and oxygen) flow to the heart, particularly during periods of exertion such as climbing stairs. If you are experiencing these symptoms, you should be seen by a physician as you may need testing to evaluate for the presence of heart disease.
Yes. Yes you should, it can be referred pain from your heart, you definitely need to check it.