How much should I worry about shortness of breath?

No one answer here. Shortness of breath (dyspnea) can be a normal sensation for people who are doing new activity, or increasing the intensity of a familiar activity. Dyspnea is a difficult symptom to measure (unlike pain or fever) but like any symptom, if its worsening or acute, its best to see an md. You won't get a good answer online. Most patient questions related to symptoms need to be addressed face to face.