My nipples have been really tender and sore the past two days. Should I be worried?

Depends. Often time it is concerning when there is pain associated with nipple discharge such as infection, malignancy, normal postpartum reaction relate to breastfeeding, monthly menstruation, etc. Nipple tenderness without discharge can often be benign but if symptom persist or worsening you should see a local OB doctor for further evaluation.