I've been feeling more tired and have some wrist swelling. When should I see the doctor?

Local or systemic? If you are having wrist swelling, that is not related to an injury, you should see a doctor to make sure that you do not have a systemic cause for inflammation. If it is related to an injury, and is painful, or not improving with a couple of days of rest and ice, you should have it looked at.
Cold? You may be experiencing joint pain associated with a common virus. If you swelling is getting worse by the day or not better after several days I would suggest a doctors visit. If you are having extreme pain/swelling that is worse with simple movements of your wrist then I would see a doctor sooner to make sure you don't have an infected wrist joint.