I hit my ribs on a tree while sleigh riding. I went for an X-ray and it wasn't broken. But now, it is swollen under my left breast muscle and I feel weak. What should I do?

Chest wall injury. X-rays sometimes can miss rib fractures, however they usually do r/o lung injury or collapse. This type of injury typically takes weeks to heal since it is impossible to rest the chest wall muscles from a bruise. I would be applying local heat and taking otc pain meds. See your pcp if you continue not to improve.
Go Back To ER ASAP. What you could be describing could be as simple as blood vessel rupture around the area of the injury causing swelling around the rib muscles(intercostals) to a more serious condition such a splenic rupture. I am concerned about the possibility of the vessels aound the spleen rupturing if you feel weak. You would need emergency medical attention and possibly surgery immediately should that be it!
Unlikely dangerous. It sounds you had quite a blow. Rib pain is tough and can affect your depth of respiration. This perception is powerful and likely making u feel weak. It is unlikely you damaged an internal organ but the spleen is in that area. If you get sweaty, weak, clammy, get back to er. I'm sure they checked blood count. Don't forget, if they gave u some narcotics that may add to the effect. Good luck.