If I only have one kidney, should I be wearing something to protect it during contact sports?

Yes. Yes, depending on the type of contact sports. Contact sports that involve high velocity contact (football, some martial arts, boxing) need protection. Those with less of a chance of impact to the abdomen or back (basketball, soccer, cheer) should discuss it with their doctor. Those contact sports with low impact to the abdomen or back (wrestling, volleyball), it is not likely needed.
No. Your kidneys are well protected. Chances of losing your solitary kidney as a result of contact sports is minimal. Risk of broken neck in football, or severe concussion or brain damage from being hit by a baseball are much greater.