Is it possible for a man to have a yeast infection that he doesn't know about?

Yes he can. Infections by 'candida albicans' (more commonly, a yeast infection) can occur anywhere on a person where it is dark and moist, including mouth, vagina, and underneath large areas of sagging skin. While vaginal yeast infections are fairly common, persons with diabetes or with an immune deficiency of some sort are more prone to oral candidiasis. See a doctor for this condition, as it is treatable.