Can a man get yeast infection without having sex?

It is possible: Possible if his organ comes in contact with a infected item. He can also have it when using antibiotics.

Not so simple: You don't "catch" yeast. It is already there. The problem occurs with excessive proliferation of yeast often because of local conditions such as sugary urine in a diabetic or a loss of normal bacteria caused by an antibiotic.