Could a 43 year old man have problems with fertility when he had fathered a child at 20?

Yes: Good healthy swimmers at 20 doesn't mean 2 decades of life in between didn't bring along a new variables that might impair fertility. Toxic exposures, infection, etc can drop fertility. The female involved may also contribute to success of failure of any parenting attempt for a variety of reasons. Assessment of the couple by a fertility team could give you some specific answers.

Yes: He may be exposed to other factors which affects his fertility, as infectious disorders, exposure to toxins or simply aging.