Is it fair to say that if someone does develop signs of infection, that exposure occurred recently and not years ago?

It depends. Some infections can be latent only to become active years or decades later, such as herpes simplex and tuberculosis. Some, like chlamydia, can asymptomatically colonize the genital tract but cause symptoms in a contact.