How to tell if i have herpes as a baby and then not have it come back until years later?

Depends: Babies/ kids can get cold sores from the cold sore form of herpes and have it come and go over time.Babies that get herpes at or around birth from their mother will generally develop symptoms within the first weeks and months and often die in the process or suffer significant brain damage. If you have genital herpes as an adult, you likely got it through sexual contact.