Can you get herpes from touching a handle in a public bathroom and then that hand touches your genitals shortly after while wiping?

No worries: Brief, fleeting contact is never enough to transmit herpes. And there never would be enough virus on a handle or door knob anyway. Herpes is NEVER transmitted by contact with contaminated objects. Even in the households of infected people, sharing kitchens, bathrooms, drinking glasses, towels etc, nobody else ever gets it except by kissing or sexual exposure. So no worries at all!