Is it possible to accidentally give yourself genital herpes from touching your lip with a just beginning/unseen sore and touching your genitals while using the restroom?

No, not possible. People are immune (or at least highly resistant) to new infections with the HSV type they already have. Your oral herpes almost certainly is HSV1, so you are at little or no risk for a new HSV1 infection, either from someone else or from yourself. Don't take chances; keep your hands away from your oral herpes outbreaks and wash hands before toilet. But little risk if you forget.