Can you get wandering-eye or cross-eye from being punched on the side of your face?

Yes. Damage the bones of the orbit surrounding the eye can injure the ye muscles or optic nerve. Proper ophthalmic evaluation is necessary to determine the cause.