Why does my lazy eye stray when i look in a certain direction?

Strabismus: Often, amblyopia is caused my strabismus or eye misalignment. During eye movements the blurred eye may lag the better eye or even be crossed at certain directions of gaze. This is commonly detected.

Lazy eye: The dominant eye is doing all the focusing work. The lazy eye will wander because it is not being used as much for gathering visual information. Thus, when not is use, it will drift in another direction.