It seems whenever I do the "finger-to-nose" exercise only my lazy left eye moves. Is that weird?

Eye movements. Interesting! When you look to the left, right, up and down, do your eyes move the same way? If so, you may simply not be converging normally. If not, there could be something wrong with that muscle or the nerve to that muscle. You should see an ophthalmologist.