Is there an otc medicine i can take for winter allergies?

Yes: I presume you must be referring to dry, itchy skin that occurs in winter also called the winter itch. It is not an allergy but caused by drying of the skin due to evaporation of skin oils. This is due to the low humidity and use of dry indoor heat. The treatment is regular application of a good skin moisturizer daily after bath and also using a humidifier to reduce dryness.

Yes: Loratadine, fexofenadine or cetirizine.