Nothing. If it is the cause then you need to stop using it.
Allergic Conjunctivi. If the cause of your allergic conjunctivitis is makeup than you have to discard what you were using and replace it with different based make up. You will have to check the ingredients in the one you were using. Yes replace it.
Dry eye vs allergy. Dry eye typically feels scratchy, irritated, like something is in your eye, light sensitive. Allergies in the eye typically causes itchiness in the eye.
Examination. Both conditions can cause the eyes to feel irritated. Allergies frequently have itching. Both can cause a goopy tear. Dry eye disease can make allergic conjunctivitis worse. They look very different and distinct on exam. Get an eye exam to get the correct diagnosis-- treatments are very different.
? Any of the 2nd generation less sedating antihistamines would do, can't tell which one would work in your case, but if it works, you need to take it regularly, read the labels carefully. You ultimately need to see your doctor / ophthalmologist as you mentioned dryness and redness too which can be something else other than just allergies, meanwhile use lubricant eye drops/artificial tears.