Does pollen season make oral allergy syndrome worse despite taking antihistamines?

Yes it can. That's why you need to know the triggers well, try to avoid them as much as you can, ask your doctor about keeping an Epipen (epinephrine) handy, just in case a severe reaction develops, see an allergist if you can for complete assessment. Check or for an allergist in your area, good luck.