It's not the flowers
Flowers bloom when other airborne plant pollen
, especially tree in the spring, is in the air. If a plant is colorful or has a pleasant odor, it is to attract insects for pollination.
Tree, grass, ragweed and other allergy causing pollen is wind borne causing symptoms in allergic people when they are exposed.