Very Likely. However, you may eat any product that does not use yeast in the making of it. That would include such breads as tortilla, pita, flatbread, biscuits, cornbread.
Sure it's allergy? Be sure that you truly have an allergy before you try to avoid yeast. Best bet is to see a board certified allergist for evaluation. If you are truly allergic to yeast, you should probably avoid breads, wine and beer that contain yeast (brewer's or baker's).