Are allergens in all foods, or just the ones that cause allergies?

Yes: Allergens are most commonly a protein although other molecules can sometimes be recognized as an allergen. So, yes most foods are allergens. An allergic reaction is caused by your immune system recognizing those unique proteins and causing a reaction. Luckily it is fairly rare, so other foods don't cause a reaction and you can eat them.

Both: Allergies are caused by specific proteins from specific foods, but any protein can trigger an allergy. Some proteins in foods are more commonly triggers for allergies, such as proteins found in peanuts or milk, but any protein in any food has the potential of triggering an allergy.