Is there a such thing as an allergy to physical activity?

All people. To some extent all people with modest to moderate extrinisic inhalent allergies have some component of exercise induced asthma, manifest as a hacking dry cough 10-15m after stopping exercise. Other people with allergies can have exercise induced urticaria, or hives. Singular is excellent for most mild cases.
Possible. Depends on activity and your sensitivities.
Acutally...Yes. Some people will develop hives on their skin (itchy red bumps - similar to what you get with mosquito bites or with some other allergic reactions) in response to exercise. This can be mild, or sometimes severe, but shouldn't be confused with people who do "not like" to exercise. If it occurs, call your doctor for an appointment, or call 911 if it involves trouble breathing.