Does an allergic skin reaction have to be itchy?

Basically YES: Almost always an allergic dermatitis or hives is itchy.

No: Itching is a manifestation of allergy and is the most "classic" reaction. However, some will also experience redness and swelling as their main symptom. Burning to the skin can also occur.

Basically yes: Without treatment, which can sometimes eliminate the itch without changing the appearance of the skin, eczema, contact allergies, hives are all very itchy.