I heard that IgE is specific to peanuts. Is that true?

No. Of the 5 types of antibodies, ige is the one involved in allergy triggering allergic rhinitis, asthma & food allergies. Cat allergy occurs when your immune system makes ige against cat proteins in cat hair or dander. Peanut allergy occurs when you make ige against peanut proteins. Cashew allergy requires ige to cashew. Each specific ige antibody defines an allergy to the substance it recognizes.
Yes. If you are allergic to peanuts (or anything else), you have specific ige that recognizes the peanut protein (or the allergen that you are sensitive to). So an individual may have some ige that recognizes peanuts and some that recognize dust mites. A blood test for ige just measures the total ige level. Specific allergy testing measures how many of the total ige are for peanuts.