Every time I wear a "cheap" necklace I get itchy bumps on the back of my neck. How do I make them go away?

Stop wearing it. Cortisone cream will help the reaction heal. Coat the necklace and any other nickel-containing materials you wear with clear nail polish to prevent your skin (immune system) from coming in contact with nickel.
Other prevention tip. Look for jewelry that's "hypoallergenic" or marked nickel free. And if you have something you love, can't part with but it's bumpy-making, then agree with dr. S's clear fingernail polish diy tip.
Don't wear nickel. Nickel is used as a hardener in metals such as jewelry. Avoids costume jewelry to prevent rash and use Hydrocortisone cream to relieve the current rash.