My 25 month old has little bumps all over the back of her arms and thighs. How can I treat this at home, or to at least make it smoother?

Moisturize/Exfoliate. Dry bumps on the thighs or backs of the arms are often keratosis pilaris (plugged follicles). Try brief baths followed by moisturizer. Products with Lactic Acid or urea help exfoliate. There are prescription meds if bad. But unless uncomfortable no treatment at all needed. Avoid harsh drying soaps. Avoid fragrances in skin products. Keratosis pilaris usually gradually goes away on its own.
Generalized bumps. I don't know if by little bumps you mean "wheals" from urticaria . Skin conditions have to be seen in person. This could be an allergic reaction and it may get worst so my suggestion take baby to the pediatrician or at least discuss it over the phone with his/ her office. Somebody has to see it.