What could small blister like bumps that itch a little but don't hurt under my arm where my bra is be? They have been there a few days

Infection. This could represent infection with yeast, bacteria or even a cystic eruption called hidraadenitis. If it is infected itvwill require medication. See physician for diagnosis and treatment.