How can you get rid of shave bumps?

Stop shaving: It sounds as if you are developing a condition called pseudofolliculitis barbae which is inflammation of the hail follicle caused by shaving too close. Give then a rest for a week or two, then don't shave as close.

Shave bumps: Insure you use fresh, sharp blades vice dull blades. If hair is > several mm long – then trim before shaving. Wash area before shaving – warm water helps to soften hair. Use a lubricant when shaving. Avoid using products that contain alcohol or are scented. Shave in direction of the hair – not against it. Be gentle when shaving – don’t push down too hard. Moisturize skin after shaving.