Razor bumps will usually go away when you stop shaving.
There are a few things you can do to prevent them in the future. Shave in the shower so the heat can soften the hair, and use a shaving gel. Always use a brand new razor, and shave in the direction that your hair grows. Moisturize after shaving. Instead of shaving, you can also use a depilatory not more than once a week.