My boyfriend's dirty finger grazed my inner labia, and now it's swollen and itching, what's going on?

Depends on the cause. An infection is possible. However, there are many more options. Sex that is too dry, too long, too rough, penis too big, vagina too small, sex under water, infections, jock itch, allergy to condoms/spermicide etc. If the problem does not resolve, it is time to call your family doctor.