Get them out
You can try to hide the problem with pain meds but it will come back.
Get to your dentist for consult and then treatment. You can take meds, use ice packs and i would rec you keep the areas as clean as you can with brushing and also warm salt water rinses. This may help to keep you comfortable until treatment is done.