Depends. Antibiotics are for infections. Keep taking the antibiotic unless told to stop. If the "ulcer" is caused by an infection, it should begin to resolve within 24 hours. If caused by irritation or by a systemic disease, you may need further treatment. Make sure to follow up with your dentist. Don't expect the problem to go away because you are taking an antibiotic. You must identify the cause.
Not necessarily. You need to see a dentist or physician to get a diagnosis. Don't delay, please.