If I am going to the bathroom so much, does it have to be a UTI or can a vaginal yeast infection cause that?

Many things. Frequency of urination can be a manifestation of many medical conditions. Vaginal yeast infections will seldom do that without associated burning on urination. See a doctor.