I missed three days of my birth control pills, if I start taking them again, will I have to wait a week before having unprotected sex?

Yes . Better safe than sorry. Three days is too long to simply restart and rely on the pill. If you have trouble remembering the pill consider an alternative method.