I missed three days of my birth control pills, if I start taking them again, will I have to wait a week before having unprotected sex?

Yes. Better safe than sorry. Three days is too long to simply restart and rely on the pill. If you have trouble remembering the pill consider an alternative method.