Is it safe and okay to take Plan B if I had unprotected sex, being that 4 weeks ago I delivered a baby?

Yes . Plan b works best when taken within 72 hours of unprotected sex. If you are in that window then yes you should take it. Use condoms or now on and schedule a visit to discuss your birth control options.
Yes. If your are not breastfeeding, you could have ovulated as soon as 2 to 4 weeks after delivery. It is reasonable to use Plan B for protection as you have no good way of knowing if you have ovulated or if you are about to ovulate. It would also be appropriate to consider ongoing contraception.