Yes and No
Sometimes yes and sometimes no.
Some over the counter medications are not safe for all babies. It is wise to have a discussion with your doctor about which medications you can use for some common problems, and write down the advice. Recheck with followup visits.