Hard to say. It cannot be good for them, but we cannot quantify how bad from a practical standpoint. It is probably worse than with an adult.
Be prepared for pbs. Infants who are exposed to cigarette smoke have 2-3 x increase in respiratory infections like ear infections and bronchitis. There is also an increase in the crib death risk. Over time the kid grows up thinking it's great to smoke because the people they love most do, leading to another generation with related health issues.
Quit. If you smoke, set a quit date and stop. Ask your healthcare provider for help in quitting. If you cannot quit, do not smoke in the house or near children.