Could low amniotic fluid in utero cause problems for the baby after delivery?

Potentially.: Persistent, chronic, severe oligohydramnios leads to pulmonary hypoplasia (small and maldeveloped lungs) that can prove lethal postnatally. Less extreme oligohydramnios is much more common and is typically a problem for labor - not postnatally if your obstetrician is competent.

Depends on cause: Babies don't need kidneys much during pregnancy because the placenta does that job.They do make pee that ends up as amniotic fluid. The right amount of fluid creates extra volume in the womb that pushes back against the muscular womb & allows the lungs to develop.(important) Things that stop or limit fluid formation includes absent kidneys or blockage in the outflow. Such defects can be a problem