How does my diet affect my baby?

It depends. Some babies are very sensitive to what moms eat while breastfeeding, others are not. Some foods (dairy, soy, certain cruciferous vegetables) may cause gas or intestinal distress in some babies. Caffeine may overstimulate certain babies, as well. A healthy diet full of fresh fruits and vegetables, water, lean proteins and whole grains, can promote health in your baby.
Totally if pregnant. What a pregnant woman eats passes into her GI tract where nutrients are extracted. Even toxins are absorbed into her blood stream to some extent along with sugars, fats, etc. Much of it then goes through her circulation and is transferred by the placenta to the baby's circulation. So many of the molecules from the food she ate go to the baby. When pregnant or lactating, moms must be very careful.
Affects baby mildly. A breastfeeding mom's diet affects her breastmilk a little bit, and so only affects the baby mildly. A well-balanced diet is best for mom, but her breastmilk will still be nutritious and quite balanced even if she is low in some nutrients. If mom continues to eat poorly, eventually she will get weak, tired, and maybe ill . . . Which would then have a bad effect on her baby.