It can affect both
Your milk flow &, more importantly, your emotional availability to your toddler. If you wish, you can wean to an open-mouthed, doubled handled cup, limiting cow's milk & milk products to 16-21 oz.
/day. Like airline attendants say, put on your own oxygen mask
before helping your child. You'll benefit from medication, cognitive-behavioral therapy, & speaking openly to your husband about this.