Most likely not
is caused by a reactivation of dormant chicken pox virus
As long as you had chicken pox (or the vaccine), your baby will be fine because you have antibodies to the virus already and will fight it off. If you did not have chicken pox, then you are at risk, and so is your baby. If you are not sure, you should get a blood test to see if you do have antibodies or not.