How dangerous is it for a 3 week old baby to get the chicken pox?

Dangerous. It would mean mo never had chickenpox otherwise the baby should not get chicken pox or at least only mild. But at 3 weeks it can be life threatening. Talk with your pediatrician because there are medicines.
Not very. First of all, you need to be sure that it is in fact chicken pox. Most babies get antibodies against the virus from their mother while in the womb, so it's unusual for a baby to come down with chicken pox during the first year, which is why children are not vaccinated until they are 1 year old. Babies who do get it tend to have a mild case. Check with your doctor, though, and get a firm diagnosis.