Can Valium cause birth defects?

Yes. All medications in this class, benzodiazepines, are category d -- known to cause birth defects in humans.
Possibly. Studies in rats have shown that it can lead to cleft palate and other palatal abnormalities. Anti-anxiety meds like valium is category d meaning it is likely to cause birth defects. I would not recommend valium at any stage of pregnancy.