Is it okay to eat cayenne chillies while pregnant?

You may. increase your chances of heartburn or reflux, but otherwise should be ok unless excessive.
Eating spicy food. should not hurt the baby. But pregnant women often suffer from heartburn more easily after eating spicy food. It is the woman's choice.