Do white women, black women, or brown women have a harder time with getting pregnant?

All the same. There are no known racial or ethnic differences in ability to get pregnant. Caucasian women are less likely to develop uterine fibroids but not all fibroids cause fertility issues or even need treatment. Socio-economic factors may matter more - how long women delay childbearing, prevalence of smoking, stds, obesity - these can all affect fertility.
Ethnic infertility. There is no evidence to suggest any racial or ethnic predisposition to infertility.